Method and apparatus for joint source-channel coding based on two-step feature selection for semantic communication

Abstract

A method according to an embodiment of the present specification comprises the steps of: determining K1 latent components on the basis of the square of a standard deviation related to each latent component among latent components based on a source; determining K2 latent components on the basis of the difference between each latent component among the K1 latent components and an average related to the corresponding latent component; and transmitting a signal generated on the basis of the K2 latent components.

Description

Two-stage feature selection-based joint source channel coding method and apparatus for semantic communication

[0001] This specification relates to a two-stage feature selection-based joint source channel encoding method and apparatus for semantic communication.

[0002] Mobile communication systems were developed to provide voice services while ensuring user mobility. However, mobile communication systems have expanded their scope to include data services as well as voice. Currently, due to the explosive increase in traffic leading to resource shortages and users demanding higher-speed services, more advanced mobile communication systems are required.

[0003] The requirements for next-generation mobile communication systems largely include the ability to accommodate explosive data traffic, a dramatic increase in transmission rates per user, a significantly increased number of connected devices, very low end-to-end latency, and high energy efficiency. To achieve this, various technologies are being researched, such as dual connectivity, massive multiple input multiple output (MMIMO), in-band full duplex, non-orthogonal multiple access (NOMA), super wideband support, and device networking.

[0004] Meanwhile, existing methods for joint source-channel coding (JSCC) have the following problems.

[0005] When the target number of channel uses or source recovery performance is fixed, existing artificial neural network-based technologies must retrain a new model to obtain one optimized for the constraints. In other words, if unoptimized constraints are applied to a previously trained model, the model's performance drops sharply. Furthermore, conventional technologies experience a rapid decline in performance regarding source recovery and channel usage when a channel state (e.g., SNR) different from that assumed during training is provided.

[0006] Furthermore, conventional methods utilize neural networks composed of a massive number of parameters, which can lead to an excessive increase in the signaling overhead required to direct information related to those parameters. Specifically, the signaling overhead required to direct the latent components selected for co-source channel coding from among the total latent components can be excessively large.

[0007] The purpose of this specification is to propose a method for common source channel coding that is universally applicable to the target source restoration and channel usage counts.

[0008] Another objective of this specification is to propose a method for reducing the signaling overhead required to indicate information about selected potential components for common source channel coding.

[0009] The technical problems to be solved in this specification are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this specification belongs from the description below.

[0010] A method according to one embodiment of the present specification includes the steps of determining K1 potential components based on the square of the standard deviation associated with each potential component among the potential components based on a source, determining K2 potential components based on the difference value between each potential component among the K1 potential components and the mean associated with the corresponding potential component, and transmitting a signal generated based on the K2 potential components.

[0011] The above signal can be generated based on power determined based on i) the difference between each latent component and the mean of the corresponding latent component and ii) the square of the standard deviation of the corresponding latent component.

[0012] The above method may further include a step of transmitting index information. The index information may represent the K2 potential components among the K1 potential components.

[0013] The above index information may be based on a sequence compressed based on run-length encoding.

[0014] The above Source can be transformed into the above latent components based on Non-linear Transformation Coding (NTC).

[0015] The above latent components may be based on the output of a first nonlinear transformation function that takes the above Source as input.

[0016] The mean and standard deviation associated with each latent component can be determined based on distribution information associated with the said latent components.

[0017] The above distribution information may be based on the output of a second non-linear transformation function that takes each of the above latent components as input.

[0018] The above method further includes the step of transmitting distribution information related to the above potential components.

[0019] The above signal is transmitted based on the first transmission power, and the distribution information can be transmitted based on the second transmission power.

[0020] The first transmission power and the second transmission power may be based on the output of a neural network. The input of the neural network may include i) the latent components, ii) the distribution information, and iii) the mean and standard deviation associated with each latent component.

[0021] A wireless device according to another embodiment of the present specification includes one or more transceivers, one or more processors, and one or more memories connected to the one or more processors and storing instructions.

[0022] The above instructions are characterized by setting the one or more processors to perform all steps of any one of the above methods based on execution by the one or more processors.

[0023] An apparatus according to another embodiment of the present specification includes one or more memories and one or more processors connected to the one or more memories.

[0024] The one or more memories are characterized by storing instructions that set the one or more processors to perform all steps of any one of the methods based on execution by the one or more processors.

[0025] One or more non-transitory computer-readable media according to another embodiment of the present specification store instructions. The instructions, executable by one or more processors, are characterized by setting the one or more processors to perform all steps of any one of the methods.

[0026] According to an embodiment of the present specification, potential components are selected in two steps based on defined criteria. Specifically, K1 potential components are determined based on the square of the standard deviation associated with each potential component, and K2 potential components are determined based on the difference between each potential component among the K1 potential components and the mean associated with that potential component.

[0027] As described above, since latent components are determined based on criteria defined considering restoration performance, versatility can be increased compared to existing methods. Specifically, when only a single artificial neural network structure framework can be used, higher performance can be provided in terms of PSNR-CBR compared to existing transmit / receive frameworks. Furthermore, when the channel state applied during the transmit / receive process changes, improved performance can be provided compared to existing transmit / receive frameworks.

[0028] In addition, according to the embodiments of this specification, latent components for joint source-channel coding are determined from among K1 latent components, which are part of the total latent components. Therefore, the signaling overhead required to indicate the determined latent components can be significantly reduced compared to conventional methods.

[0029] The effects obtainable in this specification are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which this specification belongs from the description below.

[0030] The drawings attached below are intended to aid in understanding the present specification and may provide embodiments of the present specification along with detailed descriptions. However, the technical features of the present specification are not limited to specific drawings, and features disclosed in each drawing may be combined with one another to form new embodiments. Reference numerals in each drawing may denote structural elements.

[0031] FIG. 1 is a drawing showing an example of a communication system applicable to the present specification.

[0032] FIG. 2 is a drawing showing an example of a wireless device applicable to the present specification.

[0033] FIG. 3 is a diagram illustrating a method for processing a transmission signal applicable to the present specification.

[0034] FIG. 4 is a drawing showing another example of a wireless device applicable to the present specification.

[0035] FIG. 5 is a drawing showing an example of a portable device applicable to the present specification.

[0036] FIG. 6 is a diagram showing physical channels applicable to the present specification and a signal transmission method using them.

[0037] Figure 7 is a figure showing an example of a perceptron structure.

[0038] Figure 8 shows an example of a multilayer perceptron structure.

[0039] Figure 9 is a figure showing an example of a deep neural network.

[0040] Figure 10 is a figure showing an example of a convolutional neural network.

[0041] Figure 11 is a figure showing an example of a filter operation in a convolutional neural network.

[0042] Figure 12 shows an example of a neural network structure in which a circular loop exists.

[0043] Figure 13 shows an example of the operational structure of a recurrent neural network.

[0044] Figure 14 illustrates a common source channel coding based on the existing method.

[0045] FIG. 15 illustrates a common source channel coding according to an embodiment of the present specification.

[0046] Figure 16 is a graph showing the degree of distortion of information restored based on Top K selection.

[0047] Figure 17 illustrates potential components related to two-step selection.

[0048] FIG. 18 illustrates a common source channel coding according to another embodiment of the present specification.

[0049] Figure 19 is a graph showing PSNR performance according to CBR.

[0050] Figure 20 is a graph showing PSNR performance according to SNR.

[0051] FIG. 21 is a flowchart for illustrating a method according to one embodiment of the present specification.

[0052] The following embodiments are combinations of the components and features of this specification in a specific form. Each component or feature may be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, some components and / or features may be combined to constitute the embodiments of this specification. The order of operations described in the embodiments of this specification may be changed. Some components or features of any embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment.

[0053] In the description of the drawings, procedures or steps that could obscure the gist of the specification have not been described, nor have procedures or steps that are understandable to those skilled in the art been described.

[0054] Throughout the specification, when a part is described as "comprising" or "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part," "...unit," and "module" as used in the specification refer to a unit that performs at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software. Additionally, "one (a or an)," "one," "the," and similar related terms may be used in the context describing this specification (particularly in the context of the following claims) to include both singular and plural forms, unless otherwise indicated in this specification or clearly contradicted by the context.

[0055] The embodiments of this specification have been described with a focus on the data transmission and reception relationship between a base station and a mobile station. Here, the base station refers to a terminal node of a network that communicates directly with a mobile station. Specific operations described herein as being performed by a base station may, in some cases, be performed by an upper node of the base station.

[0056] That is, in a network consisting of multiple network nodes including a base station, various operations performed for communication with a mobile station may be performed by the base station or other network nodes other than the base station. In this case, 'base station' may be replaced by terms such as fixed station, Node B, eNB (eNode B), gNB (gNode B), ng-eNB, advanced base station (ABS), or access point.

[0057] Additionally, in the embodiments of this specification, the term terminal may be replaced with terms such as user equipment (UE), mobile station (MS), subscriber station (SS), mobile subscriber station (MSS), mobile terminal, or advanced mobile station (AMS).

[0058] Furthermore, the transmitting end refers to a fixed and / or mobile node that provides data or voice services, and the receiving end refers to a fixed and / or mobile node that receives data or voice services. Therefore, in the case of the uplink, a mobile station can be the transmitting end and a base station can be the receiving end. Similarly, in the case of the downlink, a mobile station can be the receiving end and a base station can be the transmitting end.

[0059] The embodiments of this specification may be supported by standard documents disclosed in at least one of the wireless access systems, such as IEEE 802.xx systems, 3GPP (3rd Generation Partnership Project) systems, 3GPP LTE (Long Term Evolution) systems, 3GPP 5G (5th generation) NR (New Radio) systems and 3GPP2 systems, and in particular, the embodiments of this specification may be supported by the documents 3GPP TS (technical specification) 38.211, 3GPP TS 38.212, 3GPP TS 38.213, 3GPP TS 38.321 and 3GPP TS 38.331.

[0060] In addition, the embodiments of this specification may be applied to other wireless access systems and are not limited to the systems described above. For example, they may be applicable to systems applied after the 3GPP 5G NR system and are not limited to specific systems.

[0061] That is, obvious steps or parts not described in the embodiments of this specification may be described by referring to the aforementioned documents. Additionally, all terms disclosed in this specification may be explained by the aforementioned standard documents.

[0062] Hereinafter, preferred embodiments according to the present specification will be described in detail with reference to the accompanying drawings. The detailed description disclosed below, together with the accompanying drawings, is intended to describe exemplary embodiments of the present specification and is not intended to represent the only embodiment in which the technical configuration of the present specification can be implemented.

[0063] Additionally, specific terms used in the embodiments of this specification are provided to aid in understanding this specification, and the use of such specific terms may be modified in other forms without departing from the technical spirit of this specification.

[0064] The following technology can be applied to various wireless access systems such as CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), and SC-FDMA (single carrier frequency division multiple access).

[0065] For the sake of clarity in the following description, the explanation is based on 3GPP communication systems (e.g., LTE, NR, etc.), but the technical concept of the present invention is not limited thereto. LTE may refer to technology from 3GPP TS 36.xxx Release 8 onwards. Specifically, LTE technology from 3GPP TS 36.xxx Release 10 onwards is referred to as LTE-A, and LTE technology from 3GPP TS 36.xxx Release 13 onwards may be referred to as LTE-A pro. 3GPP NR may refer to technology from TS 38.xxx Release 15 onwards. 3GPP 6G may refer to technology from TS Release 17 and / or Release 18 onwards. "xxx" indicates a specific standard document number. LTE / NR / 6G may be collectively referred to as 3GPP systems.

[0066] Regarding the background technology, terms, abbreviations, etc. used in this specification, reference may be made to matters described in standard documents published prior to the present invention. For example, reference may be made to standard documents 36.xxx and 38.xxx.

[0067] Communication systems applicable to the present specification

[0068] Although not limited thereto, the various descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this specification may be applied to various fields requiring wireless communication / connection (e.g., 5G) between devices.

[0069] Examples are provided in more detail below with reference to the drawings. In the following drawings and descriptions, the same reference numerals may represent the same or corresponding hardware blocks, software blocks, or function blocks unless otherwise described.

[0070] FIG. 1 is a drawing illustrating an example of a communication system to which the present specification applies. Referring to FIG. 1, the communication system (100) to which the present specification applies includes a wireless device, a base station, and a network. Here, a wireless device refers to a device that performs communication using wireless access technology (e.g., 5G NR, LTE) and may be referred to as a communication / wireless / 5G device. Although not limited thereto, a wireless device may include a robot (100a), a vehicle (100b-1, 100b-2), an XR (extended reality) device (100c), a hand-held device (100d), a home appliance (100e), an IoT (Internet of Thing) device (100f), and an AI (artificial intelligence) device / server (100g). For example, a vehicle may include a vehicle equipped with wireless communication capabilities, an autonomous vehicle, a vehicle capable of performing inter-vehicle communication, etc. Here, the vehicle (100b-1, 100b-2) may include an unmanned aerial vehicle (UAV) (e.g., a drone). The XR device (100c) includes an augmented reality (AR) / virtual reality (VR) / mixed reality (MR) device and may be implemented in the form of a head-mounted device (HMD), a head-up display (HUD) equipped in a vehicle, a television, a smartphone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a robot, etc. The portable device (100d) may include a smartphone, a smart pad, a wearable device (e.g., a smartwatch, smart glasses), a computer (e.g., a laptop, etc.). The home appliance (100e) may include a TV, a refrigerator, a washing machine, etc. The IoT device (100f) may include a sensor, a smart meter, etc.For example, the base station (120) and network (130) may also be implemented as wireless devices, and a specific wireless device (120a) may act as a base station / network node for other wireless devices.

[0071] Wireless devices (100a to 100f) can be connected to a network (130) through a base station (120). AI technology may be applied to the wireless devices (100a to 100f), and the wireless devices (100a to 100f) can be connected to an AI server (100g) through the network (130). The network (130) can be configured using a 3G network, a 4G (e.g., LTE) network, or a 5G (e.g., NR) network. The wireless devices (100a to 100f) may communicate with each other through the base station (120) / network (130), but they may also communicate directly (e.g., sidelink communication) without going through the base station (120) / network (130). For example, vehicles (100b-1, 100b-2) can communicate directly (e.g., V2V (vehicle to vehicle) / V2X (vehicle to everything) communication). Also, IoT devices (100f) (e.g., sensors) can communicate directly with other IoT devices (e.g., sensors) or other wireless devices (100a to 100f).

[0072] Wireless communication / connection (150a, 150b, 150c) can be established between wireless devices (100a~100f) / base station (120) and between base station (120) / base station (120). Here, wireless communication / connection can be established through various wireless access technologies (e.g., 5G NR), such as uplink / downlink communication (150a), sidelink communication (150b) (or D2D communication), and communication between base stations (150c) (e.g., relay, IAB (integrated access backhaul)). Through wireless communication / connection (150a, 150b, 150c), wireless devices and base stations / wireless devices, and base stations and base stations can transmit / receive wireless signals to / from each other. For example, wireless communication / connection (150a, 150b, 150c) can transmit / receive signals through various physical channels. To this end, based on the various proposals of this specification, at least some of the following may be performed: a process for setting various configuration information for transmitting / receiving wireless signals, a process for various signal processing (e.g., channel encoding / decoding, modulation / demodulation, resource mapping / demapping, etc.), a resource allocation process, etc.

[0073] Communication systems applicable to the present specification

[0074] FIG. 2 is a drawing illustrating an example of a wireless device that can be applied to the present specification.

[0075] Referring to FIG. 2, the first wireless device (200a) and the second wireless device (200b) can transmit and receive wireless signals through various wireless access technologies (e.g., LTE, NR). Here, {the first wireless device (200a), the second wireless device (200b)} may correspond to {the wireless device (100x), the base station (120)} and / or {the wireless device (100x), the wireless device (100x)} of FIG. 1.

[0076] The first wireless device (200a) includes one or more processors (202a) and one or more memories (204a), and may additionally include one or more transceivers (206a) and / or one or more antennas (208a). The processor (202a) controls the memory (204a) and / or transceivers (206a) and may be configured to implement the descriptions, functions, procedures, proposals, methods and / or sequences of operation disclosed herein. For example, the processor (202a) may process information within the memory (204a) to generate a first information / signal and then transmit a wireless signal containing the first information / signal through the transceiver (206a). Additionally, the processor (202a) may receive a wireless signal containing a second information / signal through the transceiver (206a) and then store information obtained from the signal processing of the second information / signal in the memory (204a). Memory (204a) may be connected to the processor (202a) and may store various information related to the operation of the processor (202a). For example, memory (204a) may store software code including instructions for performing some or all of the processes controlled by the processor (202a) or for performing the descriptions, functions, procedures, proposals, methods, and / or sequences of operation disclosed in this specification. Here, the processor (202a) and memory (204a) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE, NR). A transceiver (206a) may be connected to the processor (202a) and may transmit and / or receive wireless signals through one or more antennas (208a). The transceiver (206a) may include a transmitter and / or receiver. The transceiver (206a) may be combined with an RF (radio frequency) unit. In this specification, a wireless device may refer to a communication modem / circuit / chip.

[0077] The second wireless device (200b) includes one or more processors (202b) and one or more memories (204b), and may additionally include one or more transceivers (206b) and / or one or more antennas (208b). The processor (202b) controls the memory (204b) and / or transceivers (206b) and may be configured to implement the descriptions, functions, procedures, proposals, methods and / or sequences of operation disclosed herein. For example, the processor (202b) may process information within the memory (204b) to generate a third information / signal and then transmit a wireless signal containing the third information / signal through the transceiver (206b). Additionally, the processor (202b) may receive a wireless signal containing a fourth information / signal through the transceiver (206b) and then store information obtained from the signal processing of the fourth information / signal in the memory (204b). The memory (204b) may be connected to the processor (202b) and may store various information related to the operation of the processor (202b). For example, the memory (204b) may store software code including instructions for performing some or all of the processes controlled by the processor (202b) or for performing the descriptions, functions, procedures, proposals, methods, and / or sequence diagrams of operation disclosed in this specification. Here, the processor (202b) and the memory (204b) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE, NR). The transceiver (206b) may be connected to the processor (202b) and may transmit and / or receive wireless signals through one or more antennas (208b). The transceiver (206b) may include a transmitter and / or receiver. The transceiver (206b) may be used in combination with an RF unit. In this specification, a wireless device may refer to a communication modem / circuit / chip.

[0078] Hereinafter, hardware elements of the wireless device (200a, 200b) will be described in more detail. Although not limited thereto, one or more protocol layers may be implemented by one or more processors (202a, 202b). For example, one or more processors (202a, 202b) may implement one or more layers (e.g., functional layers such as PHY (physical), MAC (media access control), RLC (radio link control), PDCP (packet data convergence protocol), RRC (radio resource control), and SDAP (service data adaptation protocol). One or more processors (202a, 202b) may generate one or more PDUs (Protocol Data Units) and / or one or more SDUs (service data units) according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed herein. One or more processors (202a, 202b) may generate messages, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this specification. One or more processors (202a, 202b) may generate a signal (e.g., baseband signal) including a PDU, SDU, message, control information, data, or information according to the functions, procedures, proposals, and / or methods disclosed in this specification and provide it to one or more transceivers (206a, 206b). One or more processors (202a, 202b) may receive a signal (e.g., baseband signal) from one or more transceivers (206a, 206b) and may obtain a PDU, SDU, message, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this specification.

[0079] One or more processors (202a, 202b) may be referred to as a controller, microcontroller, microprocessor, or microcomputer. One or more processors (202a, 202b) may be implemented by hardware, firmware, software, or a combination thereof. For example, one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), one or more digital signal processing devices (DSPDs), one or more programmable logic devices (PLDs), or one or more field programmable gate arrays (FPGAs) may be included in one or more processors (202a, 202b). Descriptions, functions, procedures, proposals, methods, and / or flowcharts disclosed herein may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, etc. Firmware or software configured to perform the descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this specification may be included in one or more processors (202a, 202b) or stored in one or more memories (204a, 204b) and driven by one or more processors (202a, 202b). The descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this specification may be implemented using firmware or software in the form of code, instructions, and / or sets of instructions.

[0080] One or more memories (204a, 204b) may be connected to one or more processors (202a, 202b) and may store various forms of data, signals, messages, information, programs, codes, instructions, and / or commands. One or more memories (204a, 204b) may be composed of ROM (read-only memory), RAM (random access memory), EPROM (erasable programmable read-only memory), flash memory, hard drives, registers, cache memory, computer-readable storage media, and / or combinations thereof. One or more memories (204a, 204b) may be located inside and / or outside of one or more processors (202a, 202b). Additionally, one or more memories (204a, 204b) may be connected to one or more processors (202a, 202b) through various technologies such as wired or wireless connections.

[0081] One or more transceivers (206a, 206b) may transmit user data, control information, wireless signals / channels, etc., as mentioned in the methods and / or operation flowcharts, etc., of this specification to one or more other devices. One or more transceivers (206a, 206b) may receive user data, control information, wireless signals / channels, etc., as mentioned in the descriptions, functions, procedures, proposals, methods and / or operation flowcharts, etc., disclosed in this specification from one or more other devices. For example, one or more transceivers (206a, 206b) may be connected to one or more processors (202a, 202b) and may transmit and receive wireless signals. For example, one or more processors (202a, 202b) may control one or more transceivers (206a, 206b) to transmit user data, control information, or wireless signals to one or more other devices. Additionally, one or more processors (202a, 202b) may control one or more transceivers (206a, 206b) to receive user data, control information, or wireless signals from one or more other devices. Additionally, one or more transceivers (206a, 206b) may be connected to one or more antennas (208a, 208b), and one or more transceivers (206a, 206b) may be configured to transmit and receive user data, control information, wireless signals / channels, etc., as described in the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed herein through one or more antennas (208a, 208b). In this specification, one or more antennas may be a plurality of physical antennas or a plurality of logical antennas (e.g., antenna ports). One or more transceivers (206a, 206b) can convert the received wireless signal / channel, etc. from an RF band signal to a baseband signal in order to process the received user data, control information, wireless signal / channel, etc. using one or more processors (202a, 202b).One or more transceivers (206a, 206b) can convert user data, control information, wireless signals / channels, etc. processed using one or more processors (202a, 202b) from baseband signals to RF band signals. To this end, one or more transceivers (206a, 206b) may include (analog) oscillators and / or filters.

[0082] FIG. 3 is a diagram illustrating a method for processing a transmission signal applicable to the present specification. For example, the transmission signal may be processed by a signal processing circuit. In this case, the signal processing circuit (300) may include a scrambler (310), a modulator (320), a layer mapper (330), a precoder (340), a resource mapper (350), and a signal generator (360). In this case, for example, the operation / function of FIG. 3 may be performed in the processor (202a, 202b) and / or transceiver (206a, 206b) of FIG. 2. Also, for example, the hardware element of FIG. 3 may be implemented in the processor (202a, 202b) and / or transceiver (206a, 206b) of FIG. 2. For example, blocks 310 to 350 may be implemented in the processor (202a, 202b) of FIG. 2, and block 360 may be implemented in the transceiver (206a, 206b) of FIG. 2, but are not limited to the above-described embodiment.

[0083] A codeword can be converted into a wireless signal through the signal processing circuit (300) of FIG. 3. Here, the codeword is an encoded bit sequence of an information block. The information block may include a transmission block (e.g., UL-SCH transmission block, DL-SCH transmission block). The wireless signal may be transmitted through various physical channels (e.g., PUSCH, PDSCH) of FIG. 6. Specifically, the codeword can be converted into a scrambled bit sequence by a scrambler (310). The scrambled sequence used for scrambling is generated based on an initialization value, which may include ID information of a wireless device, etc. The scrambled bit sequence may be modulated into a modulation symbol sequence by a modulator (320). The modulation method may include pi / 2-BPSK (pi / 2-binary phase shift keying), m-PSK (m-phase shift keying), m-QAM (m-quadrature amplitude modulation), etc.

[0084] A complex modulation symbol sequence can be mapped to one or more transmission layers by a layer mapper (330). The modulation symbols of each transmission layer can be mapped to the corresponding antenna port(s) by a precoder (340) (precoding). The output z of the precoder (340) can be obtained by multiplying the output y of the layer mapper (330) by an N*M precoding matrix W, where N is the number of antenna ports and M is the number of transmission layers. Here, the precoder (340) can perform precoding after performing transform precoding (e.g., a discrete Fourier transform (DFT)) on the complex modulation symbols. Alternatively, the precoder (340) can perform precoding without performing transform precoding.

[0085] A resource mapper (350) can map the modulation symbols of each antenna port to a time-frequency resource. The time-frequency resource may include multiple symbols (e.g., CP-OFDMA symbols, DFT-s-OFDMA symbols) in the time domain and multiple subcarriers in the frequency domain. A signal generator (360) generates a radio signal from the mapped modulation symbols, and the generated radio signal can be transmitted to another device through each antenna. To this end, the signal generator (360) may include an inverse fast Fourier transform (IFFT) module, a cyclic prefix (CP) inserter, a digital-to-analog converter (DAC), a frequency uplink converter, etc.

[0086] The signal processing process for a received signal in a wireless device can be configured as the inverse of the signal processing process (310–360) of FIG. 3. For example, a wireless device (e.g., 200a, 200b of FIG. 2) can receive a wireless signal from the outside through an antenna port / transceiver. The received wireless signal can be converted into a baseband signal through a signal restorer. To this end, the signal restorer may include a frequency downlink converter, an analog-to-digital converter (ADC), a CP remover, and a fast Fourier transform (FFT) module. Subsequently, the baseband signal can be restored into a codeword through a resource de-mapper process, a postcoding process, a demodulation process, and a de-scrambling process. The codeword can be restored into the original information block through decoding. Accordingly, a signal processing circuit (not shown) for a received signal may include a signal restorer, a resource de-mapper, a postcoder, a demodulator, a de-scrambler, and a decoder.

[0087] Wireless device structure applicable to the present specification

[0088] FIG. 4 is a drawing illustrating another example of a wireless device to which the present specification applies.

[0089] Referring to FIG. 4, the wireless device (400) corresponds to the wireless device (200a, 200b) of FIG. 2 and may be composed of various elements, components, units / parts, and / or modules. For example, the wireless device (400) may include a communication unit (410), a control unit (420), a memory unit (430), and additional elements (440). The communication unit may include a communication circuit (412) and transceiver(s) (414). For example, the communication circuit (412) may include one or more processors (202a, 202b) and / or one or more memories (204a, 204b) of FIG. 2. For example, the transceiver(s) (414) may include one or more transceivers (206a, 206b) and / or one or more antennas (208a, 208b) of FIG. 2. The control unit (420) is electrically connected to the communication unit (410), the memory unit (430), and additional elements (440) and controls the general operation of the wireless device. For example, the control unit (420) may control the electrical / mechanical operation of the wireless device based on a program / code / command / information stored in the memory unit (430). Additionally, the control unit (420) may transmit information stored in the memory unit (430) to an external (e.g., another communication device) via a wireless / wired interface through the communication unit (410), or store information received from an external (e.g., another communication device) via a wireless / wired interface through the communication unit (410) in the memory unit (430).

[0090] The additional element (440) can be configured in various ways depending on the type of wireless device. For example, the additional element (440) may include at least one of a power unit / battery, an input / output unit, a driving unit, and a computing unit. Although not limited thereto, the wireless device (400) may be implemented in the form of a robot (Fig. 1, 100a), a vehicle (Fig. 1, 100b-1, 100b-2), an XR device (Fig. 1, 100c), a portable device (Fig. 1, 100d), a home appliance (Fig. 1, 100e), an IoT device (Fig. 1, 100f), a digital broadcasting terminal, a hologram device, a public safety device, an MTC device, a medical device, a fintech device (or financial device), a security device, a climate / environment device, an AI server / device (Fig. 1, 140), a base station (Fig. 1, 120), a network node, etc. Wireless devices can be used in a movable or fixed location depending on the use—e.g., service.

[0091] In FIG. 4, various elements, components, units / parts, and / or modules within the wireless device (400) may be entirely interconnected via a wired interface, or at least partially connected via a communication unit (410). For example, within the wireless device (400), the control unit (420) and the communication unit (410) may be connected via a wire, and the control unit (420) and the first unit (e.g., 430, 440) may be connected wirelessly via the communication unit (410). Additionally, each element, component, unit / part, and / or module within the wireless device (400) may include one or more additional elements. For example, the control unit (420) may be composed of one or more sets of processors. For example, the control unit (420) may be composed of a set of a communication control processor, an application processor, an electronic control unit (ECU), a graphics processing processor, a memory control processor, etc. As another example, the memory unit (430) may be composed of RAM, DRAM (dynamic RAM), ROM, flash memory, volatile memory, non-volatile memory and / or a combination thereof.

[0092] Mobile devices to which this specification applies

[0093] FIG. 5 is a drawing illustrating an example of a portable device to which the present specification applies.

[0094] FIG. 5 illustrates a portable device to which the present specification applies. The portable device may include a smartphone, a smartpad, a wearable device (e.g., a smart watch, smart glasses), a portable computer (e.g., a laptop, etc.). The portable device may be referred to as an MS (mobile station), UT (user terminal), MSS (mobile subscriber station), SS (subscriber station), AMS (advanced mobile station), or WT (wireless terminal).

[0095] Referring to FIG. 5, the portable device (500) may include an antenna unit (508), a communication unit (510), a control unit (520), a memory unit (530), a power supply unit (540a), an interface unit (540b), and an input / output unit (540c). The antenna unit (508) may be configured as part of the communication unit (510). Blocks 510 to 530 / 540a to 540c correspond to blocks 410 to 430 / 440 of FIG. 4, respectively.

[0096] The communication unit (510) can transmit and receive signals (e.g., data, control signals, etc.) with other wireless devices and base stations. The control unit (520) can control the components of the portable device (500) to perform various operations. The control unit (520) may include an application processor (AP). The memory unit (530) can store data / parameters / programs / code / commands required for the operation of the portable device (500). Additionally, the memory unit (530) can store input / output data / information, etc. The power supply unit (540a) supplies power to the portable device (500) and may include wired / wireless charging circuits, batteries, etc. The interface unit (540b) can support the connection between the portable device (500) and other external devices. The interface unit (540b) may include various ports (e.g., audio input / output ports, video input / output ports) for connection with external devices. The input / output unit (540c) can receive or output video information / signals, audio information / signals, data, and / or information input from a user. The input / output unit (540c) may include a camera, a microphone, a user input unit, a display unit (540d), a speaker and / or a haptic module, etc.

[0097] For example, in the case of data communication, the input / output unit (540c) acquires information / signals (e.g., touch, text, voice, image, video) input by the user, and the acquired information / signals can be stored in the memory unit (530). The communication unit (510) converts the information / signals stored in the memory into wireless signals and can directly transmit the converted wireless signals to another wireless device or to a base station. Additionally, the communication unit (510) can receive wireless signals from another wireless device or base station and then restore the received wireless signals to their original information / signals. The restored information / signals are stored in the memory unit (530) and then can be output in various forms (e.g., text, voice, image, video, haptic) through the input / output unit (540c).

[0098] Physical channels and general signal transmission

[0099] FIG. 6 is a diagram illustrating physical channels applicable to the present specification and a signal transmission method using them.

[0100] When a terminal is turned on again after being turned off, or when it newly enters a cell, it performs initial cell search operations, such as synchronizing with the base station, in step S611. To do this, the terminal receives the primary synchronization channel (P-SCH) and secondary synchronization channel (S-SCH) from the base station to synchronize with the base station and obtain information such as the cell ID.

[0101] Subsequently, the terminal can obtain in-cell broadcast information by receiving a physical broadcast channel (PBCH) signal from the base station. Meanwhile, during the initial cell search phase, the terminal can check the downlink channel status by receiving a Downlink Reference Signal (DL RS). After completing the initial cell search, the terminal can obtain more specific system information by receiving the physical downlink control channel (PDCCH) and the physical downlink shared channel (PDSCH) based on the physical downlink control channel information in step S612.

[0102] Subsequently, the terminal may perform a random access procedure, such as steps S613 through S616, to complete the connection to the base station. To this end, the terminal transmits a preamble through a physical random access channel (PRACH) (S613) and receives a random access response (RAR) for the preamble through a physical downlink control channel and a corresponding physical downlink shared channel (S614). The terminal transmits a physical uplink shared channel (PUSCH) using scheduling information within the RAR (S615) and performs a contention resolution procedure, such as receiving a physical downlink control channel signal and a corresponding physical downlink shared channel signal (S616).

[0103] A terminal that has performed the procedure described above may subsequently perform the reception of a physical downlink control channel signal and / or a physical downlink shared channel signal (S617) and the transmission of a physical uplink shared channel (PUSCH) signal and / or a physical uplink control channel (PUCCH) signal (S618) as a general uplink / downlink signal transmission procedure.

[0104] Control information transmitted by a terminal to a base station is collectively referred to as uplink control information (UCI). UCI includes HARQ-ACK / NACK (hybrid automatic repeat and request acknowledgment / negative-ACK), SR (scheduling request), CQI (channel quality indication), PMI (precoding matrix indication), RI (rank indication), BI (beam indication) information, etc. In this case, UCI is generally transmitted periodically via PUCCH, but depending on the embodiment (e.g., when control information and traffic data need to be transmitted simultaneously), it may be transmitted via PUSCH. Additionally, the terminal may transmit UCI non-periodically via PUSCH in response to a request or instruction from the network.

[0105] 6G communication system

[0106] The 6G (wireless communication) system aims for (i) very high data rates per device, (ii) a very large number of connected devices, (iii) global connectivity, (iv) very low latency, (v) reduced energy consumption of battery-free IoT devices, (vi) ultra-reliable connectivity, and (vii) connected intelligence with machine learning capabilities. The vision of the 6G system can be seen in four aspects, such as "intelligent connectivity," "deep connectivity," "holographic connectivity," and "ubiquitous connectivity," and the 6G system can satisfy the requirements shown in Table 1 below. In other words, Table 1 represents the requirements of the 6G system.

[0107]

[0108] At this time, the 6G system may have key factors such as enhanced mobile broadband (eMBB), ultra-reliable low latency communications (URLC), mMTC (massive machine type communications), AI integrated communication, tactile internet, high throughput, high network capacity, high energy efficiency, low backhaul and access network congestion, and enhanced data security.

[0109] Core implementation technology of 6G systems

[0110] Artificial Intelligence

[0111] The most critical and newly introduced technology for 6G systems is AI. AI was not involved in 4G systems. 5G systems will support AI partially or to a very limited extent. However, 6G systems will be supported by AI for complete automation. Advancements in machine learning will create more intelligent networks for real-time communication in 6G. Introducing AI into communications can streamline and enhance real-time data transmission. AI can determine how complex target tasks are performed using numerous analyses. In other words, AI can increase efficiency and reduce processing latency.

[0112] Time-consuming tasks such as handover, network selection, and resource scheduling can be performed instantly by using AI. AI can also play a significant role in M2M, machine-to-human, and human-to-machine communication. Furthermore, AI can enable rapid communication in Brain-Computer Interfaces (BCI). AI-based communication systems can be supported by metamaterials, intelligent structures, intelligent networks, intelligent devices, intelligent cognitive radios, self-sustaining wireless networks, and machine learning.

[0113] Recently, attempts to integrate AI with wireless communication systems have emerged, but these have primarily focused on the application layer and network layer, particularly deep learning in the field of wireless resource management and allocation. However, such research is increasingly advancing toward the MAC layer and physical layer, with attempts to combine deep learning with wireless transmission, particularly at the physical layer. AI-based physical layer transmission refers to the application of signal processing and communication mechanisms based on AI drivers rather than traditional communication frameworks in terms of fundamental signal processing and communication mechanisms. Examples include deep learning-based channel coding and decoding, deep learning-based signal estimation and detection, deep learning-based MIMO mechanisms, and AI-based resource scheduling and allocation.

[0114] Machine learning can be used for channel estimation and channel tracking, and for power allocation and interference cancellation in the physical layer of the downlink (DL). In addition, machine learning can be used for antenna selection, power control, and symbol detection in MIMO systems.

[0115] Below, we will take a closer look at machine learning.

[0116] Machine learning refers to a series of operations for training machines to create machines capable of performing tasks that humans can or find difficult to do. Machine learning requires data and learning models. Data learning methods in machine learning can be broadly classified into three types: supervised learning, unsupervised learning, and reinforcement learning.

[0117] The purpose of neural network training is to minimize output errors. It is a process that repeatedly inputs training data into a neural network, calculates the error between the network's output and the target for the training data, and updates the weights of each node by backpropagating the error from the output layer to the input layer in a direction that reduces the error.

[0118] Supervised learning uses training data with correct answers labeled, whereas unsupervised learning may not have correct answers labeled. That is, for example, in the case of supervised learning regarding data classification, the training data may consist of data where each training data point is labeled with a category. Labeled training data is input into a neural network, and an error can be calculated by comparing the network's output (category) with the labels of the training data. The calculated error is backpropagated within the neural network (i.e., from the output layer to the input layer), and the connection weights of each node in each layer of the neural network can be updated according to this backpropagation. The amount of change in the connection weights of each node being updated can be determined by the learning rate. The neural network's calculations on the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network's learning cycle. For example, efficiency can be increased by using a high learning rate in the early stages of neural network training to enable the network to quickly achieve a certain level of performance, and accuracy can be improved by using a low learning rate in the later stages of training.

[0119] The learning method may vary depending on the characteristics of the data. For example, if the goal is to accurately predict data transmitted from the transmitting end at the receiving end in a communication system, it is desirable to perform learning using supervised learning rather than unsupervised learning or reinforcement learning.

[0120] Learning models correspond to the human brain, and while the most basic linear models can be considered, a machine learning paradigm that uses highly complex neural network structures, such as artificial neural networks, as learning models is called deep learning.

[0121] The neural network cores used for learning methods are broadly classified into deep neural networks (DNN), convolutional deep neural networks (CNN), and recurrent Boltzmann machines (RNN).

[0122] An artificial neural network is an example of connecting multiple perceptrons.

[0123] Referring to Fig. 7, the entire process of inputting an input vector x=(x1,x2,...,xd), multiplying each component by a weight (W1,W2,...,Wd), summing all the results, and then applying an activation function σ() is called a perceptron. A large artificial neural network structure can extend the simplified perceptron structure illustrated in Fig. 7 to apply input vectors to different multi-dimensional perceptrons. For convenience of explanation, input or output values ​​are referred to as nodes.

[0124] Meanwhile, the perceptron structure illustrated in Fig. 7 can be described as consisting of a total of three layers based on input and output values. An artificial neural network can be represented as shown in Fig. 8, in which there are H (d+1) dimensional perceptrons between the 1st layer and the 2nd layer, and K (H+1) dimensional perceptrons between the 2nd layer and the 3rd layer.

[0125] The layer where the input vector is located is called the input layer, the layer where the final output value is located is called the output layer, and all layers located between the input and output layers are called hidden layers. Although the example in Fig. 8 shows three layers, the input layer is excluded when counting the actual number of layers in an artificial neural network, so it can be viewed as having a total of two layers. An artificial neural network is constructed by connecting perceptrons of basic blocks in a two-dimensional manner.

[0126] The aforementioned input layer, hidden layer, and output layer can be applied not only to multilayer perceptrons but also to various artificial neural network structures such as CNNs and RNNs, which will be described later. As the number of hidden layers increases, the artificial neural network becomes deeper, and the machine learning paradigm that uses a sufficiently deep artificial neural network as a learning model is called Deep Learning. In addition, the artificial neural network used for Deep Learning is called a Deep Neural Network (DNN).

[0127] The deep neural network illustrated in Fig. 9 is a multilayer perceptron composed of eight hidden and output layers. The structure of the multilayer perceptron is referred to as a fully-connected neural network. In a fully-connected neural network, there are no connections between nodes located in the same layer, and connections exist only between nodes located in adjacent layers. A DNN possesses a fully-connected neural network structure and is composed of a combination of multiple hidden layers and activation functions; it can be effectively applied to identify correlation characteristics between inputs and outputs. Here, correlation characteristics may refer to the joint probability of the input and output. Fig. 9 is a diagram illustrating an example of a deep neural network.

[0128] Meanwhile, depending on how multiple perceptrons are connected to each other, various artificial neural network structures different from the aforementioned DNN can be formed.

[0129] In a DNN, nodes located within a single layer are arranged in a one-dimensional vertical direction. However, Figure 10 assumes a case where nodes are arranged two-dimensionally, with w nodes horizontally and h nodes vertically (the convolutional neural network structure of Figure 10). In this case, since a weight is applied for each connection during the connection process from a single input node to a hidden layer, a total of h × w weights must be considered. Since there are h × w nodes in the input layer, a total of h²w² weights are required between two adjacent layers.

[0130] Figure 10 is a figure showing an example of a convolutional neural network.

[0131] The convolutional neural network of Fig. 10 has a problem in which the number of weights increases exponentially depending on the number of connections. Therefore, instead of considering all mode connections between adjacent layers, it is assumed that there are small filters, and weighted sum and activation function operations are performed on the parts where filters overlap as in Fig. 10.

[0132] A single filter has weights corresponding to its size, and the weights can be trained to extract and output specific features on an image as factors. In Figure 10, a 3×3 filter is applied to the top-left 3×3 area of ​​the input layer, and the output value resulting from the weighted sum and activation function operation for the corresponding node is stored in z22.

[0133] The above filter performs weighted sum and activation function operations while scanning the input layer and moving by a fixed interval horizontally and vertically, and places the output value at the current filter position. This method of operation is similar to the convolution operation on images in the field of computer vision, so a deep neural network with this structure is called a convolutional neural network (CNN), and the hidden layer generated as a result of the convolution operation is called a convolutional layer. In addition, a neural network having multiple convolutional layers is called a deep convolutional neural network (DCNN).

[0134] Figure 11 is a figure showing an example of a filter operation in a convolutional neural network.

[0135] In the convolution layer, the number of weights can be reduced by calculating a weighted sum that includes only the nodes located within the area covered by the filter, starting from the node where the current filter is located. As a result, a single filter can be utilized to focus on features of a local area. Accordingly, CNNs can be effectively applied to image data processing where physical distance in a 2D area serves as an important judgment criterion. Meanwhile, multiple filters can be applied immediately before the convolution layer in a CNN, and multiple output results can be generated through the convolution operation of each filter.

[0136] Meanwhile, depending on the data attributes, there may be data where sequence characteristics are important. A structure that applies a method to an artificial neural network in which elements of the data sequence are input one by one at each timestep, taking into account the length variability and sequence relationships of such sequence data, and the output vector (hidden vector) of the hidden layer output at a specific timestep is input along with the next element in the sequence is called a recurrent neural network structure.

[0137] Referring to Fig. 12, the recurrent neural network (RNN) is structured such that, in the process of inputting elements (x1(t), x2(t), ..., xd(t)) of a time point t in a data sequence into a fully connected neural network, the previous time point t-1 is input along with the hidden vector (z1(t-1), z2(t-1), ..., zH(t-1)), and a weighted sum and activation function are applied. The reason for passing the hidden vector to the next time point in this manner is that the information in the input vectors from previous time points is considered to be accumulated in the hidden vector of the current time point.

[0138] Figure 12 shows an example of a neural network structure in which a circular loop exists.

[0139] Referring to Fig. 12, the recurrent neural network operates on the input data sequence in a predetermined time sequence.

[0140] When the input vector (x1(t), x2(t), ..., xd(t)) at time point 1 is input into the recurrent neural network, the hidden vector (z1(1), z2(1), ..., zH(1)) is input together with the input vector (x1(2), x2(2), ..., xd(2)) at time point 2, and the vector (z1(2), z2(2), ..., zH(2)) of the hidden layer is determined through a weighted sum and activation function. This process is performed repeatedly up to time point 2, time point 3, ..., time point T.

[0141] Figure 13 shows an example of the operational structure of a recurrent neural network.

[0142] Meanwhile, when multiple hidden layers are placed within a recurrent neural network, it is called a deep recurrent neural network (DRNN). Recurrent neural networks are designed to be usefully applied to sequence data (e.g., natural language processing).

[0143] In addition to DNN, CNN, and RNN, it includes various deep learning techniques such as Restricted Boltzmann Machine (RBM), Deep Belief Networks (DBN), and Deep Q-Network as neural network cores used for learning, and can be applied in fields such as computer vision, speech recognition, natural language processing, and speech / signal processing.

[0144] Semantic communication is emerging as a new paradigm, driving the deep integration of recent advancements in information and communication technology and artificial intelligence innovation. This shift in communication paradigms demands innovative theories and methodologies. To this end, unlike the conventional approach that separated source coding and channel coding based on the principles of information theory proposed by Shannon, a process of identifying the semantic information of the source is required even within the channel transmission process.

[0145] The paradigm aiming for an integrated design of source coding and channel coding has been studied as "joint source-channel coding (JSCC)," a classic topic in information theory and coding theory; however, this classical JSCC framework is based on statistical probability without considering the semantic aspects of the source. It is expected that JSCC will evolve to suit modern versions if artificial intelligence technology is introduced. Regarding the expectations for source coding, artificial intelligence is expected to significantly improve efficiency by intelligently extracting the most valuable information for human communication and machine decision-making. Furthermore, regarding channel coding, it identifies important parts of the source coding output based on the semantic information of the source, enabling those parts to be transmitted and received more accurately.

[0146] Figure 14 illustrates a common source channel coding based on the existing method.

[0147] Figure 14(a) illustrates DeepJSCC. The “Deep JSCC” transmission / reception method, first proposed based on AI to realize JSCC, has garnered significant interest from both the AI ​​and wireless communication communities. Since the JSCC portion is composed of an artificial neural network, it can effectively perform the role of JSCC by outputting symbols of real values ​​to be transmitted / received through an analog channel. From Shannon’s perspective, Deep JSCC can be viewed as performing the role of mapping from the space where the source is located to a lower-dimensional space where the source can be represented. Subsequently, the decoding device, which restores the original based on the symbols received after passing through a noise channel, is also composed of an artificial neural network. The performance of Deep JSCC surpasses that of existing techniques combining channel coding methods capable of achieving JPEG / JPEG2000 / BPG source compression and channel capacity. Specifically, when performing compression / reconstruction on CIFAR-10, a small image dataset, Deep JSCC outperforms existing techniques in terms of image reconstruction performance (e.g., Peak Signal-to-Noise Ratio, PSNR) and the number of transmit / receive channels used (e.g., channel bandwidth ratio, CBR).

[0148] However, in the case of Deep JSCC, performance may degrade compared to classical source-channel coding separation methods when the source dimensionality increases, such as with large images. Additionally, if the number of channel uses during transmission and reception or the signal-to-noise ratio (SNR) increases, the coding gain of Deep JSCC falls short of that of existing separation methods. This performance degradation of Deep JSCC stems from a structural limitation in which the length of specific parts of the source cannot be adjusted based on source distribution information, and this becomes more severe as the source dimensionality increases. Furthermore, the fact that Deep JSCC does not utilize hyperprior knowledge, which is widely used in the field of image transmission, is also considered to contribute to the performance degradation.

[0149] Figure 14(b) illustrates NTSCC. Since the development of Deep JSCC, numerous studies related to semantic communication have been conducted. Among them, the field-leading NTSCC (Nonlinear Transform Source-Channel Coding) technique integrates nonlinear transform coding (NTC) and Deep JSCC to overcome many of the aforementioned limitations of Deep JSCC. NTC, newly applied to the source, better captures the source distribution, resulting in superior compression performance compared to linear transform methods previously applied to images. NTSCC then applies Deep JSCC to the source, i.e., latent information, that has passed through the nonlinear transform. Additionally, NTSCC introduces an entropy model into the latent space and utilizes a hyperprior capable of providing distribution information for each latent component. By utilizing this distribution information in Deep JSCC, an appropriate coding method can be applied to each latent component to optimize transmission / reception costs and source recovery performance. As a result, NTSCC adapts closely to the source distribution compared to Deep JSCC and provides superior encoding gain.

[0150] However, NTSCC has the following problems. If the target number of channel uses or source recovery performance is fixed, a new model must be trained to obtain a model optimized for those constraints. In other words, if constraints that are not optimized are applied to an existing trained model, the model's performance drops sharply. NTSCC also experiences a rapid decline in performance regarding source recovery and channel use when a channel state different from the one assumed during training (e.g., SNR) is provided.

[0151] Below, a method for solving the aforementioned technical problems will be explained in detail with reference to FIG. 15.

[0152] FIG. 15 illustrates a common source channel coding according to an embodiment of the present specification.

[0153] Referring to FIG. 15, the source transmitting / receiving system is a source It consists of a sender (Tx) that compresses and transmits the signal, and a receiver (Rx) that restores the original source based on the received signal. The signal transmitted at this time It passes through a specific channel W. The source is It can be represented as a random vector following the distribution, and the evaluation criterion for a source transmission / reception system is the transmission rate per unit resource, which represents the number of bits required to transmit / receive a signal. and distortion representing the difference between the source restored by the receiver and the original source It is expressed by combining these. In particular, generally, the rate is the dimension K of the transmitted signal and the dimension of the source. It is expressed as the channel bandwidth ratio (CBR) divided by...

[0154] The process of compressing / restoring the source consists of the process of performing NTC (Nonlinear Transform Coding) on ​​the source and the process of performing JSCC.

[0155] NTC encoder It takes source x as input and passes through a non-linear transformation to obtain latent information Prints. is a learnable parameter existing in the encoder. Dimension of latent information is generally the dimension of the source It is designed to be smaller to compress information while preserving the semantic information of the source.

[0156] NTC decoder The source restored to resemble the original source through a non-linear transformation that takes compressed latent information y as input. Prints. is a learnable parameter existing in the decoder. In other words, the decoder is designed to perform the inverse function of the encoder by assuming that the receiver has received the latent information y output from the encoder without error. However, finding a function that performs a perfect inverse for a non-linear transformation is extremely difficult. Therefore, after designing the function that performs NTC as an artificial neural network structure, an appropriate loss function is defined, and the parameter that minimizes it , A process of learning is required.

[0157] In this case, the loss function is a rate representing the amount of information of the previously introduced latent information y to project the purpose of the source transmitting / receiving system. distortion representing the error between the restored source and the source It consists of. If the rate of the loss function is expressed in terms of the entropy of the latent information, it is as shown in Equation 1.

[0158]

[0159] If distortion is expressed as the mean squared error (MSE) between the source and the restored source, it is as shown in Equation 2.

[0160]

[0161] Loss function It is expressed as the weighted sum of rate and distortion as shown in Equation 3. Here is a parameter representing the weights for two metrics in model training.

[0162]

[0163] However, according to previous studies, significant correlations exist among the output latent components even when NTC is applied to the source. To address this, this system utilizes a hyperprior containing distribution information for each latent component. Hyperprior The process of obtaining it is equivalent to obtaining the latent by applying NTC to the source. Specifically, the hyperprior z is a specific non-linear transformation function that takes the latent y as input. It can be obtained from. is a parameter existing in a non-linear transformation composed of an artificial neural network. Hyperprior again When passed through the function, the output is each latent component The average of and standard deviation It becomes a stacked vector, and the corresponding latent component is designed to follow a Gaussian distribution.

[0164] To summarize, the purpose of adding a process to the system to convert the latent to the hyperprior is that the latent component The purpose is to ensure that it follows the pattern and that the correlation between latent components is minimized. When this process is added, hyperprior z must also be considered in the loss function of the training process. The rate of the loss function changes as shown in Equation 4.

[0165]

[0166] The optimization problem of the artificial neural network requiring learning in the proposed system can be expressed as the following mathematical equation 5.

[0167]

[0168] Required to proceed with NTC , , , It is assumed that the functions and the parameters belonging to them are shared between the transmitter and receiver in advance. Additionally, since the exact distribution information of the hyperprior cannot be known unlike that of the latent, the distribution is estimated based on the generated hyperprior samples, and transmission and reception are performed using entropy coding and channel coding techniques capable of achieving channel capacity. Furthermore, it is assumed that there are virtually no errors occurring in this process.

[0169] The learning process according to the embodiments of this specification is explained below in comparison with the existing source transmission / reception systems, Deep JSCC and NTSCC.

[0170] Deep JSCC does not have an NTC process that is pre-processed in the source. NTSCC performs a separate quantization process to obtain the entropy value after converting the latent and hyperprior into bit information. The system according to the embodiment of this specification adds NTC and is based on a latent suitable for the source distribution, and aims to improve source recovery performance by eliminating the quantization process that causes information loss.

[0171] The JSCC process of the system according to the embodiment of this specification involves the latent y and its distribution information obtained through the aforementioned NTC process , It utilizes [this] to generate a transmission signal based on information related to the latent. Since artificial neural networks are not used in this process, a separate training process is not required. To compress information according to the purpose of the source transmission / reception system, a 2-stage Top-K selection method is employed. This will be explained in detail below.

[0172] Top-K selection refers to selecting only the K latent components with the largest values ​​under a specific criterion. To reduce distortion occurring during the source restoration process while keeping the number of selected latent components fixed, the selection criteria that have the greatest impact on restoration performance must be determined. The selection criteria will be explained in detail below with reference to Fig. 16.

[0173] Figure 16 is a graph showing the degree of distortion of information restored based on Top K selection.

[0174] Specifically, Fig. 16 is a graph showing the degree of distortion of a reconstructed image under the assumption that the receiver perfectly decodes K selected components from the latent output through the previously trained NTC for a specific image. In this case, the degree of distortion is based on the PSNR value obtained by calculating the MSE for each image pixel (PSNR [dB] = Considering the degree of distortion according to Fig. 16, the criterion for selecting the latent component is You can consider doing it this way.

[0175] The information required to transmit and receive selected latent components is the value of the selected latent component and its index. The amount of information representing the indices of the K selected components among the total latent components increases exponentially. To address this issue regarding the amount of index information, a method for selecting latent components using a two-stage approach can be considered. This will be explained in detail below with reference to Fig. 17.

[0176] Figure 17 illustrates potential components related to two-step selection.

[0177] Referring to Fig. 17, K1 latent components are selected based on . ). In the set of selected latent components (i.e., K1 latent components) above K2 components are selected based on .

[0178] Both sender and receiver , A function that outputs Assuming that it is shared in advance and the hyperprior is perfectly received, The indices of the K1 latent components selected based on [the criteria] can be fully known. Therefore, no index information is required in the first step. Since the Top-K selection performed in the second step is conducted on the K1 components, the amount of information is significantly reduced.

[0179] Assuming that the probability of each latent component being selected by Top-K selection is equal (this assumption maximizes the required amount of index information), the amount of index information required to select K2 components out of the total latent components is is. In contrast, if the 2-stage Top-K selection method is utilized, the amount of index information This becomes. If we compare them by applying the same upper limit to the binary coefficients, the former method is, and the latter method is is. At this time, It is a binary entropy function and has a maximum value of 1. Therefore, the former is overwhelmingly larger. Thus, the 2-stage Top-K selection method can have a significant advantage in terms of communication costs.

[0180] However, since latent components are selected first under non-optimal criteria, a loss in restoration performance may occur. If we examine this from a statistical perspective, Because it follows The major component is It is highly likely that it will be large. Also, if K1 is not too small, regarding the latent components obtained through 2-stage Top-K selection and the total components A significant number of the ingredients selected based on [this criterion] overlap. To summarize, through 2-stage Top-K selection It can be expected that selecting based on [this] significantly reduces the amount of index information while also significantly reducing the loss of source restoration performance.

[0181] To further reduce the transmission and reception costs of index information for latent components selected through 2-stage Top-K selection, one may consider applying run-length encoding to the corresponding index information. Run-length encoding is a method that represents consecutive occurrences of the same value within an arbitrary data sequence using only the count and the repeating value. In other words, if there are many instances of identical values ​​repeating within a sequence, run-length encoding can be utilized efficiently. The method for applying this encoding scheme to 2-stage Top-K selection is as follows.

[0182] First, in the first step The K1 latent components selected based on the size are arranged in the form of a sequence. Then, in the second step After verifying the index information corresponding to the K2 latent components selected based on the size, the indices of the K2 selected components within a sequence of length K1 are assigned a value of 1, while the indices of the unselected components are assigned a value of 0. The resulting sequence is then compressed using run-length encoding. Further compression gains can be achieved by analyzing the distribution of symbols within the compressed sequence and applying entropy coding. The case where the compression gain from run-length encoding is maximized is... This is when it is close to 0 or 1. If the parameters of the 2-stage Top-K selection are quantitatively adjusted with this in mind, performance in terms of source compression / recovery can be further improved.

[0183] As the final step of JSCC, the process of generating the final transmission signal based on the selected latent components is carried out. For the sake of convenience of explanation, the vector composed of the selected latent components It is represented as, and the corresponding mean and deviation , It is represented as. The sender is the average of the latent components obtainable from the hyperprior. Using The corresponding average Subtracts the selected value( )silver It follows the distribution of. Subsequently, the power allocated to the corresponding component The final transmission signal for the selected latent components is generated by multiplying by . The power is the deviation of the latent components It can be determined using . The above final transmission signal is It can be expressed as.

[0184] The basis for generating the final transmission signal solely through the process of multiplying by a specific constant is as follows: This is because it has been theoretically proven that for a source following a Gaussian distribution, even if only a specific value is multiplied and passed through a noise channel, it is possible to design an encoder / decoder capable of achieving the minimum MSE (MMSE) if the distortion evaluation method is set to MSE. The transmitter is the maximum power that can be imparted to a single source, It is transmitted by properly distributing it among the latent components, and the transmitted signal passes through an additive white Gaussian noise (AWGN) channel. Therefore, the receiver It receives a signal, and at this time, the noise is It follows the distribution. Assuming the receiver knows the channel state, the values ​​of the selected latent components are decoded using the MMSE estimator of Equation 6 based on the received signal.

[0185]

[0186] However, to achieve MMSE, the optimal value suitable for the source We need to find and multiply it, which can be modeled as the optimization problem of mathematical equation 7.

[0187]

[0188] By applying the MMSE estimator (Equation 6) to the above optimization problem If we substitute it, the problem can be expressed as mathematical equation 8.

[0189]

[0190] This optimization problem is a convex optimization problem, and the solution to Equation 9 can be obtained by using the KKT (Karush-Kuhn-Tucker) conditions, which are a well-known solution method for this. Here, KKT conditions refer to necessary and sufficient conditions for a solution to a non-linear optimization problem with constraints.

[0191]

[0192] At this time, constant is the value that satisfies the constraint of Equation 8. Through this, the latent components selected by 2-stage Top-K selection are decoded to achieve MMSE performance. Finally, the receiver receives the average of the latents obtained through the hyperprior. By adding to all components, the entire decoding process of the JSCC process is completed.

[0193] When examining the latent recovered through the JSCC decoder, the latent components selected via 2-stage Top-K selection have errors generated during the encoding / decoding process added to their original values, while the unselected components are the average of those components. Roman is restored.

[0194] Below, an artificial neural network related to power distribution will be described in detail with reference to FIG. 18.

[0195] FIG. 18 illustrates a common source channel coding according to another embodiment of the present specification.

[0196] If we take the previously described assumptions regarding the transmission and reception of hyperpriors realistically, the codeword length would have to be infinite to devise an optimal encoding / decoding method capable of achieving channel capacity. This is practically impossible, and current channel encoding / decoding techniques aim to achieve the minimum frame error probability when a specific codeword length is limited. However, even if few errors occur during the process of transmitting and receiving hyperpriors, the distribution information obtainable through the received hyperpriors ( , The error of ) becomes significantly large. Distribution information is used in both the 2-Stage Top-K selection process and the power distribution process of this specification, and if an error occurs in such information, the error propagates more significantly to the values ​​of the latent component and the index information recovered by the receiver. Therefore, when there are resource constraints available for a single source, an optimal transmission / reception technique must be devised to minimize the error. To this end, a method of distributing the power used to transmit the latent and hyperprior may be considered.

[0197] Referring to FIG. 18, the total power available for one source is and the power available for transmitting latent-related information Defined as, and the power available for transmitting hyperprior It is defined as. The total power is It can be expressed as. Since it is generally difficult to find a quantitative function that shows the influence of latent information and hyperprior on source recovery, theoretically the optimal and It is virtually impossible to find the optimal solution for such power distribution. To find the optimal solution for such power distribution, the present specification utilizes an additional artificial neural network (18A).

[0198] The input to the artificial neural network (18A) is i) latent, ii) hyperprior, and iii) distribution information obtainable through the hyperprior (e.g., and Includes ). The output of the artificial neural network (18A) is i) and ii) Includes

[0199] Power for Latent It is divided into power for the values ​​of latent components and power used to transmit the index. The power for the values ​​of latent components is applied as the power corresponding to the MSE optimal solution described earlier. The power for the index is applied as the power used for the bit stream compressed through run-length encoding.

[0200] Also, power for hyperprior It is also applied as the power used for the bit stream compressed through entropy encoding. All information is transmitted through the channel according to this distributed power, and a corresponding SNR is derived. By utilizing an encoding / decoding method that minimizes the bit error probability occurring in the index and hyperprior information of the latent being transmitted / received within the corresponding SNR range, an optimal digital transmission / reception technique can be implemented.

[0201] The recovered latent is input into the NTC decoder, and the final recovered output is obtained for the original source.

[0202] The operations described above correspond to the overall process (Figs. 15, 18) in which a feature selection-based joint source-channel coding technique operates on a single source.

[0203] The performance of the embodiments of the specification can be evaluated using the CBR, which is calculated by dividing the number of channel uses during the latent and hyperprior transmission / reception processes by the number of dimensions of the source, and the MSE between the restored source and the original source. This will be explained below with reference to FIGS. 19 and 20.

[0204] Figure 19 is a graph showing PSNR performance according to CBR.

[0205] Specifically, FIG. 19 illustrates the results of comparing PSNR performance according to CBR between image source transmission / reception frameworks for the CLIC2021 dataset, which consists of 55 large images with a maximum image size of 2048X1890. In this case, all transmission / reception frameworks consider a case where only one of the trained artificial neural network models can be used. This assumes a case where there is insufficient storage capacity to store the trained artificial neural network models.

[0206] When comparing performance, the proposed feature selection-based joint source-channel coding technique exhibits the highest image compression / recovery performance compared to other image transmission / reception frameworks, except for specific CBR regions. The specific CBR regions where other transmission / reception frameworks, such as NTSCC, perform better are the areas where those frameworks fixed their artificial neural networks as targets for training. The rationale for this is that other transmission / reception frameworks also utilize artificial neural network structures in the JSCC portion and train their models targeting specific CBR and PSNR values; consequently, image compression / recovery performance degrades in non-target regions. These results demonstrate that the proposed transmission / reception framework is not constrained by the artificial neural network model trained using theoretical compression / recovery methods and can be used universally when compression / recovery constraints change depending on the situation.

[0207] Figure 20 is a graph showing PSNR performance according to SNR. Specifically, Figure 20 illustrates the image compression / recovery performance of each transmission / reception framework according to the SNR of the channel used for transmission / reception when performing image compression / recovery on the dataset. At this time, a change in the channel's SNR indicates that the channel used during the communication process may change depending on time and circumstances.

[0208] When comparing performance, the image compression / recovery performance of feature selection-based joint source-channel coding is higher compared to other image transmission / reception frameworks, except for specific SNR ranges. This trend is consistent with the explanation in Fig. 19; since existing NTSCC and Deep JSCC train artificial neural networks with a fixed SNR, transmission / reception performance deteriorates rapidly when a channel different from the target SNR is used. These results demonstrate that the proposed transmission / reception framework can be used universally even with changes in the channels used.

[0209] In terms of implementation, operations according to the embodiments described above (e.g., operations related to common source channel coding) can be processed by the device of FIGS. 1 to 5 described above (e.g., the processor (202a, 202b) of FIG. 2).

[0210] In addition, operations according to the embodiments described above (e.g., operations related to common source channel coding) may be stored in memory (e.g., 204a, 204b of FIG. 2) in the form of instructions / programs (e.g., instructions, executable code) for driving at least one processor (e.g., processor (202a, 202b) of FIG. 2).

[0211] The embodiments described above will be explained in detail below with reference to FIG. 21 in terms of the operation of a wireless device (e.g., the first wireless device (200a) and the second wireless device (200b) of FIG. 2). The methods described below are distinguished only for convenience of explanation, and it is understood that a part of one method may be substituted with a part of another method or combined with one another and applied.

[0212] FIG. 21 is a flowchart for illustrating a method according to one embodiment of the present specification.

[0213] Referring to FIG. 21, a method according to one embodiment of the present specification includes the step of determining K1 potential components (S2110), the step of determining K2 potential components among the K1 potential components (S2120), and the step of transmitting a signal generated based on the K2 potential components (S2130).

[0214] In the following, the wireless device may be a terminal or a base station. For example, the wireless device may refer to the transmitter (Tx) or receiver (Rx) in FIG. 15 or FIG. 18. For example, the wireless device may be a first wireless device or a second wireless device. The first wireless device may be a transmitter, a terminal, or a base station. For example, the second wireless device may be a receiver, a base station, or a terminal.

[0215] In S2110, the wireless device determines K1 potential components based on the square of the standard deviation associated with each potential component among the potential components based on the source.

[0216] In S2120, the wireless device is the difference value between each potential component among the K1 potential components and the mean associated with that potential component (e.g., Based on ), K2 latent components are determined. Here, K1 > K2.

[0217] In S2130, the wireless device transmits a signal generated based on the K2 potential components. For example, the first wireless device transmits the signal generated based on the K2 potential components to the second wireless device.

[0218] According to one embodiment, the signal is i) each potential component (e.g., ) and the average of the corresponding latent components (e.g., ) the difference between and ii) the square of the standard deviation of the corresponding latent component (e.g., Power determined based on ) (e.g., It can be generated based on ).

[0219] According to one embodiment, the method may further include the step of transmitting index information. Specifically, the first wireless device may transmit the index information to the second wireless device. The index information may represent the K2 latent components among the K1 latent components. The index information may be based on a sequence compressed based on run-length encoding.

[0220] According to one embodiment, the Source (e.g., x in FIG. 15 or 18) may be transformed into the latent components (e.g., y in FIG. 15 or 18) based on Non-linear Transformation Coding (NTC). Specifically, the latent components are formed by a first non-linear transformation function (e.g., g in FIG. 15 or 18) that takes the Source as input. a It can be based on the output of ).

[0221] According to one embodiment, the average associated with each potential component (e.g., ) and standard deviation (e.g.: ) can be determined based on distribution information related to the above latent components. The distribution information may refer to the hyperprior described above (e.g., z in FIG. 15 or FIG. 18). The distribution information is a second non-linear transformation function (e.g., h in FIG. 15 or FIG. 18) that takes each of the latent components as input. a It can be based on the output of ). The above average (e.g., ) and the above standard deviation (e.g., ) is a third non-linear transformation function that takes the above distribution information as input (e.g., h of FIG. 15 or FIG. 18). s It can be based on the output of ).

[0222] According to one embodiment, the method may further include the step of transmitting distribution information related to the potential components. Specifically, the first wireless device may transmit distribution information related to the potential components to the second wireless device. The first transmission power (e.g., P y ) and the second transmission power (e.g., P z ) may be based on the output of a neural network (e.g., 18A in FIG. 18). The input of the neural network may include i) the latent components, ii) the distribution information, and iii) the mean and standard deviation associated with each latent component.

[0223] According to one embodiment, the signal (received by the second wireless device) can be decoded based on the MMSE estimator described above (see Equation 6). The values ​​of the K2 latent components are i) values ​​obtained based on the decoding (e.g., ) and ii) the mean of each latent component based on the above distribution information (e.g., It can be obtained / determined based on the sum of ).

[0224] Operations based on the above-described S2110 to S2130 can be implemented by the device of FIG. 2. For example, a wireless device (200a or 200b) may control one or more transceivers (206a or 206b) and / or one or more memories (204a or 204b) to perform operations based on S2110 to S2130.

[0225] Here, the wireless communication technology implemented in the wireless devices (200a, 200b) of this specification may include LTE, NR, and 6G, as well as Narrowband Internet of Things for low-power communication. For example, NB-IoT technology may be an example of LPWAN (Low Power Wide Area Network) technology and may be implemented according to standards such as LTE Cat NB1 and / or LTE Cat NB2, but is not limited to the names mentioned above. Additionally, or generally, the wireless communication technology implemented in the wireless devices (200a, 200b) of this specification may perform communication based on LTE-M technology. For example, LTE-M technology may be an example of LPWAN technology and may be referred to by various names such as eMTC (enhanced Machine Type Communication). For example, LTE-M technology may be implemented in at least one of various standards such as 1) LTE CAT 0, 2) LTE Cat M1, 3) LTE Cat M2, 4) LTE non-BL (non-Bandwidth Limited), 5) LTE-MTC, 6) LTE Machine Type Communication, and / or 7) LTE M, and is not limited to the names mentioned above. Additionally or generally, wireless communication technology implemented in the wireless devices (200a, 200b) of this specification may include at least one of ZigBee, Bluetooth, and Low Power Wide Area Network (LPWAN) for low-power communication, and is not limited to the names mentioned above. As an example, ZigBee technology can create personal area networks (PANs) related to small / low-power digital communication based on various standards such as IEEE 802.15.4, and may be referred to by various names.

[0226] The embodiments described above are combinations of the components and features of this specification in a specific form. Each component or feature should be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, it is possible to construct embodiments of this specification by combining some components and / or features. The order of operations described in the embodiments of this specification may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment. It is obvious that embodiments may be constructed by combining claims that do not have an explicit citation relationship in the claims, or that they may be included as new claims through amendments made after filing.

[0227] Embodiments according to the present specification may be implemented by various means, e.g., hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, an embodiment of the present specification may be implemented by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0228] In the case of implementation by firmware or software, an embodiment of the present specification may be implemented in the form of a module, procedure, function, etc., that performs the functions or operations described above. The software code may be stored in memory and executed by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various known means.

[0229] It is obvious to those skilled in the art that this specification may be embodied in other specific forms without departing from the essential features of this specification. Accordingly, the detailed description set forth above should not be interpreted restrictively in all respects but should be considered illustrative. The scope of this specification shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of this specification are included within the scope of this specification.

Claims

1. Regarding the method, A step of determining K1 latent components based on the square of the standard deviation associated with each latent component among the latent components based on the source; A step of determining K2 potential components based on the difference value between each potential component and the mean associated with the corresponding potential component among the K1 potential components; and A method comprising the step of transmitting a signal generated based on the above K2 potential components.

2. In Paragraph 1, A method characterized in that the above signal is generated based on power determined based on i) the difference between each latent component and the mean of the corresponding latent component and ii) the square of the standard deviation of the corresponding latent component.

3. In Paragraph 1, It further includes a step of transmitting index information, and A method characterized in that the above index information represents the K2 potential components among the K1 potential components.

4. In Paragraph 3, A method characterized in that the above index information is based on a sequence compressed based on run-length encoding.

5. In Paragraph 1, A method characterized in that the above Source is transformed into the above latent components based on Non-linear Transformation Coding (NTC).

6. In Paragraph 5, A method characterized in that the above latent components are based on the output of a first nonlinear transformation function that takes the above Source as input.

7. In Paragraph 1, A method characterized in that the mean and standard deviation associated with each latent component are determined based on distribution information associated with the said latent components.

8. In Paragraph 7, A method characterized by the above distribution information being based on the output of a second non-linear transformation function that takes each of the above latent components as input.

9. In Paragraph 1, The method further includes the step of transmitting distribution information related to the above potential components; A method characterized in that the above signal is transmitted based on a first transmission power, and the above distribution information is transmitted based on a second transmission power.

10. In Paragraph 9, The first transmission power and the second transmission power are based on the output of a neural network, and A method characterized in that the input to the neural network comprises i) the latent components, ii) the distribution information, and iii) the mean and standard deviation associated with each latent component.

11. In wireless devices, One or more transmitters / receivers; One or more processors; and It includes one or more memories connected to the above one or more processors and storing instructions, A wireless device characterized by the above instructions being set so that the one or more processors perform all steps of the method according to any one of claims 1 to 10, based on execution by the one or more processors.

12. A device comprising one or more memories and one or more processors connected to the one or more memories, An apparatus characterized in that the above one or more memories store instructions that set the one or more processors to perform all steps of the method according to any one of claims 1 to 10, based on execution by the above one or more processors.

13. In one or more non-transitory computer-readable media storing instructions, One or more non-transitory computer-readable media characterized by instructions executable by one or more processors, wherein the one or more processors are configured to perform all steps of the method according to any one of claims 1 to 10.

Images