Device and method for processing data for semantic communication in wireless communication system

Abstract

The present disclosure is for processing data for semantic communication in a wireless communication system, and a method may comprise the steps of: performing a random access procedure; receiving a request for capability information; transmitting the capability information; performing signaling for configuration related to communication; generating a semantic representation corresponding to pre-processed data by performing semantic encoding; and transmitting the semantic representation and region information related to the pre-processing.

Description

Device and method for processing data for semantic communication in a wireless communication system

[0001] The following description relates to a wireless communication system, and specifically to an apparatus and method for processing data for semantic communication in a wireless communication system.

[0002] Wireless access systems are being widely deployed to provide various types of communication services, such as voice and data. Generally, a wireless access system is a multiple access system capable of supporting communication with multiple users by sharing available system resources (bandwidth, transmission power, etc.). Examples of multiple access systems include CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), and SC-FDMA (single carrier frequency division multiple access) systems.

[0003] In particular, as many communication devices require large communication capacities, enhanced mobile broadband (eMBB) communication technology is being proposed as an improvement over existing radio access technology (RAT). Furthermore, communication systems are being proposed that consider not only massive machine type communications (mmTC), which connects multiple devices and objects to provide various services anytime and anywhere, but also services and user equipment (UE) that are sensitive to reliability and latency. Various technical configurations are being proposed for this purpose.

[0004] The present disclosure may provide an apparatus and method for effectively transmitting and receiving semantic information of a message in a wireless communication system.

[0005] The present disclosure relates to an apparatus and method for pre-processing data for semantic communication in a wireless communication system.

[0006] The present disclosure relates to an apparatus and method for post-processing data for semantic communication in a wireless communication system.

[0007] The present disclosure relates to an apparatus and method for effectively performing task-oriented semantic communication in a wireless communication system.

[0008] The present disclosure relates to an apparatus and method for increasing system flexibility of task-oriented semantic communication in a wireless communication system.

[0009] The present disclosure relates to an apparatus and method for increasing the transmission efficiency of task-oriented semantic communication in a wireless communication system.

[0010] The present disclosure relates to an apparatus and method for performing task-oriented semantic communication using an attention map in a wireless communication system.

[0011] The present disclosure relates to an apparatus and method for further improving the performance of a task in a fixed CBR (channel bandwidth ratio) resource efficiency situation in a wireless communication system.

[0012] The present disclosure relates to an apparatus and method for controlling the amount of change in a data value of source data for semantic communication in a wireless communication system.

[0013] The present disclosure relates to an apparatus and method for lowering the CBR for transmitting a portion of data that does not have a relatively large impact on the performance of a target task in a wireless communication system.

[0014] The technical objectives to be achieved in this disclosure are not limited to those mentioned above, and other unmentioned technical problems may be considered by those skilled in the art to which the technical configuration of this disclosure applies, based on the embodiments of this disclosure described below.

[0015] As an example of the present disclosure, the method may include the steps of performing a random access access procedure, receiving a request for capability information, transmitting said capability information, performing signaling for a setting related to communication, and generating a semantic representation corresponding to pre-processed data by performing semantic encoding, and transmitting said semantic representation and region information related to said pre-processing. Here, said pre-processing may include an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by said region information, which is determined based on an attention map related to a target task performed using said data.

[0016] As an example of the present disclosure, the method comprises: performing a random access access procedure; transmitting a request for capability information; receiving the capability information; performing signaling for a setting related to communication; receiving a semantic representation corresponding to pre-processed data and region information related to the pre-processing; obtaining the pre-processed data by performing semantic decoding; and performing a target task using the restored data obtained by performing post-processing on the pre-processed data based on the region information, wherein the post-processing may include an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the region information.

[0017] As an example of the present disclosure, the device comprises a transceiver and a processor coupled to the transceiver, wherein the processor is configured to perform a random access access procedure, receive a request for capability information, transmit the capability information, perform signaling for a setting related to communication, and perform semantic encoding to generate a semantic representation corresponding to pre-processed data, and transmit the semantic representation and region information related to the pre-processing, wherein the pre-processing may include an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the region information determined based on an attention map related to a target task performed using the data.

[0018] As an example of the present disclosure, the device comprises a transceiver and a processor coupled to the transceiver, wherein the processor is configured to perform a random access access procedure, transmit a request for capability information, receive the capability information, perform signaling for a setting related to communication, receive a semantic representation corresponding to the pre-processed data and region information related to the pre-processing, obtain the pre-processed data by performing semantic decoding, and perform a target task using the restored data obtained by performing post-processing on the pre-processed data based on the region information, wherein the post-processing may include an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the region information.

[0019] As an example of the present disclosure, a terminal comprises at least one processor, and at least one memory connected to the at least one processor and storing instructions that cause the terminal to perform operations as executed by the at least one processor, wherein the operations may include: performing a random access access procedure; receiving a request for capability information; transmitting the capability information; performing signaling for a setting related to communication; generating a semantic representation corresponding to pre-processed data by performing semantic encoding; and transmitting the semantic representation and region information related to the pre-processing. Herein, the pre-processing includes an operation for increasing or decreasing the amount of change of data values ​​for at least a portion of the data indicated by the region information determined based on an attention map related to a target task performed using the data.

[0020] As an example of the present disclosure, a non-transitory computer-readable medium storing at least one program instruction may cause a terminal to perform operations as the at least one program instruction is executed by at least one processor, said operations may include: performing a random access access procedure; receiving a request for capability information; transmitting said capability information; performing signaling for a setting related to communication; generating a semantic representation corresponding to pre-processed data by performing semantic encoding; and transmitting said semantic representation and region information related to said pre-processing. Herein, said pre-processing may include an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by said region information, which is determined based on an attention map related to a target task performed using said data.

[0021] The embodiments of the present disclosure described above are merely some of the preferred embodiments of the present disclosure, and various embodiments reflecting the technical features of the present disclosure can be derived and understood by those skilled in the art based on the detailed description of the present disclosure set forth below.

[0022] The following effects may be achieved by embodiments based on the present disclosure.

[0023] According to the present disclosure, semantic communication can be performed more effectively.

[0024] The effects obtainable from the embodiments of the present disclosure are not limited to those mentioned above, and other unmentioned effects can be clearly derived and understood by a person skilled in the art to which the technical configuration of the present disclosure applies from the description of the embodiments of the present disclosure below. That is, unintended effects resulting from implementing the configuration described in the present disclosure can also be derived by a person skilled in the art from the embodiments of the present disclosure.

[0025] The drawings attached below are intended to aid in understanding the present disclosure and may provide embodiments of the present disclosure together with the detailed description. However, the technical features of the present disclosure are not limited to specific drawings, and features disclosed in each drawing may be combined with one another to form new embodiments. Reference numerals in each drawing may denote structural elements.

[0026] FIG. 1 illustrates an example of a communication system applicable to the present disclosure.

[0027] FIG. 2 illustrates an example of a wireless device applicable to the present disclosure.

[0028] FIG. 3 illustrates a method for processing a transmission signal applicable to the present disclosure.

[0029] FIG. 4 illustrates a communication procedure between a terminal and a base station applicable to the present disclosure.

[0030] FIG. 5 illustrates an example of a communication structure that can be provided in a 6G (6th generation) system applicable to the present disclosure.

[0031] FIG. 6 illustrates an electromagnetic spectrum applicable to the present disclosure.

[0032] FIG. 7 illustrates a transmitter structure applicable to the present disclosure.

[0033] FIG. 8 illustrates an example of a functional framework for the application of artificial intelligence technology applicable to the present disclosure.

[0034] FIG. 9 illustrates an example of a procedure for utilizing an artificial intelligence model applicable to the present disclosure.

[0035] FIG. 10 illustrates a communication procedure based on artificial intelligence (AI) technology applicable to the present disclosure.

[0036] FIG. 11 illustrates a communication model applicable to the present disclosure.

[0037] FIG. 12a illustrates a transmitter and a receiver for general communication.

[0038] FIG. 12b illustrates a transmitter and receiver with Deep-JSCC (deep joint source-channel coding) technology applied.

[0039] Figure 13 illustrates an example of a deep neural network (DNN) structure for implementing Deep-JSCC.

[0040] Figure 14 illustrates a block diagram of the NTC (nonlinear transform coding) technique.

[0041] Figure 15 illustrates an example of the structure of NTSCC (nonlinear transform source-channel coding) technology that fuses Deep-JSCC and NTC.

[0042] FIGS. 16a to 16d illustrate examples of channel bandwidth costs of latent variables by NTSCC.

[0043] FIGS. 17a to 17c illustrate examples of loss, PSNR (peak signal to noise ratio), and CBR (channel bandwidth ratio) values ​​for various images.

[0044] Figures 18a and 18b illustrate the performance of NTSCC technology.

[0045] Figure 19 illustrates an example of a framework for performing task-oriented semantic communication.

[0046] FIG. 20 illustrates an example of a reconstruction-based framework for performing task-oriented semantic communication.

[0047] FIG. 21 illustrates an example of a framework for semantic communication according to one embodiment of the present disclosure.

[0048] Figure 22 illustrates an example of an attention map for an image.

[0049] FIG. 23 is an example of source data divided into a plurality of patches according to one embodiment of the present disclosure.

[0050] FIG. 24a illustrates an example of a blurring matrix according to one embodiment of the present disclosure.

[0051] FIG. 24b illustrates an example of a sharpening matrix according to one embodiment of the present disclosure.

[0052] FIG. 25 illustrates an example of a pre-processing operation according to one embodiment of the present disclosure.

[0053] FIG. 26 illustrates an example of a post-processing operation according to one embodiment of the present disclosure.

[0054] FIG. 27 illustrates an example of a procedure for performing pre-processing and post-processing according to one embodiment of the present disclosure.

[0055] FIG. 28 illustrates an example of a procedure for performing semantic communication according to one embodiment of the present disclosure.

[0056] FIG. 29 illustrates an example of a procedure for performing semantic communication of a terminal according to one embodiment of the present disclosure.

[0057] FIG. 30 illustrates an example of a procedure for performing semantic communication of a base station according to one embodiment of the present disclosure.

[0058] FIGS. 31a and FIGS. 31b illustrate examples of patch-specific PSNR performance when pre-processing and post-processing according to an embodiment of the present disclosure are not applied and when they are applied.

[0059] FIG. 32 illustrates an example of a wireless device applicable to the present disclosure.

[0060] FIG. 33 illustrates an example of a portable device applicable to the present disclosure.

[0061] FIG. 34 illustrates an example of a vehicle or autonomous vehicle applicable to the present disclosure.

[0062] FIG. 35 illustrates an example of a vehicle applicable to the present disclosure.

[0063] FIG. 36 illustrates an example of an extended reality (XR) device applicable to the present disclosure.

[0064] FIG. 37 illustrates an example of a robot applicable to the present disclosure.

[0065] FIG. 38 illustrates an example of an AI device applicable to the present disclosure.

[0066] The following embodiments are combinations of the components and features of the present disclosure in a predetermined form. Each component or feature may be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, some components and / or features may be combined to form embodiments of the present disclosure. The order of operations described in the embodiments of the present disclosure may be changed. Some components or features of any embodiment may be included in other embodiments, or may be replaced with corresponding components or features of other embodiments.

[0067] In the description of the drawings, procedures or steps that could obscure the gist of the present disclosure have not been described, nor have procedures or steps that are understandable to those skilled in the art been described.

[0068] Throughout the specification, when a part is described as "comprising" or "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part," "...unit," and "module" as used in the specification refer to a unit that performs at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software. Additionally, "one (a or an)," "one," "the," and similar related terms may be used in the context describing the present disclosure (particularly in the context of the following claims) in both singular and plural forms, unless otherwise indicated in the specification or clearly contradicted by the context.

[0069] In this specification, the embodiments of the present disclosure are described with a focus on the data transmission and reception relationship between a base station and a mobile station. Here, the base station refers to a terminal node of a network that communicates directly with a mobile station. Specific operations described in this document as being performed by a base station may, in some cases, be performed by an upper node of the base station.

[0070] That is, in a network consisting of multiple network nodes including a base station, various operations performed for communication with a mobile station may be performed by the base station or other network nodes other than the base station. In this case, 'base station' may be replaced by terms such as fixed station, Node B, eNB (eNode B), gNB (gNode B), ng-eNB, advanced base station (ABS), or access point.

[0071] Additionally, in the embodiments of the present disclosure, the term terminal may be replaced with terms such as user equipment (UE), mobile station (MS), subscriber station (SS), mobile subscriber station (MSS), mobile terminal, or advanced mobile station (AMS).

[0072] Furthermore, the transmitting end refers to a fixed and / or mobile node that provides data or voice services, and the receiving end refers to a fixed and / or mobile node that receives data or voice services. Therefore, in the case of the uplink, a mobile station can be the transmitting end and a base station can be the receiving end. Similarly, in the case of the downlink, a mobile station can be the receiving end and a base station can be the transmitting end.

[0073] Embodiments of the present disclosure may be supported by standard documents disclosed in at least one of the wireless access systems, such as IEEE 802.xx systems, 3GPP (3rd Generation Partnership Project) systems, 3GPP LTE (Long Term Evolution) systems, 3GPP 5G (5th generation) NR (New Radio) systems and 3GPP2 systems, and in particular, embodiments of the present disclosure may be supported by 3GPP TS (technical specification) 38.211, 3GPP TS 38.212, 3GPP TS 38.213, 3GPP TS 38.321 and 3GPP TS 38.331 documents.

[0074] In addition, the embodiments of the present disclosure may be applied to other wireless access systems and are not limited to the systems described above. For example, they may be applicable to systems applied after the 3GPP 5G NR system and are not limited to specific systems.

[0075] That is, obvious steps or parts not described in the embodiments of the present disclosure may be described by referring to the aforementioned documents. Additionally, all terms disclosed in this document may be explained by the aforementioned standard documents.

[0076] Hereinafter, preferred embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. The detailed description disclosed below, together with the accompanying drawings, is intended to describe exemplary embodiments of the present disclosure and is not intended to represent the only embodiment in which the technical configuration of the present disclosure can be implemented.

[0077] Additionally, specific terms used in the embodiments of the present disclosure are provided to aid in understanding the present disclosure, and the use of such specific terms may be modified in other forms without departing from the technical spirit of the present disclosure.

[0078] The following technology can be applied to various wireless access systems such as CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), and SC-FDMA (single carrier frequency division multiple access).

[0079]

[0080] For the sake of clarity, the following description is based on 3GPP communication systems (e.g., LTE, NR, etc.), but the technical scope of this disclosure is not limited thereto. LTE may refer to technology from 3GPP TS 36.xxx Release 8 onwards. Specifically, LTE technology from 3GPP TS 36.xxx Release 10 onwards is referred to as LTE-A, and LTE technology from 3GPP TS 36.xxx Release 13 onwards may be referred to as LTE-A pro. 3GPP NR may refer to technology from TS 38.xxx Release 15 onwards. 3GPP 6G may refer to technology from TS Release 17 and / or Release 18 onwards. "xxx" indicates a specific standard document number. LTE / NR / 6G may be collectively referred to as 3GPP systems.

[0081] Regarding the background technology, terms, abbreviations, etc. used in this disclosure, reference may be made to standard documents published prior to this disclosure. For example, reference may be made to standard documents 36.xxx and 38.xxx.

[0082]

[0083] Communication systems applicable to the present disclosure

[0084] Although not limited thereto, the various descriptions, functions, procedures, proposals, methods, and / or flowcharts of the disclosure disclosed in this document may be applied to various fields requiring wireless communication / connection (e.g., 5G) between devices.

[0085] Examples are provided in more detail below with reference to the drawings. In the following drawings and descriptions, the same reference numerals may represent the same or corresponding hardware blocks, software blocks, or function blocks unless otherwise described.

[0086] FIG. 1 illustrates an example of a communication system to which the present disclosure applies.

[0087] Referring to FIG. 1, the communication system (100) to which the present disclosure applies includes a wireless device, a base station, and a network. Here, the wireless device refers to a device that performs communication using wireless access technology (e.g., LTE, LTE-A, LTE-A pro, NR, 5G, 5G-A, 6G) and may be referred to as a communication / wireless / 5G device. Although not limited thereto, the wireless device may include a robot (100a), a vehicle (100b-1, 100b-2), an XR (extended reality) device (100c), a hand-held device (100d), a home appliance (100e), an IoT (Internet of Thing) device (100f), and an AI (artificial intelligence) device / server (100g). For example, the vehicle may include a vehicle equipped with wireless communication capabilities, an autonomous vehicle, a vehicle capable of performing inter-vehicle communication, etc. Here, the vehicle (100b-1, 100b-2) may include an unmanned aerial vehicle (UAV) (e.g., a drone). The XR device (100c) includes an augmented reality (AR) / virtual reality (VR) / mixed reality (MR) device and may be implemented in the form of a head-mounted device (HMD), a head-up display (HUD) equipped in a vehicle, a television, a smartphone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a robot, etc. The portable device (100d) may include a smartphone, a smartpad, a wearable device (e.g., a smartwatch, smart glasses), a computer (e.g., a laptop, etc.). The home appliance (100e) may include a TV, a refrigerator, a washing machine, etc. The IoT device (100f) may include a sensor, a smart meter, etc.For example, the base station (120) and network (130) may also be implemented as wireless devices, and a specific wireless device (120a) may act as a base station / network node to other wireless devices.

[0088] Wireless devices (100a to 100f) can be connected to a network (130) through a base station (120). AI technology may be applied to the wireless devices (100a to 100f), and the wireless devices (100a to 100f) can be connected to an AI server (100g) through the network (130). The network (130) can be configured using a 3G network, a 4G (e.g., LTE) network, a 5G (e.g., NR) network, or a 6G network. The wireless devices (100a to 100f) may communicate with each other through the base station (120) / network (130), but may also communicate directly (e.g., sidelink communication) without going through the base station (120) / network (130). For example, vehicles (100b-1, 100b-2) can communicate directly (e.g., V2V (vehicle to vehicle) / V2X (vehicle to everything) communication). Also, an IoT device (100f) (e.g., a sensor) can communicate directly with another IoT device (e.g., a sensor) or other wireless devices (100a to 100f).

[0089] Wireless communication / connection (150a, 150b, 150c) can be established between wireless devices (100a to 100f) / base station (120) and between base station (120) / base station (120). Here, wireless communication / connection can be established through various wireless access technologies such as uplink / downlink communication (150a), sidelink communication (150b) (or D2D communication), and communication between base stations (150c) (e.g., relay, IAB (integrated access backhaul)). Through wireless communication / connection (150a, 150b, 150c), wireless devices and base stations / wireless devices, and base stations and base stations can transmit / receive wireless signals to / from each other. For example, wireless communication / connection (150a, 150b, 150c) can transmit / receive signals through various physical channels. To this end, based on the various proposals of the present disclosure, at least some of the following may be performed: a process for setting various configuration information for transmitting / receiving wireless signals, a process for various signal processing (e.g., channel encoding / decoding, modulation / demodulation, resource mapping / demapping, etc.), a resource allocation process, etc.

[0090]

[0091] Devices applicable to the present disclosure

[0092] FIG. 2 illustrates an example of a wireless device that can be applied to the present disclosure.

[0093] Referring to FIG. 2, the wireless device (200) can transmit and receive wireless signals through various wireless access technologies (e.g., LTE, LTE-A, LTE-A pro, NR, 5G, 5G-A, 6G). The wireless device (200) includes at least one processor (202) and at least one memory (204), and may additionally include at least one transceiver (206) and / or at least one antenna (208).

[0094] The processor (202) controls the memory (204) and / or the transceiver (206) and may be configured to implement the descriptions, functions, procedures, proposals, methods, and / or sequences of operation disclosed in this document. For example, the processor (202) may process information within the memory (204) to generate a first information / signal and then transmit a wireless signal containing the first information / signal through the transceiver (206). Additionally, the processor (202) may receive a wireless signal containing a second information / signal through the transceiver (206) and then store information obtained from the signal processing of the second information / signal in the memory (204). The memory (204) may be connected to the processor (202) and may store various information related to the operation of the processor (202). For example, memory (204) may store software code containing instructions for performing some or all of the processes controlled by the processor (202) or for performing the descriptions, functions, procedures, proposals, methods, and / or sequences of operations disclosed in this document. Here, the processor (202) and memory (204) may be part of a communication modem / circuit / chip designed to implement wireless communication technology. A transceiver (206) may be connected to the processor (202) and may transmit and / or receive wireless signals through at least one antenna (208). The transceiver (206) may include a transmitter and / or receiver. The transceiver (206) may be interchangeable with a radio frequency (RF) unit. In this disclosure, a wireless device may mean a communication modem / circuit / chip.

[0095] Hereinafter, hardware elements of the wireless device (200) will be described in more detail. Although not limited thereto, at least one protocol layer may be implemented by at least one processor (202). For example, at least one processor (202) may implement at least one layer (e.g., functional layers such as PHY (physical), MAC (media access control), RLC (radio link control), PDCP (packet data convergence protocol), RRC (radio resource control), and SDAP (service data adaptation protocol). At least one processor (202) may generate at least one PDU (Protocol Data Unit) and / or at least one SDU (service data unit) according to the descriptions, functions, procedures, proposals, methods and / or operation sequences disclosed in this document. At least one processor (202) may generate messages, control information, data, or information according to the descriptions, functions, procedures, proposals, methods and / or operation sequences disclosed in this document. At least one processor (202) may generate a signal (e.g., baseband signal) including a PDU, SDU, message, control information, data, or information according to the functions, procedures, proposals, and / or methods disclosed in this document and provide it to at least one transceiver (206). At least one processor (202) may receive a signal (e.g., baseband signal) from at least one transceiver (206) and may obtain a PDU, SDU, message, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this document.

[0096] At least one processor (202) may be referred to as a controller, microcontroller, microprocessor, or microcomputer. At least one processor (202) may be implemented by hardware, firmware, software, or a combination thereof. For example, at least one application-specific integrated circuit (ASIC), at least one digital signal processor (DSP), at least one digital signal processing device (DSPD), at least one programmable logic device (PLD), or at least one field programmable gate array (FPGA) may be included in at least one processor (202). The descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this document may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, etc. Firmware or software configured to perform the descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this document may be included in at least one processor (202) or stored in at least one memory (204) and driven by at least one processor (202). The descriptions, functions, procedures, proposals, methods, and / or flowcharts disclosed in this document may be implemented using firmware or software in the form of code, instructions, and / or sets of instructions.

[0097] At least one memory (204) may be connected to at least one processor (202) and may store various forms of data, signals, messages, information, programs, codes, instructions, and / or commands. At least one memory (204) may be composed of ROM (read-only memory), RAM (random access memory), EPROM (erasable programmable read-only memory), flash memory, hard drive, registers, cache memory, computer read storage media, and / or combinations thereof. At least one memory (204) may be located inside and / or outside of at least one processor (202). Additionally, at least one memory (204) may be connected to at least one processor (202) via various technologies, such as wired or wireless connections.

[0098] At least one transceiver (206) may transmit user data, control information, wireless signals / channels, etc., as mentioned in the methods and / or operation flowcharts, etc. of this document to at least one other device. At least one transceiver (206) may receive user data, control information, wireless signals / channels, etc., as mentioned in the descriptions, functions, procedures, proposals, methods and / or operation flowcharts, etc. disclosed in this document from at least one other device. For example, at least one transceiver (206) may be connected to at least one processor (202) and may transmit and receive wireless signals. For example, at least one processor (202) may control at least one transceiver (206) to transmit user data, control information, or wireless signals to at least one other device. Additionally, at least one processor (202) may control at least one transceiver (206) to receive user data, control information, or wireless signals from at least one other device. Additionally, at least one transceiver (206) may be connected to at least one antenna (208), and at least one transceiver (206) may be configured to transmit and receive user data, control information, wireless signals / channels, etc., as described in the descriptions, functions, procedures, proposals, methods, and / or operation sequence diagrams disclosed in this document through at least one antenna (208). In this document, at least one antenna may be a plurality of physical antennas or a plurality of logical antennas (e.g., antenna ports). At least one transceiver (206) may convert the received wireless signals / channels, etc., from RF band signals to baseband signals in order to process the received user data, control information, wireless signals / channels, etc., using at least one processor (202). At least one transceiver (206) may convert the processed user data, control information, wireless signals / channels, etc., from baseband signals to RF band signals using at least one processor (202).To this end, at least one transceiver (206) may include an (analog) oscillator and / or filter.

[0099] The components of the wireless device described with reference to FIG. 2 may be referred to by other terms in terms of their function. For example, the processor (202) may be referred to as the control unit, the transceiver (206) as the communication unit, and the memory (204) as the storage unit. In some cases, the communication unit may be used to mean at least a part of the processor (202) and the transceiver (206).

[0100] The structure of the wireless device described with reference to FIG. 2 can be understood as the structure of at least part of various devices. For example, the structure of the wireless device exemplified in FIG. 2 may be at least part of the various devices described with reference to FIG. 1 (e.g., robot (100a), vehicle (100b-1, 100b-2), XR device (100c), portable device (100d), home appliance (100e), IoT device (100f), AI device / server (100g)). Furthermore, according to various embodiments, the device may include other components in addition to the components exemplified in FIG. 2.

[0101] For example, the device may be a portable device such as a smartphone, smartpad, wearable device (e.g., smart watch, smart glasses), or portable computer (e.g., laptop, etc.). In this case, the device may further include at least one of a power supply unit that supplies power and includes a wired / wireless charging circuit, a battery, etc., an interface unit that includes at least one port for connection with another device (e.g., audio input / output port, video input / output port), and an input / output unit for inputting and outputting video information / signals, audio information / signals, data, and / or information input by a user.

[0102] For example, the device may be a mobile device such as a mobile robot, vehicle, train, manned / unmanned aerial vehicle (AV), or ship. In this case, the device may further include at least one of a drive unit comprising at least one of an engine, motor, power train, wheel, brake, and steering device of the device; a power supply unit that supplies power and includes a wired / wireless charging circuit, battery, etc.; a sensor unit that senses state information, environmental information, and user information of the device or its surroundings; an autonomous driving unit that performs functions such as path maintenance, speed control, and destination setting; and a position measurement unit that acquires position information of the moving body through a GPS (global positioning system) and various sensors.

[0103] For example, the device may be an XR device such as an HMD, a HUD (head-up display) equipped in a vehicle, a television, a smartphone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a robot, etc. In this case, the device may further include at least one of a power supply unit that supplies power and includes a wired / wireless charging circuit, a battery, etc., an input / output unit that acquires control information, data, etc. from the outside and outputs a generated XR object, and a sensor unit that senses state information, environment information, and user information of the device or the surroundings of the device.

[0104] For example, the device may be a robot that can be classified into industrial, medical, household, military, etc., depending on the purpose or field of use. In this case, the device may further include at least one of a sensor unit that senses state information, environmental information, and user information of the device or its surroundings, and a drive unit that performs various physical actions, such as moving robot joints.

[0105] For example, the device may be an AI device such as a TV, projector, smartphone, PC, laptop, digital broadcasting terminal, tablet PC, wearable device, set-top box (STB), radio, washing machine, refrigerator, digital signage, robot, vehicle, etc. In this case, the device may further include at least one of an input unit that acquires various types of data from the outside, an output unit that generates output related to sight, hearing, or touch, a sensor unit that senses state information, environmental information, and user information of the device or its surroundings, and a training unit that learns a model composed of an artificial neural network using training data.

[0106] The structure of the wireless device illustrated in FIG. 2 can be understood as part of a RAN node (e.g., base station, DU, RU, RRH, etc.). That is, the device illustrated in FIG. 2 may be a RAN node. In this case, the device may further include a wired transceiver for front haul and / or back haul communication. However, if the front haul and / or back haul communication is based on wireless communication, at least one transceiver (206) illustrated in FIG. 2 is used for front haul and / or back haul communication, and the wired transceiver may not be included.

[0107]

[0108] FIG. 3 illustrates a method for processing a transmission signal applicable to the present disclosure. For example, the transmission signal may be processed by a signal processing circuit. In this case, the signal processing circuit (300) may include scramblers (310), modulators (320), a layer mapper (330), a precoder (340), resource mappers (350), and signal generators (360). In this case, for example, the operation / function of FIG. 3 may be performed in the processor (202) and / or transceiver (206) of FIG. 2. Also, for example, the hardware elements of FIG. 3 may be implemented in the processor (202) and / or transceiver (206) of FIG. 2. For example, blocks 310 to 360 may be implemented in the processor (202) of FIG. 2. Additionally, blocks 310 to 350 may be implemented in the processor (202) of FIG. 2, and block 360 may be implemented in the transceiver (206) of FIG. 2, and are not limited to the embodiments described above.

[0109] A codeword can be converted into a wireless signal through the signal processing circuit (300) of FIG. 3. Here, the codeword is an encoded bit sequence of an information block. The information block may include a transmission block (e.g., UL-SCH transmission block, DL-SCH transmission block). Here, the information block may include AI-related data (e.g., training data, AI model data, input data, output data, etc.), and the codeword may be an encoded bit sequence corresponding to the AI-related data. The wireless signal may be transmitted through various physical channels (e.g., PUSCH, PDSCH). Specifically, the codeword may be converted into a scrambled bit sequence by scramblers (310). The scrambled sequence used for scrambling is generated based on an initialization value, which may include ID information of the wireless device, etc. The scrambled bit sequence may be modulated into a modulation symbol sequence by modulators (320). Modulation methods may include pi / 2-BPSK (pi / 2-binary phase shift keying), m-PSK (m-phase shift keying), m-QAM (m-quadrature amplitude modulation), etc.

[0110] A complex modulation symbol sequence can be mapped to at least one transmission layer by a layer mapper (330). Here, a transmission layer is a logical resource unit for mapping signals or data transmitted through spatial resources to antenna ports, and one transmission layer can correspond to one stream or one antenna port. Each complex modulation symbol included in the complex modulation symbol sequence is mapped to at least one transmission layer, thereby determining which antenna port it will be transmitted through. The modulation symbols of each transmission layer can be mapped to the corresponding antenna port(s) by a precoder (340). The output z of the precoder (340) can be obtained by multiplying the output y of the layer mapper (330) by an N-XM precoding matrix W, where N is the number of antenna ports and M is the number of transmission layers. Here, the precoder (340) can perform precoding after performing transform precoding (e.g., a discrete Fourier transform (DFT)) on the complex modulation symbols. Additionally, the precoder (340) can perform precoding without performing transform precoding.

[0111] Resource mappers (350) can map the modulation symbols of each antenna port to time-frequency resources. The time-frequency resources may include multiple symbols (e.g., CP-OFDMA symbols, DFT-s-OFDMA symbols) in the time domain and multiple subcarriers in the frequency domain. Signal generators (360) generate radio signals from the mapped modulation symbols, and the generated radio signals can be transmitted to other devices through each antenna. To this end, each of the signal generators (360) may include an inverse fast Fourier transform (IFFT) module, a cyclic prefix (CP) inserter, a digital-to-analog converter (DAC), a frequency uplink converter, etc.

[0112] The signal processing process for a received signal in a wireless device can be configured as the inverse of the signal processing process (310 to 360) of FIG. 3. For example, a wireless device (e.g., 200 in FIG. 2) can receive a wireless signal from the outside through an antenna port / transceiver. The received wireless signal can be converted into a baseband signal through a signal restorer. To this end, the signal restorer may include a frequency downlink converter, an analog-to-digital converter (ADC), a CP remover, and a fast Fourier transform (FFT) module. Subsequently, the baseband signal can be restored into a codeword through a resource de-mapper process, a postcoding process, a demodulation process, and a de-scrambling process. The codeword can be restored into the original information block through decoding. Accordingly, a signal processing circuit (not shown) for a received signal may include a signal restorer, a resource de-mapper, a postcoder, a demodulator, a de-scrambler, and a decoder.

[0113] The signal processing circuit (300) described with reference to FIG. 3 is illustrated as including a plurality of scramblers (310), modulators (320), a plurality of resource mappers (350), and a plurality of signal generators (360). However, at least one of the scramblers, modulators, resource mappers, and signal generators may be implemented as a single integrated structure. That is, the number of at least one of the scramblers, modulators, resource mappers, and signal generators may be less than the number of layers. Furthermore, at least one of the components illustrated in FIG. 3 may be omitted.

[0114]

[0115] FIG. 4 illustrates a communication procedure between a terminal and a base station applicable to the present disclosure. FIG. 4 illustrates the operation of a terminal (410) and a base station (420) transmitting and / or receiving data, and the operation performed prior to this.

[0116] Referring to FIG. 4, in step 401, the terminal (410) and the base station (420) perform synchronization. For example, the terminal (410) performs an initial cell search operation. Specifically, the terminal (410) can detect at least one synchronization signal transmitted from the base station (420) according to a predefined rule. Here, the synchronization signal may include a plurality of synchronization signals (e.g., primary synchronization signal, secondary synchronization signal) classified according to structure or use. Through this, the terminal (410) can identify the boundaries of the frame, subframe, slot, and / or symbol of the base station (420) and obtain information about the base station (420) (e.g., cell identifier).

[0117] In step 403, the terminal (410) obtains system information transmitted from the base station (420). The system information is information related to the attributes, characteristics, and / or capabilities of the base station (420) required to connect to the base station (420) and use the service, and can be classified according to content (e.g., whether it is essential for connection), transmission structure (e.g., channel used, whether it is provided on-demand), etc., and can be classified, for example, into a master information block (MIB) and a system information block (SIB). If necessary, the terminal (410) may transmit a signal requesting the system information prior to receiving the system information. The system information may include information related to AI functions. For example, the system information is information required for operations performed based on AI, and may include at least one of information related to an AI model, information related to training, and information related to inference / prediction. However, the request and provision of the system information may be performed after the random access procedure described later.

[0118] In step 405, the terminal (410) and the base station (420) perform a random access procedure. The terminal (410) may transmit and / or receive at least one message for the random access procedure (e.g., random access preamble, RAR (random access response) message, etc.) based on information related to the random access channel of the base station (420) obtained through system information (e.g., channel location, channel structure, structure of supported preamble, etc.). For example, the terminal (410) may transmit a preamble (e.g., MSG1) through the random access channel, receive a RAR message (e.g., MSG2), transmit a message (e.g., MSG3) containing information related to the terminal (410) (e.g., identification information) to the base station (420) using scheduling information included in the RAR message, and receive a message (e.g., MSG4) for contention resolution and / or connection establishment. As another example, MSG1 and MSG3 can be transmitted and received as a single message, or MSG2 and MSG4 can be transmitted and received as a single message.

[0119] In step 407, the terminal (410) and the base station (420) perform signaling of control information. Here, the control information may be defined in various layers, such as a layer that controls the connection (e.g., a radio resource control (RRC) layer), a layer that handles mapping between logical channels and transmission channels (e.g., a media access control (MAC) layer), and a layer that handles physical channels (e.g., a physical (PHY) layer). For example, the terminal (410) and the base station (420) may perform at least one of signaling to establish a connection, signaling to determine settings related to communication, and signaling to indicate allocated resources. Additionally, the signaling of control information may be performed to convey information related to AI functions. For example, information related to AI functions is information necessary for operations performed based on AI, and may include at least one of information related to an AI model, information related to training, and information related to inference / prediction. More specifically, the information related to the AI ​​function signaled in step 407 can be combined and / or combined with the information related to the AI ​​function signaled in step 403, and both can be defined as having a hierarchical, mutually complementary, or substitute structure.

[0120] In step 409, the terminal (410) and the base station (420) transmit and / or receive data. That is, the terminal (410) and the base station (420) can process, transmit and / or receive data based on the signaling of control information. For example, when transmitting data, the terminal (410) or the base station (420) may perform at least one of channel encoding, rate matching, scrambling, constellation mapping, layer mapping, waveform modulation, antenna mapping, and resource mapping on the information bits. Conversely, when receiving data, the terminal (410) or the base station (420) may perform at least one of signal extraction from resources, antenna-specific waveform demodulation, signal placement considering layer mapping, constellation demapping, descrambling, and channel decoding. Here, the transmitted data is AI-related data, and may include, for example, data for AI-based operations or data generated by AI-based operations.

[0121] Steps 401 through 409 described with reference to FIG. 4 must not necessarily be performed in the order exemplified in FIG. 4, and the order of at least some of the steps may vary. Additionally, at least some of steps 401 through 409 may be combined into a single step or omitted. That is, the steps exemplified in FIG. 4 may be performed in various modified forms.

[0122]

[0123] 6G communication systems and core implementation technologies of 6G systems

[0124] 5G systems define various operating bands within FR1 (frequency range 1), which includes 410 MHz to 7125 MHz, and FR2 (frequency range 2), which includes 24,250 MHz to 71,000 MHz. Various frequencies are being discussed as operating bands for subsequent 6G systems, and the use of frequencies higher than those of 5G systems is also being considered for wider bandwidth and higher transmission speeds. As one example, the use of the THz (Terahertz) frequency band, which includes approximately 100 GHz to 10 THz, is being discussed. The THz frequency band is a band that possesses both the penetrability of radio waves and the directivity of optical waves, and communication using the THz frequency band is expected to play a transitional role from existing radio-based communication to optical-based communication.

[0125] 6G systems utilizing the THz frequency band as described above aim for: i) very high data rates per device, ii) a very large number of connected devices, iii) global connectivity, iv) very low latency, v) reduced energy consumption of battery-free IoT devices, vi) ultra-reliable connectivity, and vii) connected intelligence with machine learning capabilities. The vision of 6G systems can be four aspects, such as "intelligent connectivity," "deep connectivity," "holographic connectivity," and "ubiquitous connectivity," and 6G systems can be designed to satisfy requirements such as those shown in [Table 1] below.

[0126] Per device peak data rate1 TbpsE2E latency1 msMaximum spectral efficiency100 bps / HzMobility supportup to 1000 km / hrSatellite integrationFullyAIFullyAutonomous vehicleFullyXRFullyHaptic CommunicationFully

[0127] At this time, the 6G system may have key factors such as enhanced mobile broadband (eMBB), ultra-reliable low latency communications (URLC), massive machine type communications (mMTC), AI integrated communication, tactile internet, high throughput, high network capacity, high energy efficiency, low backhaul and access network congestion, and enhanced data security. FIG. 5 illustrates an example of a communication structure that can be provided by a 6G system applicable to the present disclosure. Referring to FIG. 5, the 6G system is expected to have simultaneous wireless communication connectivity 50 times higher than that of a 5G wireless communication system. URLLC, a key feature of 5G, is expected to become an even more dominant technology in 6G communication by providing end-to-end latency of less than 1ms. In this case, 6G systems will have significantly superior volumetric spectral efficiency, unlike the frequently used area-spectral efficiency. Since 6G systems can provide very long battery life and advanced battery technology for energy harvesting, mobile devices in 6G systems may not need to be charged separately. New network characteristics in 6G may be as follows.

[0128] - Satellite Integrated Network: 6G is expected to be integrated with satellites to provide a global mobile population. Integrating terrestrial, satellite, and airborne networks into a single wireless communication system is crucial for 6G.

[0129] - Connected Intelligence: Unlike previous generations of wireless communication systems, 6G is innovative and will update wireless evolution from "connected things" to "connected intelligence." AI can be applied at each stage of the communication process (or at each step of the signal processing described below).

[0130] - Seamless integration of wireless information and energy transfer: 6G wireless networks will transfer power to charge the batteries of devices such as smartphones and sensors. Therefore, wireless information and energy transfer (WIET) will be integrated.

[0131] - Ubiquitous Super 3D Connectivity: Connectivity to the network and core network functions of drones and very low Earth orbit satellites will create Super 3D connectivity in 6G ubiquitous.

[0132] Some general requirements regarding the new network characteristics of 6G mentioned above may be as follows.

[0133] - Small cell networks: The idea of ​​small cell networks was introduced to improve the quality of received signals in cellular systems as a result of increased throughput, energy efficiency, and spectrum efficiency. Consequently, small cell networks are an essential feature of communication systems for 5G and beyond 5G (5GB). Therefore, 6G communication systems also adopt the characteristics of small cell networks.

[0134] - Ultra-dense heterogeneous network: Ultra-dense heterogeneous networks will be another important characteristic of 6G communication systems. Multi-tier networks composed of heterogeneous networks improve overall QoS and reduce costs.

[0135] - High-capacity backhaul: Backhaul connections are characterized as high-capacity backhaul networks to support high-volume traffic. High-speed fiber optics and free-space optics (FSO) systems can be possible solutions to this problem.

[0136] - Radar technology integrated with mobile technology: High-precision localization (or location-based services) through communication is one of the functions of 6G wireless communication systems. Therefore, radar systems will be integrated with 6G networks.

[0137] - Softwarization and virtualization: Softwarization and virtualization are two important features that form the basis of the design process in 5GB networks to ensure flexibility, reconfigurability, and programmability. Additionally, billions of devices can be shared across a shared physical infrastructure.

[0138] To satisfy the aforementioned characteristics, technologies such as artificial intelligence (AI), THz (Terahertz) communication, optical wireless technology, FSO backhaul network, massive MIMO technology, blockchain, 3D networking, quantum communication, unmanned aerial vehicles, cell-free communication, wireless information and energy transfer (WIET), integration of sensing and communication, integration of access backhaul networks, holographic beamforming, big data analysis, and large intelligent surface (LIS) may be adopted as core implementation technologies of the 6G system.

[0139] For example, THz communication can be utilized in 6G systems. THz communication is a communication that uses a spectrum in the frequency band between 0.3 THz and 3 THz with a corresponding wavelength in the range of 0.1 mm to 1 mm as shown in Fig. 6. Referring to Fig. 6, the frequency band of the THz wave is located in the intermediate region between the infrared band and the millimeter wave band; accordingly, the THz wave can be understood as a radio wave with the shortest wavelength and, at the same time, a light wave with the longest wavelength. As a result, the THz wave shares some characteristics of infrared and microwave waves, and specifically, can simultaneously possess the penetrability of electromagnetic waves and the directivity of light waves.

[0140]

[0141] FIG. 7 illustrates a transmitter structure applicable to the present disclosure.

[0142] Referring to Fig. 7, in order to modulate data onto an optical signal, the optical source of a laser can be passed through an optical wave guide to change the phase of the signal. At this time, data is loaded by changing electrical characteristics through a microwave contact, etc. Therefore, the optical modulator output is formed as a modulated waveform.

[0143] Data may be provided by a data signal generator. Here, the data may include various user data, configuration information, control information, etc. transmitted through a channel. Furthermore, the data may include data related to AI-based operations, for example, information for configuring an AI model, input / output data for tasks of an AI model, etc. To this end, components related to AI functions (e.g., an AI processing unit) may be included in the data signal generator or may interact with the data signal generator.

[0144] An O / E converter can generate THz pulses based on optical rectification by a nonlinear crystal, O / E conversion by a photoconductive antenna, emission from a bundle of relativistic electrons, etc. THz pulses generated in such a manner can have a length ranging from femtoseconds to picoseconds. The O / E converter performs down-conversion by utilizing the non-linearity of the device.

[0145] When considering the usage of the THz spectrum, it is highly likely that multiple contiguous GHz bands will be used for fixed or mobile service applications for THz systems. According to outdoor scenario criteria, available bandwidth can be classified based on an oxygen attenuation of 10^2 dB / km in the spectrum up to 1 THz. Accordingly, a framework in which the available bandwidth is composed of multiple band chunks can be considered. As an example of the above framework, if the length of the THz pulse for a single carrier is set to 50 ps, ​​the bandwidth (BW) becomes approximately 20 GHz.

[0146] Effective down-conversion from the infrared band to the THz band depends on how the nonlinearity of the photoelectric converter (O / E converter) is utilized. In other words, to achieve down-conversion to the desired THz band, it is required to design an O / E converter with the most ideal nonlinearity for transferring to that specific band. If an O / E converter that does not match the target frequency band is used, there is a high probability of errors occurring regarding the amplitude and phase of the corresponding pulse.

[0147] In a single-carrier system, a THz transceiver system can be implemented using a single photoelectric converter. Depending on the channel environment, in a multi-carrier system, as many photoelectric converters as there are carriers may be required. This phenomenon will be particularly pronounced in multi-carrier systems utilizing multiple broadbands according to the plans related to the aforementioned spectrum applications. In this regard, a frame structure for the multi-carrier system may be considered. A signal down-frequency converted based on a photoelectric converter can be transmitted in a specific resource region (e.g., a specific frame). The frequency domain of the specific resource region may include multiple chunks. Each chunk may consist of at least one component carrier (CC).

[0148]

[0149] AI technology can be introduced in 6G systems. Efficient resource management and optimization are required to maintain connectivity between various services and devices. AI technology may include techniques capable of performing data analysis, pattern recognition, and predictive modeling using AI / ML (artificial intelligence / machine learning) models. Here, an AI / ML model can be understood as a set of parameter values ​​and / or weight values ​​related to mathematical formulas or algorithms generated through learning, designed to discover patterns in input data or perform predictions. To create such an AI / ML model, an AI / ML model training procedure is required to build the model by learning the relationship between input and output in a data-driven manner. Various learning algorithms, such as supervised learning, unsupervised learning, and reinforcement learning, can be utilized as training algorithms. Users can input specific data into a trained AI / ML model to generate output, and the procedure of obtaining output data by inputting input data into the AI / ML model can be referred to as AI / ML 'inference' or 'prediction'.

[0150] Network control parameters can be obtained as output through AI / ML inference using trained AI / ML models. Users can improve network efficiency by utilizing these output parameter values. For example, AI technology can be applied in various fields, such as wireless network resource allocation, traffic management, fault prediction, and Quality of Service (QoS) management. In particular, machine learning can efficiently allocate resources even in dynamically changing network environments based on real-time data. Therefore, AI technology can be utilized to provide hyper-connectivity and ultra-low latency.

[0151] In this case, AI / ML inference can be performed based on a combination of various devices. For example, the UE and the network can jointly perform AI / ML inference, and such an AI / ML model may be referred to as a two-sided AI / ML model or a two-sided model. In this case, the UE may perform the first part of the inference first, and the base station may perform the remaining inference, and vice versa. As another example, all inference may be performed by the UE, and such an AI / ML model may be referred to as a UE-side AI / ML model or a UE-side model.

[0152] In addition, life cycle management (LCM) for AI / ML models can be performed. Life cycle management may include model training, model deployment, model inference, model monitoring, and model updating. To this end, support may be required for data collection, model training, functionality / model identification, model delivery / transfer, model inference operations, functionality / model selection / enable / disable / fallback, functionality / model monitoring, model updating, and UE capabilities.

[0153] For example, AI / ML technology can be operated based on a functional framework such as FIG. 8. FIG. 8 illustrates an example of a functional framework for the application of AI / ML technology applicable to the present disclosure. First, a data collection function (810) generates training data (801), monitoring data (803), and / or inference data (805) containing processed input data by performing data preparation on input data collected from objects (e.g., UE, RAN node, network node, etc.). A model training function (820), having received the training data (801) from the data collection function (810), performs training on an AI / ML model using the training data (801) and provides the trained / updated model (813) to a model repository (840). The model repository (840) can store and retain the received trained / updated model (813).

[0154] A management function (830) may be used to control AI / ML model training. The management function (830) may control the operation of AI / ML models or AI / ML functions, or supervise their performance. To this end, the management function (830) may receive monitoring data (830) from the data collection function (810) and receive inference output (809) from the inference function (840). The management function (830) performs the role of managing the inference operation so that it can be performed efficiently based on the data received from the data collection function (810) and the inference function (840). That is, the management function (830) may transmit performance feedback or a retraining request (807) to the model training function (820) to improve the inference operation. Here, the performance feedback may be used to direct the learning goal or as a reward for reinforcement learning. Additionally, the management function (830) can provide management instructions (811) that instruct the inference function (840) to select AI / ML models or AI / ML-based functions to use, enable / disable them, or switch to non-AI / ML operations.

[0155] The inference function (840) generates an inference output (809) by performing inference and / or prediction using inference data (805) received by the data collection function (810). Here, the inference output (809) refers to the inference output of the AI / ML model used by the inference function (840), and the details of the inference output may vary depending on the use case. The AI / ML model used by the inference function (840) can be controlled by the management function (830). That is, the management function (830) can transmit a model transmission / delivery request signal (815) to the model repository function (850) to request the necessary AI / ML model, and the model repository function (850) can transmit the corresponding AI / ML model to the inference function (840) via a model transmission / delivery signal (817). Thus, the inference function (840) can perform inference using the AI / ML model (817) according to the received management instructions (811).

[0156] Additionally, the management function (830) can trigger or perform a specified task / action based on the inference output (809). Thus, the management function (830) can trigger a task / action on other objects (e.g., at least one UE, at least one RAN node, at least one network node, etc.) or on itself. Any one of the functions exemplified in FIG. 8 described above may be performed by two or more entities among the RAN, network node, network operator's OAM, or UE in collaboration. This may be referred to as a split AI operation.

[0157] To use an AI / ML model, not all of the functions (810 to 850) illustrated in FIG. 8 must be used, and the method of combining them is not limited to a specific method. Therefore, the functions (810 to 850) may be operated in an integrated manner, or some functions may be omitted. Furthermore, the functions (810 to 850) illustrated in FIG. 8 are not necessarily limited to being implemented as separate devices or apparatuses. For example, some or all of the functions (810 to 850) may be included in the processor (202) of FIG. 2. Additionally, the model storage function (850) may be included in the memory (204) of FIG. 2.

[0158]

[0159] FIG. 9 illustrates an example of a procedure for utilizing an AI model applicable to the present disclosure. FIG. 9 illustrates a case where a model training function (820) is included in a network node and a model inference function (840) is included in a RAN node. Referring to FIG. 9, in step 1, RAN node 1 and RAN node 2 transmit input data (e.g., training data) for training an AI model to a network node. Here, RAN node 1 and RAN node 2 may also transmit data collected from a UE (e.g., UE measurements related to RSRP, RSRQ, SINR of a serving cell and neighboring cells, UE location, velocity, etc.) to the network node. In step 2, the network node trains the AI ​​model using the received training data. In step 3, the network node distributes / updates the AI ​​model to RAN node 1 and / or RAN node 2. RAN node 1 and / or RAN node 2 may continue to perform model training based on the received AI model. In this procedure, it is assumed that the AI ​​model is deployed / updated only to RAN Node 1. In Step 4, RAN Node 1 receives input data (e.g., inference data) for AI model inference from the UE and RAN Node 2. In Step 5, RAN Node 1 generates output data (e.g., prediction or decision) by performing AI model-based inference using the received inference data. In Step 6, if applicable, RAN Node 1 may transmit model performance feedback to the network nodes. In Step 7, RAN Node 1, RAN Node 2, and the UE (or 'RAN Node 1 and UE', or 'RAN Node 1 and RAN Node 2') perform an action based on the output data. For example, in the case of a load balancing action, the UE may move from RAN Node 1 to RAN Node 2. In Step 8, RAN Node 1 and RAN Node 2 transmit feedback information to the network nodes.

[0160] Network nodes can manage AI models based on feedback information regarding the inference results of AI models. For example, network nodes can perform additional training on AI models or generate additional information about AI models (e.g., performance information, accuracy information, etc.). If additional training is performed on AI models, network nodes can distribute the updated AI models to RAN node 1.

[0161] As explained with reference to Fig. 9, model training can be performed by network nodes, and inference using the model can be performed by RAN node 1. In other words, the model training and inference functions can be distributed. Generally, model training requires a large amount of computational resources because it involves optimization using large amounts of data and complex algorithms. In contrast, inference is a process of drawing conclusions about new data using an already trained model, and therefore requires relatively fewer computational resources compared to model training. Therefore, by using the procedure of Fig. 9, model training can be performed through network nodes if the computational resources of the UE or RAN nodes are insufficient. Additionally, security regarding the AI ​​model can be ensured because the AI ​​model is not exposed to the UE.

[0162] FIG. 9 illustrates a case where the model training function (820) is included in a network node and the model inference function (840) is included in a RAN node, but the present disclosure is not limited thereto. For example, if the computational resources of the RAN node are sufficient, both the model training function (820) and the model inference function (840) may be included in RAN node 1. In this case, RAN node 1 receives training data for training an AI model from the UE and RAN node 2. RAN node 1 trains the AI ​​model using the received training data. Subsequently, RAN node 1 receives inference data for AI model inference from the UE and RAN node 2. RAN node 1 generates output data by performing AI model-based inference using the received inference data. Based on the output data, the UE, RAN node 1, and RAN node 2 can perform communication-related operations (e.g., handover, cell change). Subsequently, the UE and RAN node 2 can transmit feedback regarding the operations to RAN node 1. Therefore, RAN Node 1 can train the AI ​​model and update the AI ​​model it will use through feedback information regarding the inference results of the AI ​​model. According to the aforementioned method, since signaling with the network is not required for AI model training and inference, the network load or the latency to train the AI ​​model or receive inference results can be reduced. Additionally, since the UE performs inference using the AI ​​model, the UE's personal information is not transmitted to network nodes, etc. Consequently, security regarding personal information can be enhanced.

[0163] As another example, the model training function (820) may be included in the RAN node, and the model inference function (840) may be included in the UE. The RAN node receives training data for training an AI model from the UE and trains the AI ​​model using the received training data. The RAN node distributes the trained AI model to the UE. The UE can generate output data by performing inference based on the received AI model. At this time, the data for inference can be received from the RAN node or data acquired by the UE itself can be used. The UE and the RAN node can perform communication-related operations based on the output data generated by inference. Subsequently, the UE can send feedback regarding the operation to the RAN node. Thus, through feedback information regarding the inference results of the AI ​​model, the RAN node can train the AI ​​model and distribute the updated AI model to the UE. According to the method described above, the load on the RAN node can be reduced by having the model training function (820) in the RAN node and the model inference function (840) in the UE perform inference. In addition, if the UE performs inference using data acquired independently, even if the UE loses its connection to the RAN node after receiving the AI ​​model, the UE can continue to perform inference using the received AI model and operate based on the inference results.

[0164] According to the aforementioned framework and procedure, an AI model can be trained and utilized in a wireless communication system. The model training function (820) and the model inference function (840) can be combined in various ways and are not necessarily limited to the case of FIG. 9. In the aforementioned framework and procedure, various data such as input data, training data, and inference data are introduced, and the specific details of the aforementioned data may vary depending on the task in which the AI ​​model is utilized. For example, information used in the various embodiments of the present disclosure described below may be included in the aforementioned data.

[0165]

[0166] FIG. 10 illustrates an AI technology-based communication procedure applicable to the present disclosure. The detailed procedure illustrated in FIG. 10 may be combined with various embodiments of the present disclosure described below. For example, data generated according to various embodiments of the present disclosure may be used for operations (e.g., setup, training, inference, and / or data transmission / reception) in at least one of the detailed procedure illustrated in FIG. 10. As another example, the result of the inference illustrated in FIG. 10 may be used to transmit and / or receive data according to various embodiments of the present disclosure.

[0167] Referring to FIG. 10, in step S1001, at least one of the UE (1010), RAN node (1020), and network node (1030) performs an initial connection procedure. For example, in this step, at least one of an initial cell search operation, a system information acquisition operation, a random access operation, and a registration operation may be performed. In step S1003, at least one of the UE (1010), RAN node (1020), and network node (1030) performs a configuration procedure. Through the configuration procedure, parameters, resources, connections, and / or entities necessary to perform subsequent procedures at layers between the UE (1010) and the RAN node (1020) and / or at least one layer between the UE (1010) and the network node (1030) may be determined and / or created. At this time, the configuration procedure may be performed based on information, status, and / or characteristics of the AI ​​model used for subsequent training and inference.

[0168] In step S1005, at least one of the UE (1010), RAN node (1020), and network node (1030) performs a model training procedure. At least one of the UE (1010), RAN node (1020), and network node (1030) may collect training data and perform training using the training data. For example, the model training procedure may be performed as described with reference to FIG. 15, FIG. 16, or FIG. 17. If an offline trained model is used, this step may be omitted.

[0169] In step S1007, at least one of the UE (1010), RAN node (1020), and network node (1030) performs a task using a trained model. That is, the task may be performed by the result of inference and / or prediction using the trained model. For example, the task may be a procedure belonging to a communication protocol, a preliminary operation for subsequent data transmission and / or reception, related to data transmission and / or reception, or related to data processing (e.g., encoding, decoding, etc.).

[0170] In step S1009, at least one of the UE (1010), RAN node (1020), and network node (1030) transmits and / or receives data. At this time, the result of the task performed in step 1007 may be used. In some cases, the task performed in step 1007 may include the transmission and / or reception of data, in which case this step may be omitted as it is part of step 1007.

[0171]

[0172] Specific embodiments of the present disclosure

[0173] The present disclosure relates to a technology for increasing data transmission rate efficiency by pre-processing / post-processing source data for semantic communication in a wireless communication system. Specifically, the present disclosure relates to a technology for performing pre-processing and / or post-processing on at least a portion of source data based on the degree to which it affects the result of a task when performing a task using source data transmitted via semantic communication.

[0174]

[0175] Problems related to communication can be divided into three levels as illustrated in FIG. 11, according to the philosophy of Shannon and Weaver. FIG. 11 illustrates a communication model applicable to the present disclosure. The problem of Level A (1110) is a technical problem related to how accurately symbols in communication can be transmitted, and the problem of Level B (1120) is a semantic problem related to how accurately the transmitted symbols convey the desired meaning. The problem of Level C (1130) is an effectiveness problem related to how effectively the received meaning influences the operation in the desired manner.

[0176] Shannon's information theory focuses only on technical problems at Level A (1110). However, Weaver explains that if semantic transmitters, semantic receivers, and semantic noise are added to Shannon's communication model, Shannon's information theory is general enough to account for problems at Level B (1120) and Level C (1130).

[0177] Meanwhile, one of the various goals of 6G communication is to enable various new services that interconnect people and machines. Therefore, in addition to considering only the technical problems of Level A (1110) in the communication system, it is necessary to provide a semantic communication method that considers the semantic problems of Level B (1120). Semantic communication means that a first device and a second device corresponding to the source and destination, respectively, efficiently transmit and receive semantic information using background knowledge, which is common information. Referring to the communication model of FIG. 11, if the meaning of the intended message sent by the first device corresponding to the source is accurately interpreted by the second device corresponding to the destination, it can be said that correct semantic communication has been performed.

[0178] For semantic communication, a source can generate a semantic representation based on given or collected raw data and transmit the generated semantic representation to a destination. The destination interprets and reasons the received semantic representation according to the source's intent. In this context, semantic communication requires an approach that considers not the reduction of reconstruction errors occurring during the process of restoring the received semantic representation to the original raw data, but rather whether the downstream task—the task performed at the destination—can operate according to the source's intent using the received semantic representation. Therefore, the destination utilizes its own background knowledge when performing reasoning operations, and it is desirable that the background knowledge contained in the data transmitted from the source be reflected in the destination's background knowledge to obtain correct interpretation results. In other words, since the destination performs interpretation and reasoning operations on the semantic representation using background knowledge, it is necessary for the source and the destination to share the same background knowledge.

[0179]

[0180] NTSCC (nonlinear transform source-channel coding)

[0181] Among existing technologies capable of performing semantic communication, NTSCC-based technology is currently known as the state-of-the-art (SOTA) technology. NTSCC technology is a fusion of compression technology based on nonlinear transform coding (NTC) and deep learning-based Deep-Joint Source-Channel Coding (Deep-JSCC). Deep-JSCC technology can be explained by comparing it to general communication as follows.

[0182] FIG. 12a illustrates a transmitter and a receiver for general communication. Referring to FIG. 12a, in the case of general communication, the transmitter [receives] source data to be transmitted The number of representation bits of the source data is reduced by source encoding using a source encoder. This can be understood as a type of compression effect. Additionally, the transmitter provides robustness against channel noise by performing channel encoding using a channel encoder. Subsequently, the transmitter generates symbols using a modulator and transmits them to the receiver through a noisy channel. The receiver demodulates the signal received through the noisy channel using a demodulator, and, reversing the procedure performed by the transmitter, uses a channel decoder to overcome noise in the received data and a source decoder to reconstruct the result of the source data. Acquires.

[0183] FIG. 12b illustrates a transmitter and a receiver applying Deep-JSCC technology. Referring to FIG. 12b, Deep-JSCC technology [applies] to source data to be transmitted from the transmitter It passes only through the joint source-channel encoder and is transmitted immediately through the noise channel. In other words, the joint source-channel encoder can simultaneously perform the roles of a source encoder, a channel encoder, and a modulator. At the receiver, the signal received through the noise channel passes through the joint source-channel decoder, through which the result of reconstructing the source data is obtained immediately ...is obtained. That is, the joint source-channel decoder performs the roles of source decoder, channel decoder, and demodulator. To design such a joint source-channel encoder / decoder, a deep neural network (DNN) structure can be used.

[0184] Figure 13 illustrates an example of a DNN structure for implementing Deep-JSCC. Referring to Figure 13, the joint source-channel encoder / decoder uses a DNN structure such as a convolutional network or a fully connected network. To train the DNN structure, for example, A mean squared error (MSE) loss function such as the one above can be used. When performing communication based on a joint source-channel encoder / decoder trained using the MSE loss function, the transmitted source data and data restored from the receiver A low MSE can be obtained, which means that high PSNR (peak signal-to-noise ratio) performance can be achieved.

[0185]

[0186] Next, the nonlinear transform coding (NTC) technique is described as follows. Figure 14 illustrates a block diagram of the NTC technique. Referring to Figure 14, The block is an analysis transform block, and the source data latent variable from Extracts. The block can be implemented as a DNN. The block is an analysis transform for side-information block, and the latent variable Side information for obtaining mean and variance information about Extracts. The block can be implemented as a DNN. The block is an entropy encoding block that performs lossless compression by performing entropy encoding on the input data. The block is a synthesis transform block, and the latent variable Original source data based on Reconstructs. The block can be implemented as a DNN. Blocks are side information using latent variables Estimate the average value for . The block can be implemented as a DNN. Blocks are side information using latent variables Estimate the variance or standard deviation for. The block can be implemented as a DNN. The block performs a rounding operation on the input value.

[0187] For example, based on the aforementioned NTC structure, Training can be performed based on a loss function such as the one shown. In the exemplified loss function, is a value(s) that does not depend on the DNN parameters forming the NTC block diagram and can be ignored during the training process. Also, In a given state, always Since the value is determined deterministically, The value can be treated as a value of 0. is a metric indicating distortion between the data of the transmitter and receiver, and which can occur when using side information It is the rate of, and Is It is the rate of.

[0188] As another example, based on the aforementioned NTC structure, Training can be performed based on a loss function such as the one shown. In the exemplified loss function, Is It is the rate of, Is It is the rate of, is an indicator of distortion between the data of the transmitter and receiver.

[0189] Next, let's examine the NTSCC technology, which fuses Deep-JSCC and NTC. Figure 15 illustrates an example of the structure of the NTSCC technology that fuses Deep-JSCC and NTC. In Figure 15, it can be understood that the area inside the dotted lines follows the NTC structure, while the area outside the dotted lines follows the Deep-JSCC structure. The functions of the blocks in the part following the NTC structure are as follows.

[0190] The block is an entropy encoding block, of the NTC structure. It performs the same role as a block. The block is a channel encoding block, Channel coding is performed on the block's output to increase robustness to the channel. A block is a channel block, meaning a channel between a transmitter and a receiver, and can be defined as an AWGN channel, etc. The block is an entropy encoding block, of the NTC structure. It performs the same role as a block. The block acts as a channel decoding block and performs channel decoding for the signal received through the channel. Blocks are latent variables As a block determining the mean and variance or standard deviation for, of the NTC structure Block and It simultaneously performs the role of a block.

[0191] For example, training for NTSCC is, It can be performed based on a loss function such as. Here, , , , It means. The value has a meaning similar to the rate in NTC. in a given situation It refers to the channel bandwidth cost for, Is It refers to the channel bandwidth cost for, means distortion of source data between the transmitter and the receiver.

[0192] The channel bandwidth cost of a latent variable by NTSCC is visualized as shown in FIGS. 16a to 16d. FIGS. 16a to 16d illustrate examples of the channel bandwidth cost of a latent variable by NTSCC. Referring to FIGS. 16a to 16d, it is confirmed that the channel bandwidth cost is high for parts of the source image data where pixel values ​​change rapidly. In addition, It is confirmed that when the value is larger (e.g., when the loss function is trained to use less channel bandwidth cost), the allocation of channel bandwidth cost to the boundary portions of the image data is performed preferentially. On the other hand, When the value is somewhat low, it is confirmed that the allocation of channel bandwidth costs for regions other than the boundary portions of the image data is performed.

[0193] Comparing PSNR and CBR (channel bandwidth ratio) using various images can yield results such as those shown in FIGS. 17a through 17c. FIGS. 17a through 17c illustrate examples of loss, PSNR, and CBR values ​​for various images. Referring to FIGS. 17a through 17c, it is confirmed that PSNR and CBR values ​​vary depending on the image. The image exemplified in FIG. 17a has the lowest CBR value and, consequently, the highest PSNR value. The image exemplified in FIG. 17c has the highest CBR value and, consequently, the lowest PSNR value. The reason the image exemplified in FIG. 17a has the lowest CBR value is that the background of the image is blurred; that is, there are relatively few areas where the image pixel values ​​change abruptly. Conversely, in the case of the image exemplified in FIG. 17c, there are relatively many areas where pixel values ​​change abruptly, so the image has a relatively high CBR value.

[0194] The performance of the NTSCC technology is shown in Figures 18a and 18b. Figures 18a and 18b illustrate the performance of the NTSCC technology. Figure 18a is a graph comparing PSNR vs. CBR performance for NTSCC and other techniques, and Figure 18b is a graph comparing MS-SSIM (multi-scale structural similarity index method) vs. CBR performance for NTSCC and other techniques. Referring to Figures 18a and 18b, it is confirmed that the NTSCC technique has higher PNSR or MS-SSIM performance than the BPG+LDPC, NTC+LDPC, and Deep-JSCC techniques based on the same CBR value. Here, BPG+LDPC is the case where the 5G LDPC channel coding technique is applied to the BPG source coding technique, and NTC+LDPC is the case where the 5G LDPC channel coding technique is applied to the NTC technique. In the PSNR vs. CBR performance graph, BPG+Capacity and NTC+Capacity represent cases where ideal channel coding satisfying channel capacity is applied to each source coding technique. Therefore, although the NTSCC technique appears to have slightly lower PSNR vs. CBR performance than the BPG+Capacity and NTC+Capacity techniques, it actually demonstrates better performance than the BPG+LPDC or NTC+LDPC techniques, making it a SOTA technique. In the case of MS-SSIM performance, it is confirmed that the NTSCC technique has better MS-SSIM performance than BPP+Capacity.

[0195] As shown in Figures 18a and 18b, to demonstrate various CBR versus PSNR or MS-SSIM performance, an NTSCC DNN structure trained for each CBR is required. That is, the Lagrange multiplier used in the loss function for training the NTSCC DNN structure By adjusting the values, different DNN structures can be generated. For example, When =256, an NTSCC DNN structure can be generated at the performance point with the lowest CBR in the PSNR or MS-SSIM versus CBR graph. When =4, an NTSCC DNN structure can be generated at the performance point with the highest CBR in the PSNR or MS-SSIM versus CBR graph. Therefore, NTSCC technology has the disadvantage that multiple NTSCC DNN structures must be constructed to adapt to various CBRs.

[0196]

[0197] Task-oriented semantic communication

[0198] A framework for performing task-oriented semantic communication can be configured in various structures. First, an example of a general framework for performing task-oriented semantic communication is shown in FIG. 19. FIG. 19 illustrates an example of a framework for performing task-oriented semantic communication. Referring to FIG. 19, in the case of a general framework for task-oriented semantic communication, the transmitter converts original data into a semantic representation through a semantic encoder and transmits the semantic representation to the receiver through a channel. The receiver performs a task using the received corrupted semantic representation and obtains a task output. That is, the receiver can perform the task directly based on the semantic representation without performing reconstruction on the received semantic representation. In the case of such general task-oriented semantic communication frameworks, it is necessary for the tasking module and semantic encoder to be trained under the assumption of an appropriate channel so that the task can be performed immediately using the semantic representation received at the receiver. In other words, there is a disadvantage regarding system flexibility in that high task performance can only be guaranteed if end-to-end learning is performed. However, when configuring such a framework, there is an advantage in that optimal performance can be obtained in terms of transmission and reception resource efficiency because only the semantics necessary to perform the task can be effectively extracted and transmitted and received as semantic representations.

[0199]

[0200] Next, a reconstruction-based framework for performing task-oriented semantic communication is described as follows. Figure 20 illustrates an example of a reconstruction-based framework for performing task-oriented semantic communication. Referring to Figure 20, in the case of a reconstruction-based task-oriented semantic communication framework, a semantic representation transmitted from a transmitter is received by a receiver via a channel, and the receiver performs reconstruction first by taking the received semantic representation as input to a semantic decoder. Subsequently, the reconstructed data is input into a task module, and the receiver performs the task. This reconstruction-based task-oriented semantic communication has the advantage of allowing the task module to use the existing trained DNN structure as is. This advantage can lead to the ability to enable adaptive design for the task-oriented semantic communication system. However, when performing task-oriented semantic communication based on restoration, the efficiency of resources for transmission and / or reception may be lower than that of a general framework because even information not needed when performing the actual task must be transmitted and / or received for restoration.

[0201] As a means to enhance the advantages of system fluidity and resource efficiency in restoration-based task-oriented semantic communication, this disclosure proposes a pre- / post-processing technique capable of adapting semantic encoder / decoder technologies, such as NTSCC, where CBR can vary depending on source data, without the need for multiple DNN structures. The proposed technique improves transmission efficiency while maintaining the advantages of restoration-based task-oriented semantic communication by adjusting transmission efficiency for parts where the target task performed at the destination can operate well, based on the attention map of the source data. In other words, this disclosure proposes a technique that can further improve task performance under fixed CBR resource efficiency conditions when performing task-oriented semantic communication.

[0202]

[0203] The present disclosure proposes a technique capable of enhancing task performance by performing pre-processing based on blurring and sharpening of source data at a transmitter and performing deblurring and inverse sharpening of received data at a receiver. The semantic communication framework proposed in the present disclosure may be represented as shown in FIG. 21. FIG. 21 illustrates an example of a framework for semantic communication according to an embodiment of the present disclosure.

[0204]

[0205] 1) Attention map generation

[0206] When using a transformer-based DNN structure, an attention map for the target task of the source data can be obtained. The attention map is an area that significantly influences the performance result of the target task and can be specified as shown in Fig. 22. Fig. 22 illustrates an example of an attention map for an image. Referring to Fig. 22, an example of an attention map for a classification task of image data is shown. Areas with high values ​​can be interpreted as image parts that have a greater influence when performing the classification task, while areas with low values ​​can be interpreted as image parts that have less influence when performing the classification task. In this case, the threshold T for the attention map att Determine the value, and the threshold T att Parts with attention values ​​greater than the value are converted to 1, and the threshold T att The attention map can be quantized by converting parts with attention values ​​smaller than the value to 0. In the proposed technique, the quantized attention map can be used to perform pre-processing or post-processing.

[0207] Here, one or multiple thresholds may be defined. In this case, intervals may be defined with adjacent thresholds as boundaries, and for each interval, an attention map quantized with values ​​such as 0, 1, 2, 3, … may be generated. The quantized attention map may be used to perform pre-processing or post-processing, and specific operations for using it are described in detail below.

[0208] Since the target task is performed at the receiver, if the transmitter is to generate an attention map, the transmitter may also possess a transformer-based DNN to perform the target task. In this case, the transmitter can perform a prediction operation using the transformer-based DNN and acquire the attention map generated during the prediction operation. For example, the attention map can be acquired by calculating the correlation of embedding vectors input to the transformer. At this time, the attention map can be generated based on the attention values ​​of patches for embedding vectors or embedding tokens for the task to be performed using the transformer. If the attention map acquired in this way includes attention values ​​at the patch level, attention values ​​at smaller units (e.g., pixels) can be determined through interpolation.

[0209]

[0210] 2) Blurring and sharpening for pre-processing

[0211] The transmitter preprocesses the source data to be sent to aid in the execution of the destination's target task and transmits it. In this process, an attention-based transformer-based DNN structure can be used. The attention map may contain quantized attention values ​​based on a specific threshold.

[0212] The process of performing preprocessing on image data by considering a quantized attention map using a single threshold is as follows. The examples presented below are for illustrative purposes only; the scope of the proposed technology is not limited to using a single threshold, nor is the proposed technology applicable only when the data type is an image. First, the transmitter acquires an attention map for the source data to be transmitted and generates a quantized attention map consisting of 0s and 1s by quantizing based on a specific threshold. Subsequently, the transmitter constructs patches by dividing the source data into regions of a specific size. An example of the divided source data is shown in FIG. 23. FIG. 23 is an example of source data divided into a plurality of patches according to an embodiment of the present disclosure. In the case of FIG. 23, the size of the patch of the image The transmitter designates a patch that does not contain 1 of the attention map among the patches obtained as in FIG. 23 as a blurring region and a patch that contains 1 of the attention map as a sharpening region. Through this, each of the patches can be designated as a blurring region and a sharpening region. In the present disclosure, information indicating the blurring region or sharpening region designated for each patch may be referred to as 'RI (region information)', 'patch information', 'blurring indicator', 'sharpening indicator', 'blurring / sharpening indicator', or other terms having an equivalent technical meaning.

[0213] Based on RI, pre-processing is performed using a blurring matrix for the blurring region and a sharpening matrix for the sharpening region. For example, the blurring matrix and the sharpening matrix may be square matrices having the form shown in FIGS. 24a and FIGS. 24b. FIGS. 24a shows an example of a blurring matrix, and FIGS. 24b shows an example of a sharpening matrix. In this case, the blurring matrix may be composed of a matrix in which the norm value is smaller than the norm of the identity matrix (e.g., a value of 3 for a 3×3 square matrix), and the sharpening matrix may be composed of a matrix in which the norm value is larger than the norm of the identity matrix. Additionally, it is desirable that the blurring matrix and the sharpening matrix be matrices that have an inverse matrix to enable inverse blurring and inverse sharpening.

[0214] Preprocessing can be performed as shown in FIG. 25 using a blurring matrix and / or a sharpening matrix. FIG. 25 illustrates an example of a preprocessing operation according to one embodiment of the present disclosure. FIG. 25 illustrates preprocessing using blurring and / or sharpening for a single patch. Referring to FIG. 25, each patch of image data has a size Therefore, a single patch is length through vectorization. It is converted into vector data. By multiplying the blurred area by the blurring matrix and the sharpening area by the sharpening matrix, the vector data is again It is rearranged to the size of and converted into a blurred or sharpened patch. Similar to the operation described above, blurring and / or sharpening using RI information can be performed on all patches.

[0215] The receiver performs inverse blurring and inverse sharpening. This results in a decrease in the CBR value when using a semantic encoder such as NTSCC for blurred patches. For sharpened patches, it results in an increase in the CBR value. Accordingly, for blurred patches, the receiver can perform reception with lower PSNR or MS-SSIM performance, and for sharpened patches, the receiver can perform reception with higher PSNR or MS-SSIM performance. Inverse blurring and inverse sharpening are described in detail below.

[0216] If there is one blurring matrix and one sharpening matrix agreed upon between the transmitter and the receiver, the receiver can distinguish between the blurring and sharpening regions using only RI information consisting of 0s and 1s; therefore, the transmitter needs to additionally transmit RI information consisting of 0s and 1s to the receiver. However, if two or more blurring matrices and sharpening matrices are agreed upon between the transmitter and the receiver, it is necessary to construct RI information according to the index of each matrix and transmit it to the receiver, rather than distinguishing RI information only by 0s and 1s. For example, as shown in [Table 2] below, it can be assumed that the transmitter and the receiver share matrices.

[0217] Blurring / Inverse Blurring Matrix Sharpening / Inverse Sharpening Matrix 0 / 2 / 1 / 3 /

[0218] In this case, RI includes at least one value consisting of 0, 1, 2, or 3; to achieve this, when performing quantization on the attention map, it is necessary to perform quantization using three thresholds instead of a single threshold. For each patch, in order of highest to lowest attention map values , , , Pre-processing can be applied in the order of . By transmitting RI information consisting of 0, 1, 2, or 3 to the receiver, the receiver can obtain blurring and / or sharpening matrix information applied to the patches. That is, RI may include elements indicating the index of a pre-processing or post-processing operation applied to each of the patches included in the data. To this end, information regarding the mapping relationship between RI and pre-processing / post-processing operations may be shared between the transmitter and the receiver. Here, the information regarding the mapping relationship may indicate the matrix to be applied according to the value of an element included in RI (e.g., a quantization result value per patch).

[0219] In transmitting RI information configured as described above from a transmitter to a receiver, additional resources may be consumed. When the source data is an image, the CBR value required to transmit and / or receive RI information can be determined as shown in [Equation 1] below.

[0220]

[0221] In [Mathematical Formula 1], is the total number of blurring / de-blurring matrices and sharpening / de-sharpening matrices agreed upon between the sender and / or receiver, is channel SNR, represents the patch size. That is, as the number of matrices agreed upon between the transmitter and receiver increases, the CBR value required for RI transmission and reception increases. Additionally, as the channel SNR and patch size increase, the CBR value decreases.

[0222] 3) Reverse blurring and reverse sharpening for post-processing

[0223] The receiver performs post-processing operations using inverse blurring and inverse sharpening. The receiver performs attention-based restoration of the original source data using semantic representations and RI information received from the transmitter through the channel. First, the post-processing in the case using an attention map generated using a single threshold and an RI generated based thereon is as follows. The example described below is for illustrative purposes only, and the scope of the proposed technology is not limited to using a single threshold, nor is it applicable only to image-type data.

[0224] First, the receiver performs image data restoration based on semantic representation using a semantic decoder such as an NTSCC decoder. Through this, the image is a blurred / sharpened image based on RI information. Therefore, the receiver can obtain the original image using the RI information and the inverse-blurring / inverse-sharpening matrix. Inverse-blurring and inverse-sharpening using the inverse-blurring / inverse-sharpening matrix can be performed as shown in FIG. 26. FIG. 26 illustrates an example of a post-processing operation according to an embodiment of the present disclosure. Referring to FIG. 26, the receiver vectorizes each patch of the blurred / sharpened image obtained as the output of the semantic decoder, thereby length It is converted into vector data. Then, the blurred region is multiplied by an inverse-blurring matrix and the sharpened region is multiplied by an inverse-sharpening matrix, so that each patch is again It is rearranged to the size of and finally restored.

[0225] Blurred regions of the restored image exhibit lower PSNR or MS-SSIM performance compared to when transmitting and receiving without blurring. Conversely, sharpened regions exhibit higher PSNR or MS-SSIM performance compared to when transmitting and receiving without sharpening. Since sharpened regions refer to parts of the attention map that have a greater impact on task execution, improving the data quality of these sharpened regions ultimately leads to enhanced task performance.

[0226] In addition, since the CBR for sharpened areas can be increased while the CBR for blurred areas can be lowered, it is possible to perform transmission and reception while maintaining the CBR value required for data transmission by properly adjusting these characteristics. Through this, it is possible to achieve higher task performance in terms of task execution performance even when the CBR is fixed, compared to when only conventional semantic encoders / decoders are used.

[0227]

[0228] 4) CBR and task performance adaptation

[0229] The proposed technique described above can control the transmitted CBR value by adjusting the threshold and matrix when performing quantization on the attention map. For the convenience of explanation, the process of controlling the CBR when the threshold is one is described below.

[0230] First, the threshold If set to, the set threshold by Blurring is performed on parts with attention values ​​smaller than the value, and Sharpening can be performed on parts that have an attention value greater than the value. For example, RI is generated based on the attention values, and blurring and / or sharpening can be performed based on RI. The CBR for the blurred parts is reduced compared to when no blurring is performed, and the CBR for the sharpened parts is increased compared to when no sharpening is performed.

[0231] If you wish to further reduce CBR, it is possible to increase the area requiring blurring and decrease the area requiring sharpening by increasing the threshold required for RI settings. That is, human threshold If you set it, The part with a smaller attention value increases, and Since the portion with a larger attention value is reduced, the overall CBR value can be reduced by utilizing this.

[0232] Also, the same threshold It is possible to further reduce CBR by adjusting the blurring and sharpening matrices used in situations where [it] is used. For example, the currently selected blurring matrix and sharpening matrix Blurring matrix with a lower Frobenius norm value and sharpening matrix The overall CBR value can be reduced by using . Here, and It can satisfy.

[0233] To improve task execution performance, it is possible to use the aforementioned method in reverse. For example, the threshold Bigger than By quantizing the attention map using , task performance can be improved instead of enhancing CBR. In addition, Sharpening matrix satisfying By using , task performance can be improved.

[0234]

[0235] A procedure for performing pre-processing and post-processing according to the proposed technology described above is as shown in FIG. 27. FIG. 27 illustrates an example of a procedure for performing pre-processing and post-processing according to an embodiment of the present disclosure. Referring to FIG. 27, to perform pre-processing / post-processing, a transmitter and a receiver may each possess a blurring matrix and a sharpening matrix, and an inverse blurring matrix and an inverse sharpening matrix paired therewith, as background knowledge. Based on this, the transmitter may generate a pre-processed image using an attention map-based RI, generate a semantic representation using a semantic encoder such as an NTSCC encoder, and transmit it to the receiver along with the RI. The receiver may perform post-processing using the restored data using a semantic decoder such as an NTSCC decoder. Through this, the receiver can obtain a task output by performing a task using the finally obtained data.

[0236]

[0237] To explain the various embodiments described above, the case where the data is an image is presented as an example. When the data is an image, a patch, etc., is defined as an area containing a certain number of pixels. However, for other types of content, the data can be interpreted in units of embedding vectors, and a patch can be defined as embedding vector(s) of a certain size. Additionally, blurring / sharpening operations are operations that adjust the amount of change in data values ​​to be greater or lesser, and can be interpreted as operations that adjust the amount of change in data values ​​even when the data is not an image.

[0238]

[0239] To perform the pre-processing / post-processing proposed in the present invention, signaling such as that shown in FIG. 28 may be performed between a first device (e.g., source) and a second device (e.g., destination). FIG. 28 illustrates an example of a procedure for performing semantic communication according to one embodiment of the present disclosure. FIG. 28 illustrates a signal exchange between a first device (2810) operating as a transmitter and source and a second device (2820) operating as a receiver and destination.

[0240] Referring to FIG. 28, in step S2801, the first device (2810) and the second device (2820) perform initialization for semantic communication. For example, the first device (2810) determines at least one parameter related to semantic communication and transmits setting information including the determined at least one parameter to the second device (2820). Specifically, in the initialization step, the first device (2810) and the second device (2820) determine a patch size setting, a number of thresholds and a blurring matrix and a sharpening matrix accordingly, and perform signaling for the determined settings.

[0241] In step S2803, the second device (2820) transmits information about the target task to the first device (2810). In other words, the second device (2820) transmits information about the target task to the first device (2810).

[0242] In step S2805, the first device (2810) selects a region for pre-processing. For example, the first device (2810) can generate an attention map of the data to be transmitted using information of the received target task, and generate an RI based on the attention map.

[0243] In step S2807, the first device (2810) performs blurring and / or sharpening on the input data. That is, the first device (2810) can perform blurring and / or sharpening on the data based on the generated RI.

[0244] In step S2809, the first device (2810) performs semantic encoding. For example, the first device (2810) may perform semantic encoding using a semantic encoder such as an NTSCC encoder. By performing semantic encoding, the first device (2810) may generate semantic symbols containing semantic representations.

[0245] In step S2811, the first device (2810) transmits semantic symbols and RI to the second device (2820). To do this, the first device (2810) may generate data symbols by performing channel encoding and modulation on the RI, map the semantic symbols and data symbols to resources, and transmit a signal containing the semantic symbols and data symbols.

[0246] In step S2813, the second device (2820) performs semantic decoding. In other words, the second device (2820) performs semantic decoding on the received semantic symbols using a semantic decoder such as an NTSCC decoder. Through this, the second device (2820) can obtain data. Here, the data is in a blurred and / or sharpened state.

[0247] In step S2815, the second device (2820) performs reverse-blurring and / or reverse-sharpening on the received data. At this time, the second device (2820) may use the received RI. That is, the second device (2820) can obtain restored data by performing reverse-blurring and sharpening on the data using the output and RI information obtained in step S2813.

[0248] In step S2817, the second device (2820) performs a task. That is, the second device (2820) can perform a target task using the restored data.

[0249]

[0250] FIG. 29 illustrates an example of a procedure for performing semantic communication of a terminal according to one embodiment of the present disclosure. FIG. 29 illustrates a method performed by a terminal that generates and transmits a semantic representation.

[0251] Referring to FIG. 29, in step S2901, the terminal performs an initial connection procedure. The terminal detects a synchronization signal from a base station, receives system information, and can perform a random access procedure. Specifically, the terminal obtains information related to a random access channel included in the system information and can transmit a random access preamble based on the obtained information. At this time, the terminal can select a random access channel occasion (RO) for transmitting the random access preamble based on the detected synchronization signal.

[0252] In step S2903, the terminal transmits capability information. The capability information includes information indicating the hardware or software capabilities of the terminal. Although not illustrated in FIG. 29, the terminal may receive a request message for capability information from a base station and transmit a message containing capability information in response to the request message. According to one embodiment, the capability information may be information related to semantic communication and may include at least one of the following: whether task-oriented semantic communication is supported, whether restoration-based semantic communication is supported, whether pre-processing / post-processing is supported, and information related to available pre-processing / post-processing matrices. According to another embodiment, information related to semantic communication may be transmitted through a procedure other than capability information. For example, information related to semantic communication may be transmitted through the signaling of step S2905 below.

[0253] In step S2905, the terminal performs signaling for configuration related to communication. For example, the terminal may perform control signaling with a base station for configuration necessary to perform communication. According to one embodiment, the terminal may transmit and / or receive configuration information for initializing semantic communication. Specifically, the terminal may receive from the base station configuration information related to at least one of the following: a patch size for pre-processing / post-processing of data for semantic communication, the number of threshold(s) for quantizing an attention map, the value of the threshold(s) for quantizing an attention map, a matrix for reducing the change in data values ​​(e.g., a blurring matrix), a matrix for increasing the change in data values ​​(e.g., a sharpening matrix), RI, and a mapping relationship between pre-processing / post-processing operations. That is, the terminal may receive configuration information necessary for semantic communication according to various embodiments of the present disclosure.

[0254] In step S2907, the terminal performs preprocessing on the data and generates a semantic representation. Specifically, the terminal generates an attention map for the data based on the target task and generates an RI that indicates the area where preprocessing operations are performed based on the attention map. Then, the terminal can generate a semantic representation by performing preprocessing (e.g., blurring and / or sharpening) based on the RI and performing semantic encoding on the preprocessed data. At this time, the preprocessing is an operation to increase or decrease the amount of change of data values ​​on a patch-by-patch basis, and can be performed to decrease the amount of change of data values ​​belonging to patch(s) that have a relatively small impact on the task performance identified by the attention map, or to increase the amount of change of data values ​​belonging to patch(s) that have a relatively large impact on the task performance identified by the attention map.

[0255] In step S2909, the terminal transmits a semantic expression and an RI. At this time, the semantic expression and the RI may be transmitted as a single transmission block. Alternatively, the semantic expression and the RI may be transmitted as separate transmission blocks. Alternatively, the semantic expression may be transmitted as a transmission block and the RI as control information.

[0256]

[0257] FIG. 30 illustrates an example of a procedure for performing semantic communication of a base station according to one embodiment of the present disclosure. FIG. 30 illustrates a method performed by a base station receiving a semantic representation.

[0258] Referring to FIG. 30, in step S3001, the base station performs an initial connection procedure. The base station can transmit a synchronization signal to the terminal, transmit system information, and perform a random access procedure. Specifically, the base station can transmit information related to the random access channel through the system information and detect a random access preamble transmitted from the terminal based on the information.

[0259] In step S3003, the base station receives capability information. The capability information includes information indicating the hardware or software capabilities of the terminal. Although not illustrated in FIG. 30, the base station may transmit a request message for capability information to the terminal and receive a message containing the transmitted capability information in response to the request message. According to one embodiment, the capability information may be information related to semantic communication and may include at least one of the following: whether task-oriented semantic communication is supported, whether restoration-based semantic communication is supported, whether pre-processing / post-processing is supported, and information related to available pre-processing / post-processing matrices. According to another embodiment, information related to semantic communication may be received through a procedure other than capability information. For example, information related to semantic communication may be received through the signaling of step S3005 below.

[0260] In step S3005, the base station performs signaling for settings related to communication. For example, the base station may perform control signaling with the terminal for settings necessary to perform communication. According to one embodiment, the base station may transmit and / or receive setting information for initializing semantic communication. Specifically, the base station may transmit to the terminal setting information related to at least one of the following: a patch size for pre-processing / post-processing of data for semantic communication, the number of threshold(s) for quantizing the attention map, the value of the threshold(s) for quantizing the attention map, a matrix for reducing the change in data values ​​(e.g., a blurring matrix), a matrix for increasing the change in data values ​​(e.g., a sharpening matrix), RI, and a mapping relationship between pre-processing / post-processing operations. That is, the base station may transmit setting information necessary for semantic communication according to various embodiments of the present disclosure.

[0261] In step S3007, the base station receives a semantic representation and an RI. At this time, the semantic representation and the RI may be received as a single transmission block. Alternatively, the semantic representation and the RI may be received as separate transmission blocks. Alternatively, the semantic representation may be received as a transmission block and the RI as control information.

[0262] In step S3009, the base station restores data based on a semantic representation and performs post-processing. Specifically, the base station may restore data by performing semantic decoding on the semantic representation and perform post-processing on the restored data based on RI. The post-processing is an operation to reverse an operation to increase or decrease the amount of change of patch-unit data values ​​performed at the terminal, and may include an operation corresponding to the element values ​​of RI (e.g., inverse blurring or inverse sharpening). Here, one element value of RI may indicate an operation for one patch.

[0263] In step S3011, the base station performs a task. In other words, the base station performs the task using the restored and post-processed data. The task may be performed using a Transformer-based artificial intelligence model. For example, the task may include classifying the data into classes, generating new data based on the data, etc., and the output of the task may include a payload that the terminal intends to transmit or environmental information for communication (e.g., channel information).

[0264]

[0265] FIGS. 29 and FIGS. 30 are examples of operations of a terminal and a base station, which may be performed when the terminal transmits a semantic expression to the base station. Conversely, the base station may transmit a semantic expression to the terminal. In this case, the terminal may perform steps S2101, S2103, and S2105 of FIG. 29, and steps S3007, S3009, and S3011 of FIG. 30. Additionally, the base station may perform steps S2101, S2103, and S2105 of FIG. 30, and steps S2907 and S2909 of FIG. 29.

[0266]

[0267] In an environment where task-oriented semantic communication is performed, restoration-based schemes offer the advantage of flexibility, allowing the system to be designed using pre-learned task modules. However, in the case of restoration-based task-oriented semantic communication, since transmitted and received data must be restored, there may be resource inefficiency in terms of performing actual tasks.

[0268] By performing pre-processing and post-processing according to various embodiments of the present disclosure, the efficiency of transmission and / or reception resources can be increased when performing restoration-based task-oriented semantic communication using a semantic encoder / decoder such as an NTSCC encoder / decoder. Interpreted in terms of data restoration performance, this means that transmission and reception are performed to improve task execution performance based on the same resource efficiency. The effects of this proposed technology are visualized in FIGS. 31a and 31b. FIGS. 31a and 31b illustrate examples of patch-specific PSNR performance when pre-processing and post-processing according to embodiments of the present disclosure are not applied and when they are applied. As shown in FIG. 31a, when pre-processing and post-processing are not performed, all parts of the data will be restored at the receiver with similar PSNR performance. On the other hand, as shown in Fig. 31b, when pre / post-processing is performed, the blurred parts of the data can be restored with relatively low PSNR performance, while the sharpened parts can be restored with relatively high PSNR performance. That is, since the sharpened parts are the regions in the attention map that have a greater impact on the task, the PSNR performance of the sharpened parts increases, thereby achieving an improvement in task performance. In other words, task performance can be improved while using the same resources in terms of task execution.

[0269]

[0270] Hereinafter, examples of wireless device applications to which various embodiments of the present disclosure are applied will be described.

[0271] FIG. 32 illustrates an example of a wireless device applicable to the present disclosure. The wireless device may be implemented in various forms depending on the use—example / service (see FIG. 1).

[0272] Referring to FIG. 32, the wireless device (200) corresponds to the wireless device (200) of FIG. 2 and may be composed of various elements, components, units / parts, and / or modules. For example, the wireless device (200) may include a communication unit (210), a control unit (220), a memory unit (230), and additional elements (240). The communication unit may include a communication circuit (212) and transceiver(s) (214). For example, the communication circuit (212) may include one or more processors (202) and / or one or more memories (204) of FIG. 2. For example, the transceiver(s) (214) may include one or more transceivers (206) and / or one or more antennas (208) of FIG. 2. The control unit (220) is electrically connected to the communication unit (210), the memory unit (230), and additional elements (240) and controls the overall operation of the wireless device. For example, the control unit (220) can control the electrical / mechanical operation of the wireless device based on a program / code / command / information stored in the memory unit (230). Additionally, the control unit (220) can transmit information stored in the memory unit (230) to an external entity (e.g., another communication device) via a wireless / wired interface through the communication unit (210), or store information received from an external entity (e.g., another communication device) via a wireless / wired interface through the communication unit (210) in the memory unit (230).

[0273] The additional element (240) may be configured in various ways depending on the type of wireless device. For example, the additional element (240) may include at least one of a power unit / battery, an input / output unit (I / O unit), a driving unit, and a computing unit. Although not limited thereto, the wireless device may be implemented in the form of a robot (Fig. 1, 100a), a vehicle (Fig. 1, 100b-1, 100b-2), an XR device (Fig. 1, 100c), a portable device (Fig. 1, 100d), a home appliance (Fig. 1, 100e), an IoT device (Fig. 1, 100f), a digital broadcasting terminal, a hologram device, a public safety device, an MTC device, a medical device, a fintech device (or financial device), a security device, a climate / environment device, an AI server / device (Fig. 1, 400), a base station (Fig. 1, 200), a network node, etc. Depending on the use—e.g., service—the wireless device may be movable or used in a fixed location.

[0274] In FIG. 32, various elements, components, units / parts, and / or modules within the wireless device (200) may be entirely interconnected via a wired interface, or at least some of them may be wirelessly connected via a communication unit (210). For example, within the wireless device (200), the control unit (220) and the communication unit (210) may be wired, and the control unit (220) and the first unit (e.g., 230, 240) may be wirelessly connected via the communication unit (210). Additionally, each element, component, unit / part, and / or module within the wireless device (200) may include one or more additional elements. For example, the control unit (220) may be composed of one or more sets of processors. For example, the control unit (220) may be composed of a set of a communication control processor, an application processor, an Electronic Control Unit (ECU), a graphics processing processor, a memory control processor, etc. As another example, the memory unit (130) may be composed of RAM (Random Access Memory), DRAM (Dynamic RAM), ROM (Read Only Memory), flash memory, volatile memory, non-volatile memory and / or a combination thereof.

[0275] Hereinafter, an implementation example of FIG. 32 will be described in more detail with reference to the drawings.

[0276] FIG. 33 illustrates an example of a portable device applicable to the present disclosure. The portable device may include a smartphone, a smartpad, a wearable device (e.g., a smartwatch, smart glasses), or a portable computer (e.g., a laptop). The portable device may be referred to as a Mobile Station (MS), a User Terminal (UT), a Mobile Subscriber Station (MSS), a Subscriber Station (SS), an Advanced Mobile Station (AMS), or a Wireless Terminal (WT).

[0277] Referring to FIG. 33, the portable device (200) may include an antenna unit (208), a communication unit (210), a control unit (220), a memory unit (230), a power supply unit (240a), an interface unit (240b), and an input / output unit (240c). The antenna unit (208) may be configured as part of the communication unit (210). Blocks 210 to 230 / 240a to 240c of FIG. 33 correspond to blocks 210 to 230 / 240 of FIG. 32, respectively.

[0278] The communication unit (210) can transmit and receive signals (e.g., data, control signals, etc.) with other wireless devices and base stations. The control unit (220) can control the components of the portable device (200) to perform various operations. The control unit (220) may include an AP (Application Processor). The memory unit (230) can store data / parameters / programs / code / commands required for the operation of the portable device (200). Additionally, the memory unit (230) can store input / output data / information, etc. The power supply unit (240a) supplies power to the portable device (200) and may include wired / wireless charging circuits, batteries, etc. The interface unit (240b) can support the connection between the portable device (200) and other external devices. The interface unit (240b) may include various ports (e.g., audio input / output ports, video input / output ports) for connection with external devices. The input / output unit (240c) can receive or output video information / signals, audio information / signals, data, and / or information input by a user. The input / output unit (240c) may include a camera, a microphone, a user input unit, a display unit (240d), a speaker and / or a haptic module, etc.

[0279] For example, in the case of data communication, the input / output unit (240c) acquires information / signals (e.g., touch, text, voice, image, video) input by the user, and the acquired information / signals can be stored in the memory unit (230). The communication unit (210) converts the information / signals stored in the memory into wireless signals and can directly transmit the converted wireless signals to another wireless device or to a base station. Additionally, the communication unit (210) can receive wireless signals from another wireless device or base station and then restore the received wireless signals to their original information / signals. The restored information / signals are stored in the memory unit (230) and then can be output in various forms (e.g., text, voice, image, video, haptic) through the input / output unit (240c).

[0280] FIG. 34 illustrates an example of a vehicle or autonomous vehicle applicable to the present disclosure. The vehicle or autonomous vehicle may be implemented as a mobile robot, a vehicle, a train, an aerial vehicle (AV), a ship, etc.

[0281] Referring to FIG. 34, a vehicle or autonomous vehicle (200-1) may include an antenna unit (208-1), a communication unit (210-1), a control unit (220-1), a driving unit (240a-1), a power supply unit (240b-1), a sensor unit (240c-1), and an autonomous driving unit (240d-1). The antenna unit (208-1) may be configured as part of the communication unit (210-1). Blocks 210-1 / 230-1 / 240a-1 to 240d-1 of FIG. 34 correspond to blocks 210 / 230 / 240 of FIG. 32, respectively.

[0282] The communication unit (210-1) can transmit and receive signals (e.g., data, control signals, etc.) with external devices such as other vehicles, base stations (e.g., base stations, roadside base stations (Road Side Unit), etc.), and servers. The control unit (220-1) can perform various operations by controlling elements of the vehicle or autonomous vehicle (200-1). The control unit (220-1) may include an Electronic Control Unit (ECU). The driving unit (240a-1) can drive the vehicle or autonomous vehicle (200-1) on the ground. The driving unit (240a-1) may include an engine, motor, power train, wheels, brakes, steering device, etc. The power supply unit (240b-1) supplies power to the vehicle or autonomous vehicle (200-1) and may include wired / wireless charging circuits, batteries, etc. The sensor unit (240c-1) can obtain vehicle status, surrounding environment information, user information, etc. The sensor unit (240c-1) may include an IMU (inertial measurement unit) sensor, a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight detection sensor, a heading sensor, a position module, a vehicle forward / reverse sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor, a temperature sensor, a humidity sensor, an ultrasonic sensor, an illuminance sensor, a pedal position sensor, etc. The autonomous driving unit (240d-1) may implement technologies such as maintaining the driving lane, technologies for automatically adjusting speed such as adaptive cruise control, technologies for automatically driving along a predetermined path, and technologies for automatically setting a path and driving when a destination is set.

[0283] For example, the communication unit (210-1) can receive map data, traffic information data, etc. from an external server. The autonomous driving unit (240d-1) can generate an autonomous driving path and a driving plan based on the acquired data. The control unit (220-1) can control the drive unit (240a-1) so that the vehicle or the autonomous vehicle (200-1) moves along the autonomous driving path according to the driving plan (e.g., speed / direction control). During autonomous driving, the communication unit (210-1) can acquire the latest traffic information data from an external server non-periodically and can acquire surrounding traffic information data from surrounding vehicles. Additionally, during autonomous driving, the sensor unit (240c-1) can acquire vehicle status and surrounding environment information. The autonomous driving unit (240d-1) can update the autonomous driving path and the driving plan based on the newly acquired data / information. The communication unit (210-1) can transmit information regarding the vehicle location, autonomous driving path, driving plan, etc. to an external server. An external server can predict traffic information data in advance using AI technology, etc., based on information collected from vehicles or autonomous vehicles, and can provide the predicted traffic information data to vehicles or autonomous vehicles. If the device (220-2) is an autonomous vehicle, it can perform the same procedure as the vehicle or autonomous vehicle (200-1). In addition, if the device (220-2) is a base station or a roadside base station, the device (220-2) can transmit data and control signals to the vehicle or autonomous vehicle (200-1) through the communication unit (210-2).

[0284] FIG. 35 illustrates an example of a vehicle applicable to the present disclosure. The vehicle may be implemented as a means of transport, a train, an aircraft, a ship, etc. Referring to FIG. 35, the vehicle (200) may include a communication unit (210), a control unit (220), a memory unit (230), an input / output unit (240a), and a position measurement unit (240b). Here, blocks 210 to 230 / 240a to 240b correspond to blocks 210 to 230 / 240 of FIG. 32, respectively.

[0285] The communication unit (210) can transmit and receive signals (e.g., data, control signals, etc.) with other vehicles or external devices such as base stations. The control unit (220) can control the components of the vehicle (200) to perform various operations. The memory unit (230) can store data / parameters / programs / codes / commands that support various functions of the vehicle (100). The input / output unit (240a) can output AR / VR objects based on information within the memory unit (230). The input / output unit (240a) may include a HUD. The position measurement unit (240b) can acquire position information of the vehicle (200). The position information may include absolute position information of the vehicle (200), position information within the driving line, acceleration information, position information relative to surrounding vehicles, etc. The position measurement unit (240b) may include GPS and various sensors.

[0286] For example, the communication unit (210) of the vehicle (200) can receive map information, traffic information, etc. from an external server and store it in the memory unit (230). The location measurement unit (240b) can acquire vehicle location information through GPS and various sensors and store it in the memory unit (230). The control unit (220) creates a virtual object based on map information, traffic information, and vehicle location information, etc., and the input / output unit (240a) can display the created virtual object on the glass window inside the vehicle (240a-1, 240a-2). In addition, the control unit (220) can determine whether the vehicle (200) is operating normally within the driving line based on the vehicle location information. If the vehicle (200) deviates abnormally from the driving line, the control unit (220) can display a warning on the glass window inside the vehicle through the input / output unit (240a). Additionally, the control unit (220) can broadcast a warning message regarding a driving abnormality to surrounding vehicles through the communication unit (210). Depending on the situation, the control unit (220) can transmit the vehicle's location information and information regarding the driving / vehicle abnormality to relevant authorities through the communication unit (210).

[0287] FIG. 36 illustrates an example of an XR device applicable to the present disclosure. The XR device may be implemented as an HMD, a Head-Up Display (HUD) equipped in a vehicle, a television, a smartphone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a robot, etc.

[0288] Referring to FIG. 36, the XR device (200a) may include a communication unit (210), a control unit (220), a memory unit (230), an input / output unit (240a), a sensor unit (240b), and a power supply unit (240c). Here, blocks 210 to 230 / 240a to 240c of FIG. 36 correspond to blocks 210 to 230 / 240 of FIG. 32, respectively.

[0289] The communication unit (210) can transmit and receive signals (e.g., media data, control signals, etc.) with external devices such as other wireless devices, mobile devices, or media servers. The media data may include video, images, sound, etc. The control unit (220) can control the components of the XR device (200a) to perform various operations. For example, the control unit (220) may be configured to control and / or perform procedures such as video / image acquisition, (video / image) encoding, metadata generation, and processing. The memory unit (230) may store data / parameters / programs / code / commands required for driving the XR device (200a) or creating an XR object. The input / output unit (240a) acquires control information, data, etc. from the outside and can output the created XR object. The input / output unit (240a) may include a camera, microphone, user input unit, display unit, speaker and / or haptic module, etc. The sensor unit (240b) can obtain XR device status, surrounding environment information, user information, etc. The sensor unit (240b) may include a proximity sensor, an illuminance sensor, an accelerometer, a magnetic sensor, a gyroscope, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone and / or radar, etc. The power supply unit (240c) supplies power to the XR device (200a) and may include a wired / wireless charging circuit, a battery, etc.

[0290] For example, the memory unit (230) of the XR device (200a) may contain information (e.g., data, etc.) necessary for creating an XR object (e.g., AR / VR / MR object). The input / output unit (240a) may receive a command to operate the XR device (200a) from the user, and the control unit (220) may operate the XR device (200a) according to the user's operation command. For example, if the user intends to watch movies, news, etc. through the XR device (200a), the control unit (220) may transmit content request information to another device (e.g., mobile device (200b)) or a media server through the communication unit (230). The communication unit (230) may download / stream content such as movies, news, etc. from another device (e.g., mobile device (200b)) or a media server to the memory unit (230). The control unit (220) controls and / or performs procedures such as video / image acquisition, (video / image) encoding, and metadata generation / processing for the content, and can generate / output an XR object based on information about the surrounding space or real object acquired through the input / output unit (240a) / sensor unit (240b).

[0291] Additionally, the XR device (200a) is wirelessly connected to the mobile device (200b) through the communication unit (210), and the operation of the XR device (200a) can be controlled by the mobile device (200b). For example, the mobile device (200b) can act as a controller for the XR device (200a). To this end, the XR device (200a) can acquire three-dimensional position information of the mobile device (200b), and then generate and output an XR object corresponding to the mobile device (200b).

[0292] FIG. 37 illustrates an example of a robot applicable to the present disclosure. Robots may be classified into industrial, medical, domestic, military, etc., depending on the purpose or field of use.

[0293] Referring to FIG. 37, the robot (200) may include a communication unit (210), a control unit (220), a memory unit (230), an input / output unit (240a), a sensor unit (240b), and a driving unit (240c). Here, blocks 210 to 230 / 240a to 240c of FIG. 37 correspond to blocks 210 to 230 / 240 of FIG. 32, respectively.

[0294] The communication unit (210) can transmit and receive signals (e.g., driving information, control signals, etc.) with external devices such as other wireless devices, other robots, or control servers. The control unit (220) can control the components of the robot (200) to perform various operations. The memory unit (230) can store data / parameters / programs / codes / commands that support various functions of the robot (200). The input / output unit (240a) can acquire information from outside the robot (200) and output information to outside the robot (200). The input / output unit (240a) may include a camera, microphone, user input unit, display unit, speaker and / or haptic module, etc. The sensor unit (240b) can obtain internal information of the robot (200), surrounding environment information, user information, etc. The sensor unit (240b) may include a proximity sensor, an illuminance sensor, an accelerometer, a magnetic sensor, a gyroscope, an inertial sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone, a radar, etc. The driving unit (240c) may perform various physical movements, such as moving robot joints. Additionally, the driving unit (240c) may enable the robot (200) to travel on the ground or fly in the air. The driving unit (240c) may include an actuator, a motor, a wheel, a brake, a propeller, etc.

[0295] FIG. 38 illustrates an example of an AI device applicable to the present disclosure.

[0296] AI devices can be implemented as stationary devices or mobile devices, such as TVs, projectors, smartphones, PCs, laptops, digital broadcasting terminals, tablet PCs, wearable devices, set-top boxes (STBs), radios, washing machines, refrigerators, digital signage, robots, vehicles, etc.

[0297] Referring to FIG. 38, the AI ​​device (200) may include a communication unit (210), a control unit (220), a memory unit (230), an input / output unit (240a / 240b), a learning processor unit (240c), and a sensor unit (240d). Blocks 210 to 230 / 240a to 240d of FIG. 38 correspond to blocks 210 to 230 / 140 of FIG. 32, respectively.

[0298] The communication unit (210) can transmit and receive wired and wireless signals (e.g., sensor information, user input, learning model, control signal, etc.) with external devices such as other AI devices (e.g., 100a to 100f, 120 in FIG. 1) or AI servers (e.g., 100g in FIG. 1) using wired and wireless communication technology. To do this, the communication unit (210) can transmit information within the memory unit (230) to an external device or transmit signals received from an external device to the memory unit (230).

[0299] The control unit (220) can determine at least one executable operation of the AI ​​device (200) based on information determined or generated using a data analysis algorithm or a machine learning algorithm. The control unit (220) can perform the determined operation by controlling the components of the AI ​​device (200). For example, the control unit (220) can request, search, receive, or utilize data from the learning processor unit (240c) or the memory unit (230), and can control the components of the AI ​​device (200) to execute a predicted operation or an operation determined to be desirable among at least one executable operation. Additionally, the control unit (220) can collect historical information, including the operation content of the AI ​​device (200) or user feedback regarding the operation, and store it in the memory unit (230) or the learning processor unit (240c), or transmit it to an external device such as an AI server (Fig. 1, 100g). The collected historical information can be used to update the learning model.

[0300] The memory unit (230) can store data that supports various functions of the AI ​​device (200). For example, the memory unit (230) can store data obtained from the input unit (240a), data obtained from the communication unit (210), output data from the learning processor unit (240c), and data obtained from the sensing unit (140). Additionally, the memory unit (230) can store control information and / or software code required for the operation / execution of the control unit (220).

[0301] The input unit (240a) can acquire various types of data from outside the AI ​​device (200). For example, the input unit (220) can acquire training data for model training and input data to which the training model is applied. The input unit (240a) may include a camera, a microphone and / or a user input unit, etc. The output unit (240b) can generate output related to visual, auditory, or tactile senses, etc. The output unit (240b) may include a display unit, a speaker and / or a haptic module, etc. The sensing unit (140d) can obtain at least one of internal information of the AI ​​device (200), surrounding environment information of the AI ​​device (200), and user information using various sensors. The sensing unit (140d) may include a proximity sensor, an illuminance sensor, an accelerometer, a magnetic sensor, a gyroscope, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone and / or radar, etc.

[0302] The learning processor unit (240c) can train a model composed of an artificial neural network using training data. The learning processor unit (240c) can perform AI processing together with the learning processor unit of the AI ​​server (Fig. 1, 100g). The learning processor unit (240c) can process information received from an external device through the communication unit (210) and / or information stored in the memory unit (230). Additionally, the output value of the learning processor unit (240c) can be transmitted to an external device through the communication unit (210) and / or stored in the memory unit (230).

[0303]

[0304] The proposed methods described above may be implemented independently, but they may also be implemented in the form of a combination (or merger) of some of the proposed methods. Rules may be defined so that the base station informs the terminal of the application status of the proposed methods (or information regarding the rules of the proposed methods) through a predefined signal (e.g., a physical layer signal or an upper layer signal).

[0305] The present disclosure may be embodied in other specific forms without departing from the technical ideas and essential features described herein. Accordingly, the above detailed description should not be interpreted restrictively in all respects and should be considered illustrative. The scope of the present disclosure shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present disclosure are included within the scope of the present disclosure. Furthermore, embodiments may be constructed by combining claims that are not explicitly related in the claims, or new claims may be included by amendments made after filing.

[0306] One embodiment can be applied to various wireless access systems. Examples of various wireless access systems include the 3GPP (3rd Generation Partnership Project) or 3GPP2 system.

[0307] One embodiment can be applied not only to the various wireless access systems mentioned above, but also to all technical fields utilizing the various wireless access systems. Furthermore, the proposed method can be applied to mmWave and THz communication systems utilizing the ultra-high frequency band.

[0308] Additionally, some embodiments may be applied to various applications such as autonomous vehicles and drones.

Claims

1. Regarding the method, Step of performing a random access connection procedure; Step of receiving a request for capability information; A step of transmitting the above capability information; A step of performing signaling for communication-related settings; and A step of generating a semantic representation corresponding to pre-processed data by performing semantic encoding; It includes the step of transmitting region information related to the above semantic representation and the above pre-processing, The above pre-processing method includes an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the region information determined based on an attention map related to a target task performed using the data.

2. In Claim 1, The above pre-processing method comprises at least one of a first operation that reduces the amount of change in data values ​​within at least one patch belonging to a part that has a relatively small impact on the performance of the target task, or a second operation that increases the amount of change in data values ​​within at least one patch belonging to a part that has a relatively large impact on the performance of the target task.

3. In Claim 2, The above first operation includes a blurring operation, and The above second operation is a method including a sharpening operation.

4. In Claim 1, The above configuration comprises at least one of a patch size for pre-processing or post-processing of the data, at least one number of thresholds for quantizing the attention map, at least one value of the threshold for quantizing the attention map, a matrix for reducing the amount of change of data values, a matrix for increasing the amount of change of data values, or a mapping relationship between the region information and the pre-processing or post-processing operation.

5. In Claim 1, A step of generating the attention map for the above data; A step of generating region information based on the attention map and at least one threshold; and A method further comprising the step of performing the pre-processing based on the above area information.

6. In Claim 5, A method in which the above-mentioned region information includes elements indicating the index of a pre-processing or post-processing operation applied to each of the patches included in the above-mentioned data.

7. In Claim 5, The step of performing the above pre-processing is, A step of dividing the above data into multiple patches; A step of determining an operation to be applied to at least some of the patches based on the value of an attention map corresponding to the data value included in the patches; and A method comprising the step of performing the above operation on at least some of the above patches.

8. Regarding the method, Step of performing a random access connection procedure; A step of transmitting a request for capability information; A step of receiving the above capability information; A step of performing signaling for communication-related settings; and A step of receiving a semantic representation corresponding to pre-processed data and region information related to the pre-processing; A step of obtaining the pre-processed data by performing semantic decoding; The method includes the step of performing a target task using restored data obtained by performing post-processing on the pre-processed data based on the area information above. The above post-processing method includes an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the area information.

9. In Claim 8, The above post-processing method comprises at least one of a first operation that reduces the amount of change in data values ​​within at least one patch belonging to a portion that has a relatively small impact on the performance of the target task, or a second operation that increases the amount of change in data values ​​within at least one patch belonging to a portion that has a relatively large impact on the performance of the target task.

10. In Claim 9, The above first operation includes an inverse-sharpening operation, and The above second operation is a method including a deblurring operation.

11. In the device, Transmitter / receiver; and It includes a processor coupled to the above-mentioned transmitter and receiver, The above processor is, Perform a random access connection procedure, Receive a request for capability information, Transmit the above capability information, and Performs signaling for communication-related settings, By performing semantic encoding, a semantic representation corresponding to the pre-processed data is generated, and It is configured to transmit region information related to the above semantic representation and the above pre-processing, and The above pre-processing is a device comprising an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the region information determined based on the attention map related to the target task performed using the data.

12. In the device, Transmitter / receiver; and It includes a processor coupled to the above-mentioned transmitter and receiver, The above processor is, Perform a random access connection procedure, Send a request for capability information, and Receive the above capability information, Performs signaling for communication-related settings, Receive a semantic representation corresponding to the pre-processed data and region information related to the pre-processing, and The above-mentioned pre-processed data is obtained by performing semantic decoding, and It is configured to perform a target task using restored data obtained by performing post-processing on the pre-processed data based on the above area information, and The above post-processing is a device comprising an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the area information.

13. In the terminal, At least one processor; It includes at least one memory connected to the at least one processor and storing instructions that cause a terminal to perform operations as executed by the at least one processor, The above operations are, Step of performing a random access connection procedure; Step of receiving a request for capability information; A step of transmitting the above capability information; A step of performing signaling for communication-related settings; and A step of generating a semantic representation corresponding to pre-processed data by performing semantic encoding; It includes the step of transmitting region information related to the above semantic representation and the above pre-processing, The above pre-processing includes a terminal for increasing or decreasing the amount of change of data values ​​for at least a portion of the data indicated by the region information determined based on the attention map related to the target task performed using the data.

14. In a non-transitory computer-readable medium storing at least one program instruction, The above at least one program instruction causes the terminal to perform operations as it is executed by at least one processor, and The above operations are, Step of performing a random access connection procedure; Step of receiving a request for capability information; A step of transmitting the above capability information; A step of performing signaling for communication-related settings; and A step of generating a semantic representation corresponding to pre-processed data by performing semantic encoding; It includes the step of transmitting region information related to the above semantic representation and the above pre-processing, A computer-readable medium comprising an operation to increase or decrease the amount of change of data values ​​for at least a portion of the data indicated by the region information determined based on an attention map related to a target task performed using the data.

Images