Method and device for detecting and correcting semantic error in wireless communication system

Abstract

The present disclosure is for detecting and correcting a semantic error in a wireless communication system. A method performed by a first device may comprise the steps of: establishing a connection with a second device; receiving a first message requesting for capability information from the second device; transmitting a second message including the capability information to the second device; receiving configuration information for communication from the second device; and performing semantic communication supporting multiple tasks, on the basis of the configuration information.

Description

Method and device for detecting and correcting semantic errors in wireless communication systems

[0001] The following description relates to a wireless communication system, and to a device and method for detecting and correcting semantic errors in a wireless communication system.

[0002] Wireless access systems are widely deployed to provide various types of communication services, such as voice and data. Typically, wireless access systems are multiple access systems that support communications with multiple users by sharing available system resources (e.g., bandwidth, transmission power). Examples of multiple access systems include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), and single-carrier frequency division multiple access (SC-FDMA).

[0003] In particular, as numerous communication devices demand greater communication capacity, enhanced mobile broadband (eMBB) communication technologies are being proposed, surpassing existing radio access technology (RAT). Furthermore, massive machine type communications (mMTC), which connects multiple devices and objects to provide diverse services anytime and anywhere, as well as communication systems that consider reliability and latency-sensitive services / user equipment (UE), are being proposed. Various technological configurations are being proposed for these purposes.

[0004] The present disclosure relates to a method and device for effectively performing semantic communication in a wireless communication system.

[0005] The present disclosure relates to a method and apparatus for efficiently detecting semantic errors in a wireless communication system.

[0006] The present disclosure relates to a method and apparatus for efficiently correcting semantic errors in a wireless communication system.

[0007] The present disclosure relates to a method and apparatus for detecting and correcting semantic errors based on attention information in a wireless communication system.

[0008] The present disclosure relates to a method and apparatus for detecting semantic errors in a wireless communication system based on the difference between attention information for original data and attention information for data reconstructed at a semantic level.

[0009] The present disclosure relates to a method and apparatus for generating information for semantic error detection and correction based on a unified semantic representation for a plurality of tasks in a wireless communication system.

[0010] The present disclosure relates to a method and apparatus for generating information for semantic error detection and correction based on a plurality of attention maps corresponding to a plurality of tasks in a wireless communication system.

[0011] The present disclosure relates to a method and apparatus for generating information for semantic error detection and correction based on the combination of a plurality of attention maps corresponding to different tasks in a wireless communication system.

[0012] The present disclosure relates to a method and apparatus for generating information for requesting semantic error correction based on data restored from a semantic representation transmitted from a source to a destination in a wireless communication system.

[0013] The present disclosure relates to a method and apparatus for generating information for requesting semantic error correction based on attention information corresponding to original data and attention information corresponding to data restored at a semantic level in a wireless communication system.

[0014] The present disclosure relates to a method and apparatus for generating feedback information requesting semantic error correction based on a difference between union attention information corresponding to original data and union attention information corresponding to data restored at a semantic level in a wireless communication system.

[0015] The technical objectives to be achieved in the present disclosure are not limited to those mentioned above, and other technical tasks not mentioned can be considered by a person having ordinary skill in the technical field to which the technical configuration of the present disclosure is applied from the embodiments of the present disclosure described below.

[0016] As an example of the present disclosure, a method performed by a first device in a wireless communication system may include the steps of establishing a connection with a second device, receiving a first message requesting capability information from the second device, transmitting a second message including the capability information to the second device, receiving configuration information for communication from the second device, and performing semantic communication supporting a plurality of tasks based on the configuration information. The configuration information may include information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

[0017] As an example of the present disclosure, a method performed by a second device in a wireless communication system may include the steps of establishing a connection with a first device, transmitting a first message requesting capability information to the first device, receiving a second message including the capability information from the first device, transmitting configuration information for communication to the first device, and performing semantic communication supporting a plurality of tasks based on the configuration information. The configuration information may include information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

[0018] As an example of the present disclosure, in a wireless communication system, a first device includes a transceiver and a processor connected to the transceiver, wherein the processor establishes a connection with a second device, receives a first message requesting capability information from the second device, transmits a second message including the capability information to the second device, receives configuration information for communication from the second device, and controls the first device to perform semantic communication supporting a plurality of tasks based on the configuration information, wherein the configuration information includes information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

[0019] As an example of the present disclosure, in a wireless communication system, a second device includes a transceiver and a processor connected to the transceiver, wherein the processor establishes a connection with a first device, transmits a first message requesting capability information to the first device, receives a second message including the capability information from the first device, transmits configuration information for communication to the first device, and performs semantic communication supporting a plurality of tasks based on the configuration information, wherein the configuration information may include information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

[0020] As an example of the present disclosure, a communication device includes at least one processor, and at least one computer memory connected to the at least one processor and storing instructions that direct operations when executed by the at least one processor, wherein the operations may include: establishing a connection with another communication device; receiving a first message requesting capability information from the other communication device; transmitting a second message including the capability information to the other communication device; receiving configuration information for communication from the other communication device; and performing semantic communication supporting a plurality of tasks based on the configuration information. The configuration information may include information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

[0021] As an example of the present disclosure, a non-transitory computer-readable medium storing at least one instruction includes at least one instruction executable by a processor, the at least one instruction controlling a device to establish a connection with another device, receive a first message requesting capability information from the other device, transmit a second message including the capability information to the other device, receive configuration information for communication from the other device, and perform semantic communication supporting a plurality of tasks based on the configuration information, wherein the configuration information may include information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

[0022] The above-described aspects of the present disclosure are only some of the preferred embodiments of the present disclosure, and various embodiments reflecting the technical features of the present disclosure can be derived and understood by a person having ordinary skill in the art based on the detailed description of the present disclosure to be described below.

[0023] The following effects may be achieved by embodiments based on the present disclosure.

[0024] According to the present disclosure, semantic communication can be effectively performed in a wireless communication system.

[0025] The effects that can be obtained from the embodiments of the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly derived and understood by those skilled in the art to which the technical configuration of the present disclosure is applied, from the description of the embodiments of the present disclosure below. In other words, unintended effects that result from implementing the configuration described in the present disclosure can also be derived by those skilled in the art from the embodiments of the present disclosure.

[0026] The accompanying drawings are intended to aid understanding of the present disclosure and, together with detailed descriptions, may provide embodiments of the present disclosure. However, the technical features of the present disclosure are not limited to specific drawings, and the features disclosed in each drawing may be combined with each other to form new embodiments. Reference numerals in each drawing may indicate structural elements.

[0027] Figure 1 illustrates an example of a communication system applicable to the present disclosure.

[0028] FIG. 2 illustrates an example of a wireless device applicable to the present disclosure.

[0029] FIG. 3 illustrates a method for processing a transmission signal applicable to the present disclosure.

[0030] Figure 4 illustrates a communication procedure between a terminal and a base station applicable to the present disclosure.

[0031] FIG. 5 illustrates an example of a communication structure that can be provided in a 6G (6th generation) system applicable to the present disclosure.

[0032] Figure 6 illustrates an electromagnetic spectrum applicable to the present disclosure.

[0033] FIG. 7 illustrates a THz wireless communication transceiver applicable to the present disclosure.

[0034] Figure 8 illustrates a THz signal generation method applicable to the present disclosure.

[0035] FIG. 9 illustrates a wireless communication transceiver applicable to the present disclosure.

[0036] Figure 10 illustrates a transmitter structure applicable to the present disclosure.

[0037] Figure 11 illustrates a modulator structure applicable to the present disclosure.

[0038] Figure 12 illustrates the structure of a perceptron included in an artificial neural network applicable to the present disclosure.

[0039] Figure 13 illustrates an artificial neural network structure applicable to the present disclosure.

[0040] Figure 14 illustrates a communication model applicable to the present disclosure.

[0041] Figure 15 illustrates an example of a semantic communication error.

[0042] Figure 16 shows an example of visualizing the relationships between words in a translation task using the attention technique.

[0043] Figure 17 illustrates an example of a vision transformer model.

[0044] Figure 18 illustrates an example in which multiple downstream tasks are performed.

[0045] FIG. 19 illustrates the functional structure of a source for performing union attention map-based semantic error detection and correction according to one embodiment of the present disclosure.

[0046] FIG. 20 illustrates another functional structure of a source for performing union attention map-based semantic error detection and correction according to one embodiment of the present disclosure.

[0047] FIG. 21 illustrates the functional structure of a destination for performing union attention map-based semantic error detection and correction according to one embodiment of the present disclosure.

[0048] FIG. 22 illustrates another functional structure of a destination for performing union attention map-based semantic error detection and correction according to one embodiment of the present disclosure.

[0049] FIG. 23 illustrates an example of a procedure for performing semantic communication of a first device according to one embodiment of the present disclosure.

[0050] FIG. 24 illustrates an example of a procedure for performing semantic communication of a second device according to one embodiment of the present disclosure.

[0051] FIG. 25 illustrates an example of a procedure for supporting semantic error correction of a device operating as a source according to one embodiment of the present disclosure.

[0052] FIG. 26 illustrates an example of a procedure for performing semantic error correction of a device operating as a destination according to one embodiment of the present disclosure.

[0053] FIG. 27 illustrates an example of a procedure supporting union attention map-based error detection and correction according to one embodiment of the present disclosure.

[0054] FIG. 28 illustrates an example of a procedure for performing union attention map-based error detection and correction according to one embodiment of the present disclosure.

[0055] FIG. 29 illustrates an example of an initial setup procedure for semantic communication according to one embodiment of the present disclosure.

[0056] FIGS. 30A to 30G illustrate examples of data generated according to an exemplary procedure according to one embodiment of the present disclosure.

[0057] The following embodiments combine the components and features of the present disclosure in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, some components and / or features may be combined to form embodiments of the present disclosure. The order of operations described in the embodiments of the present disclosure may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment.

[0058] In the description of the drawings, procedures or steps that may obscure the gist of the present disclosure are not described, and procedures or steps that can be understood by a person skilled in the art are also not described.

[0059] Throughout the specification, when a part is said to "comprising" or "including" a component, this does not mean that other components may be included, but rather that other components may be excluded, unless otherwise specifically stated. In addition, terms such as "...part," "...unit," and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software. In addition, the words "a" or "an," "one," "the," and similar related words may be used in the context of describing the present disclosure (especially in the context of the claims below) to include both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context.

[0060] Embodiments of the present disclosure described herein focus on the data transmission and reception relationship between a base station and a mobile station. Here, the base station is understood as a terminal node of a network that directly communicates with the mobile station. Certain operations described herein as being performed by the base station may, in some cases, be performed by an upper node of the base station.

[0061] That is, in a network consisting of multiple network nodes including a base station, various operations performed for communication with a mobile station may be performed by the base station or other network nodes other than the base station. In this case, the term 'base station' may be replaced by terms such as fixed station, Node B, eNB (eNode B), gNB (gNode B), ng-eNB, advanced base station (ABS), or access point.

[0062] Additionally, in the embodiments of the present disclosure, the term terminal may be replaced with terms such as user equipment (UE), mobile station (MS), subscriber station (SS), mobile subscriber station (MSS), mobile terminal, or advanced mobile station (AMS).

[0063] Additionally, a transmitter refers to a fixed and / or mobile node that provides data or voice services, and a receiver refers to a fixed and / or mobile node that receives data or voice services. Therefore, for uplink, a mobile station can be the transmitter, and a base station can be the receiver. Similarly, for downlink, a mobile station can be the receiver, and a base station can be the transmitter.

[0064] Embodiments of the present disclosure are wireless access systems, such as IEEE 802.xx systems, 3GPP (3 rd Generation Partnership Project) system, 3GPP LTE (Long Term Evolution) system, 3GPP 5G (5 th generation) NR (New Radio) system and 3GPP2 system, and in particular, the embodiments of the present disclosure may be supported by 3GPP TS (technical specification) 38.211, 3GPP TS 38.212, 3GPP TS 38.213, 3GPP TS 38.321 and 3GPP TS 38.331 documents.

[0065] Furthermore, the embodiments of the present disclosure can be applied to other wireless access systems and are not limited to the systems described above. For example, they can also be applied to systems implemented after the 3GPP 5G NR system, and are not limited to a specific system.

[0066] That is, obvious steps or parts not described in the embodiments of the present disclosure can be explained by referring to the above documents. In addition, all terms disclosed in this document can be explained by the above standard documents.

[0067] Hereinafter, preferred embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. The detailed description set forth below, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the present disclosure and is not intended to represent the only embodiments in which the technical configurations of the present disclosure may be implemented.

[0068] Additionally, specific terms used in the embodiments of the present disclosure are provided to aid in understanding of the present disclosure, and the use of such specific terms may be changed to other forms without departing from the technical spirit of the present disclosure.

[0069] The following technology can be applied to various wireless access systems such as CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), and SC-FDMA (single carrier frequency division multiple access).

[0070] For clarity, the following description is based on 3GPP communication systems (e.g., LTE, NR, etc.), but the technical spirit of the present disclosure is not limited thereto. LTE may refer to technology after 3GPP TS 36.xxx Release 8. Specifically, LTE technology after 3GPP TS 36.xxx Release 10 may be referred to as LTE-A, and LTE technology after 3GPP TS 36.xxx Release 13 may be referred to as LTE-A pro. 3GPP NR may refer to technology after TS 38.xxx Release 15. 3GPP 6G may refer to technology after TS Release 17 and / or Release 18. "xxx" refers to a standard document detail number. LTE / NR / 6G may be collectively referred to as a 3GPP system.

[0071] For background information, terms, abbreviations, etc. used in this disclosure, reference may be made to standards documents published prior to this disclosure. For example, reference may be made to the 36.xxx and 38.xxx standard documents.

[0072] Communication system applicable to the present disclosure

[0073] Although not limited thereto, the various descriptions, functions, procedures, proposals, methods and / or operational flowcharts of the present disclosure disclosed in this document may be applied to various fields requiring wireless communication / connectivity (e.g., 5G) between devices.

[0074] Hereinafter, more specific examples will be provided with reference to the drawings. In the drawings / descriptions below, the same drawing reference numerals may represent identical or corresponding hardware blocks, software blocks, or functional blocks, unless otherwise described.

[0075] Figure 1 illustrates an example of a communication system applied to the present disclosure.

[0076] Referring to FIG. 1, a communication system (100) applied to the present disclosure includes a wireless device, a base station, and a network. Here, the wireless device refers to a device that performs communication using a wireless access technology (e.g., LTE, LTE-A, LTE-A pro, NR, 5G, 5G-A, 6G) and may be referred to as a communication / wireless / 5G device. Although not limited thereto, the wireless device may include a robot (100a), a vehicle (100b-1, 100b-2), an XR (extended reality) device (100c), a hand-held device (100d), a home appliance (100e), an IoT (Internet of Things) device (100f), and an AI (artificial intelligence) device / server (100g). For example, the vehicle may include a vehicle equipped with a wireless communication function, an autonomous vehicle, a vehicle capable of performing vehicle-to-vehicle communication, etc. Here, the vehicles (100b-1, 100b-2) may include unmanned aerial vehicles (UAVs) (e.g., drones). The XR devices (100c) include augmented reality (AR) / virtual reality (VR) / mixed reality (MR) devices, and may be implemented in the form of head-mounted devices (HMDs), head-up displays (HUDs) installed in vehicles, televisions, smartphones, computers, wearable devices, home appliances, digital signage, vehicles, robots, etc. The portable devices (100d) may include smartphones, smart pads, wearable devices (e.g., smartwatches, smart glasses), computers (e.g., laptops, etc.), etc. The home appliances (100e) may include TVs, refrigerators, washing machines, etc. The IoT devices (100f) may include sensors, smart meters, etc.For example, the base station (120) and the network (130) may also be implemented as wireless devices, and a specific wireless device (120a) may act as a base station / network node to other wireless devices.

[0077] Wireless devices (100a to 100f) can be connected to a network (130) via a base station (120). AI technology can be applied to the wireless devices (100a to 100f), and the wireless devices (100a to 100f) can be connected to an AI server (100g) via a network (130). The network (130) can be configured using a 3G network, a 4G (e.g., LTE) network, a 5G (e.g., NR), or a 6G network. The wireless devices (100a to 100f) can communicate with each other via the base station (120) / network (130), but can also communicate directly (e.g., sidelink communication) without going through the base station (120) / network (130). For example, vehicles (100b-1, 100b-2) can communicate directly (e.g., V2V (vehicle to vehicle) / V2X (vehicle to everything) communication). Additionally, an IoT device (100f) (e.g., a sensor) can communicate directly with another IoT device (e.g., a sensor) or another wireless device (100a to 100f).

[0078] Wireless communication / connection (150a, 150b, 150c) can be established between wireless devices (100a to 100f) / base stations (120), and base stations (120) / base stations (120). Here, the wireless communication / connection can be established through various wireless access technologies such as uplink / downlink communication (150a), sidelink communication (150b) (or D2D communication), and base station-to-base station communication (150c) (e.g., relay, IAB (integrated access backhaul)). Through the wireless communication / connection (150a, 150b, 150c), the wireless device and base station / wireless device, and base stations and base stations can transmit / receive wireless signals to / from each other. For example, the wireless communication / connection (150a, 150b, 150c) can transmit / receive signals through various physical channels. To this end, based on various proposals of the present disclosure, at least some of various configuration information setting processes for transmitting / receiving wireless signals, various signal processing processes (e.g., channel encoding / decoding, modulation / demodulation, resource mapping / demapping, etc.), resource allocation processes, etc. may be performed.

[0079] Devices applicable to the present disclosure

[0080] FIG. 2 illustrates an example of a wireless device applicable to the present disclosure.

[0081] Referring to FIG. 2, the wireless device (200) can transmit and receive wireless signals via various wireless access technologies (e.g., LTE, LTE-A, LTE-A pro, NR, 5G, 5G-A, 6G). The wireless device (200) includes at least one processor (202) and at least one memory (204), and may additionally include at least one transceiver (206) and / or at least one antenna (208).

[0082] The processor (202) controls the memory (204) and / or the transceiver (206), and may be configured to implement the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed in this document. For example, the processor (202) may process information in the memory (204) to generate first information / signal, and then transmit a wireless signal including the first information / signal via the transceiver (206). In addition, the processor (202) may receive a wireless signal including second information / signal via the transceiver (206), and then store information obtained from signal processing of the second information / signal in the memory (204). The memory (204) may be connected to the processor (202) and may store various information related to the operation of the processor (202). For example, the memory (204) may store software code including instructions for performing some or all of the processes controlled by the processor (202), or for performing the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed herein. Here, the processor (202) and the memory (204) may be part of a communication modem / circuit / chip designed to implement wireless communication technology. The transceiver (206) may be connected to the processor (202) and may transmit and / or receive wireless signals via at least one antenna (208). The transceiver (206) may include a transmitter and / or a receiver. The transceiver (206) may be used interchangeably with an RF (radio frequency) unit. In the present disclosure, a wireless device may also mean a communication modem / circuit / chip.

[0083] Hereinafter, the hardware elements of the wireless device (200) will be described in more detail. Although not limited thereto, at least one protocol layer may be implemented by at least one processor (202). For example, at least one processor (202) may implement at least one layer (e.g., a functional layer such as physical (PHY), media access control (MAC), radio link control (RLC), packet data convergence protocol (PDCP), radio resource control (RRC), and service data adaptation protocol (SDAP)). At least one processor (202) may generate at least one Protocol Data Unit (PDU) and / or at least one Service Data Unit (SDU) according to the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. At least one processor (202) may generate a message, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. At least one processor (202) can generate a signal (e.g., a baseband signal) including a PDU, an SDU, a message, control information, data or information according to the functions, procedures, proposals and / or methods disclosed in this document, and provide the signal to at least one transceiver (206). At least one processor (202) can receive a signal (e.g., a baseband signal) from at least one transceiver (206) and obtain the PDU, SDU, message, control information, data or information according to the descriptions, functions, procedures, proposals, methods and / or operational flowcharts disclosed in this document.

[0084] At least one processor (202) may be referred to as a controller, a microcontroller, a microprocessor, or a microcomputer. The at least one processor (202) may be implemented by hardware, firmware, software, or a combination thereof. For example, at least one application specific integrated circuit (ASIC), at least one digital signal processor (DSP), at least one digital signal processing device (DSPD), at least one programmable logic device (PLD), or at least one field programmable gate array (FPGA) may be included in the at least one processor (202). The descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, etc. The descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document may be included in the at least one processor (202), or may be stored in at least one memory (204) and executed by the at least one processor (202). The descriptions, functions, procedures, suggestions, methods and / or flowcharts disclosed in this document may be implemented using firmware or software in the form of code, instructions and / or sets of instructions.

[0085] At least one memory (204) can be connected to at least one processor (202) and can store various forms of data, signals, messages, information, programs, codes, instructions and / or commands. The at least one memory (204) can be configured as a read only memory (ROM), a random access memory (RAM), an erasable programmable read only memory (EPROM), a flash memory, a hard drive, a register, a cache memory, a computer readable storage medium and / or a combination thereof. The at least one memory (204) can be located internally and / or externally to the at least one processor (202). In addition, the at least one memory (204) can be connected to the at least one processor (202) via various technologies such as a wired or wireless connection.

[0086] At least one transceiver (206) can transmit user data, control information, wireless signals / channels, etc., mentioned in the methods and / or flowcharts of this document to at least one other device. At least one transceiver (206) can receive user data, control information, wireless signals / channels, etc. mentioned in the descriptions, functions, procedures, proposals, methods and / or flowcharts disclosed in this document from at least one other device. For example, at least one transceiver (206) can be connected to at least one processor (202) and can transmit and receive wireless signals. For example, at least one processor (202) can control at least one transceiver (206) to transmit user data, control information, or wireless signals to at least one other device. Furthermore, at least one processor (202) can control at least one transceiver (206) to receive user data, control information, or wireless signals from at least one other device. In addition, at least one transceiver (206) may be connected to at least one antenna (208), and at least one transceiver (206) may be configured to transmit and receive user data, control information, wireless signals / channels, etc. mentioned in the descriptions, functions, procedures, proposals, methods and / or operation flowcharts disclosed in this document through at least one antenna (208). In this document, at least one antenna may be a plurality of physical antennas or a plurality of logical antennas (e.g., antenna ports). At least one transceiver (206) may convert the received wireless signals / channels, etc. from RF band signals to baseband signals in order to process the received user data, control information, wireless signals / channels, etc. using at least one processor (202). At least one transceiver (206) may convert the processed user data, control information, wireless signals / channels, etc. from baseband signals to RF band signals using at least one processor (202).For this purpose, at least one transceiver (206) may include an (analog) oscillator and / or filter.

[0087] The components of the wireless device described with reference to FIG. 2 may be referred to by different terms in terms of functionality. For example, the processor (202) may be referred to as a control unit, the transceiver (206) as a communication unit, and the memory (204) as a storage unit. In some cases, the communication unit may be used to mean at least a portion of the processor (202) and the transceiver (206).

[0088] The structure of the wireless device described with reference to FIG. 2 can be understood as the structure of at least a portion of various devices. For example, the structure of the wireless device illustrated in FIG. 2 can be at least a portion of various devices described with reference to FIG. 1 (e.g., a robot (100a), a vehicle (100b-1, 100b-2), an XR device (100c), a portable device (100d), a home appliance (100e), an IoT device (100f), an AI device / server (100g)). Furthermore, according to various embodiments, in addition to the components illustrated in FIG. 2, the device may further include other components.

[0089] For example, the device may be a portable device such as a smartphone, a smart pad, a wearable device (e.g., a smart watch, smart glasses), or a portable computer (e.g., a laptop, etc.). In this case, the device may further include at least one of a power supply unit that supplies power and includes a wired / wireless charging circuit, a battery, etc., an interface unit that includes at least one port for connection with another device (e.g., an audio input / output port, a video input / output port), and an input / output unit for inputting and outputting image information / signals, audio information / signals, data, and / or information input from a user.

[0090] For example, the device may be a mobile device such as a mobile robot, a vehicle, a train, an aerial vehicle (AV), a ship, etc. In this case, the device may further include at least one of a driving unit including at least one of an engine, a motor, a power train, wheels, brakes, and a steering unit of the device, a power supply unit including a wired / wireless charging circuit, a battery, etc. that supplies power, a sensor unit that senses status information, environmental information, and user information of the device or its surroundings, an autonomous driving unit that performs functions such as path maintenance, speed control, and destination setting, and a position measurement unit that obtains location information of the mobile device through a global positioning system (GPS) and various sensors.

[0091] For example, the device may be an XR device such as an HMD, a head-up display (HUD) installed in a vehicle, a television, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a robot, etc. In this case, the device may further include at least one of a power supply unit that supplies power and includes a wired / wireless charging circuit, a battery, etc., an input / output unit that obtains control information, data, etc. from the outside and outputs the generated XR object, and a sensor unit that senses status information, environmental information, and user information of the device or the surroundings of the device.

[0092] For example, the device may be a robot that can be classified into industrial, medical, household, military, etc. types depending on the purpose or field of use. In this case, the device may further include at least one of a sensor unit that senses status information, environmental information, and user information of the device or its surroundings, and a driving unit that performs various physical actions, such as moving the robot joints.

[0093] For example, the device may be an AI device such as a TV, a projector, a smartphone, a PC, a laptop, a digital broadcasting terminal, a tablet PC, a wearable device, a set-top box (STB), a radio, a washing machine, a refrigerator, digital signage, a robot, a vehicle, etc. In this case, the device may further include at least one of an input unit that acquires various types of data from the outside, an output unit that generates output related to sight, hearing, or touch, a sensor unit that senses status information, environmental information, and user information of the device or its surroundings, and a training unit that trains a model composed of an artificial neural network using learning data.

[0094] The structure of the wireless device illustrated in FIG. 2 may be understood as a part of a RAN node (e.g., a base station, DU, RU, RRH, etc.). That is, the device illustrated in FIG. 2 may be a RAN node. In this case, the device may further include a wired transceiver for front haul and / or back haul communications. However, if the front haul and / or back haul communications are based on wireless communications, at least one transceiver (206) illustrated in FIG. 2 may be used for front haul and / or back haul communications, and a wired transceiver may not be included.

[0095] FIG. 3 illustrates a method for processing a transmission signal applicable to the present disclosure. For example, the transmission signal may be processed by a signal processing circuit. At this time, the signal processing circuit (300) may include a scrambler (310), a modulator (320), a layer mapper (330), a precoder (340), a resource mapper (350), and a signal generator (360). At this time, for example, the operations / functions of FIG. 3 may be performed in the processor (202) and / or the transceiver (206) of FIG. 2. Furthermore, for example, the hardware elements of FIG. 3 may be implemented in the processor (202) and / or the transceiver (206) of FIG. 2. For example, blocks 310 to 360 may be implemented in the processor (202) of FIG. 2. Additionally, blocks 310 to 350 may be implemented in the processor (202) of FIG. 2, and block 360 may be implemented in the transceiver (206) of FIG. 2, and are not limited to the above-described embodiment.

[0096] The codeword can be converted into a wireless signal through the signal processing circuit (300) of FIG. 3. Here, the codeword is an encoded bit sequence of an information block. The information block may include a transport block (e.g., a UL-SCH transport block, a DL-SCH transport block). The wireless signal may be transmitted through various physical channels (e.g., a PUSCH, a PDSCH). Specifically, the codeword can be converted into a bit sequence scrambled by a scrambler (310). The scramble sequence used for scrambling is generated based on an initialization value, and the initialization value may include ID information of the wireless device, etc. The scrambled bit sequence can be modulated into a modulation symbol sequence by a modulator (320). The modulation scheme may include pi / 2-binary phase shift keying (pi / 2-BPSK), m-phase shift keying (m-PSK), m-quadrature amplitude modulation (m-QAM), etc.

[0097] A complex modulation symbol sequence can be mapped to at least one transmission layer by a layer mapper (330). The modulation symbols of each transmission layer can be mapped to corresponding antenna port(s) by a precoder (340). The output z of the precoder (340) can be obtained by multiplying the output y of the layer mapper (330) by a precoding matrix W of NХM, where N is the number of antenna ports and M is the number of transmission layers. Here, the precoder (340) can perform precoding after performing transform precoding (e.g., discrete Fourier transform (DFT) transform) on the complex modulation symbols. Additionally, the precoder (340) can perform precoding without performing transform precoding.

[0098] The resource mapper (350) can map modulation symbols of each antenna port to time-frequency resources. The time-frequency resources can include multiple symbols (e.g., CP-OFDMA symbols, DFT-s-OFDMA symbols) in the time domain and multiple subcarriers in the frequency domain. The signal generator (360) generates a wireless signal from the mapped modulation symbols, and the generated wireless signal can be transmitted to another device through each antenna. To this end, the signal generator (360) can include an inverse fast Fourier transform (IFFT) module, a cyclic prefix (CP) inserter, a digital-to-analog converter (DAC), a frequency uplink converter, and the like.

[0099] The signal processing process for a received signal in a wireless device may be configured in reverse order of the signal processing process (310 to 360) of FIG. 3. For example, a wireless device (e.g., 200 of FIG. 2) may receive a wireless signal from the outside through an antenna port / transceiver. The received wireless signal may be converted into a baseband signal through a signal restorer. For this purpose, the signal restorer may include a frequency downlink converter, an analog-to-digital converter (ADC), a CP remover, and a fast Fourier transform (FFT) module. Thereafter, the baseband signal may be restored to a codeword through a resource demapper process, a postcoding process, a demodulation process, and a descrambling process. The codeword may be restored to the original information block through decoding. Therefore, a signal processing circuit (not shown) for a received signal may include a signal restorer, a resource demapper, a postcoder, a demodulator, a descrambler, and a decoder.

[0100] Figure 4 illustrates a communication procedure between a terminal and a base station applicable to the present disclosure. Figure 4 illustrates operations of a terminal (410) and a base station (420) transmitting and / or receiving data and operations performed prior thereto.

[0101] Referring to FIG. 4, in step 401, the terminal (410) and the base station (420) perform synchronization. For example, the terminal (410) performs an initial cell search operation. Specifically, the terminal (410) can detect at least one synchronization signal transmitted from the base station (420) according to a predefined rule. Here, the synchronization signal can include multiple synchronization signals classified according to structure or purpose (e.g., primary synchronization signal, secondary synchronization signal). Through this, the terminal (410) can check the boundary of the frame, subframe, slot, and / or symbol of the base station (420) and obtain information about the base station (420) (e.g., cell identifier).

[0102] In step 403, the terminal (410) obtains system information transmitted from the base station (420). The system information is information related to the properties, characteristics, and / or capabilities of the base station (420) required to access the base station (420) and use the service, and may be classified according to the content (e.g., whether it is essential for access), transmission structure (e.g., channel used, whether provided on-demand), etc., and may be classified into, for example, a master information block (MIB) and a system information block (SIB). If necessary, the terminal (410) may transmit a signal requesting system information before receiving the system information. However, the request and provision of the system information may be performed after the random access procedure described below.

[0103] In step 405, the terminal (410) and the base station (420) perform a random access procedure. The terminal (410) may transmit and / or receive at least one message (e.g., a random access preamble, a random access response (RAR) message, etc.) for the random access procedure based on information related to the random access channel of the base station (420) obtained through system information (e.g., channel position, channel structure, supported preamble structure, etc.). For example, the terminal (410) may transmit a preamble (e.g., MSG1) through the random access channel, receive an RAR message (e.g., MSG2), transmit a message (e.g., MSG3) including information related to the terminal (410) (e.g., identification information) to the base station (420) using scheduling information included in the RAR message, and receive a message (e.g., MSG4) for contention resolution and / or connection establishment. As another example, MSG1 and MSG3 may be sent and received as one message, or MSG2 and MSG4 may be sent and received as one message.

[0104] In step 407, the terminal (410) and the base station (420) perform signaling of control information. Here, the control information may be defined in various layers, such as a layer that controls a connection (e.g., a radio resource control (RRC) layer), a layer that handles mapping between logical channels and transport channels (e.g., a media access control (MAC) layer), and a layer that handles physical channels (e.g., a physical (PHY) layer). For example, the terminal (410) and the base station (420) may perform at least one of signaling for establishing a connection, signaling for determining settings related to communication, and signaling for indicating allocated resources.

[0105] In step 409, the terminal (410) and the base station (420) transmit and / or receive data. In other words, the terminal (410) and the base station (420) can process, transmit, and / or receive data based on the signaling of the control information. For example, when transmitting data, the terminal (410) or the base station (420) can perform at least one of channel encoding, rate matching, scrambling, constellation mapping, layer mapping, waveform modulation, antenna mapping, and resource mapping on the information bits. Conversely, when receiving data, the terminal (410) or the base station (420) can perform at least one of signal extraction from resources, waveform demodulation for each antenna, signal arrangement considering layer mapping, constellation demapping, descrambling, and channel decoding.

[0106] 6G communication systems and core implementation technologies of 6G systems

[0107] The 5G system defines various operating bands within FR1 (frequency range 1), which covers 410 MHz to 7125 MHz, and FR2 (frequency range 2), which covers 24,250 MHz to 71,000 MHz. Various frequencies are being discussed as operating bands for the subsequent 6G system, and the use of higher frequencies than 5G systems is also being considered for wider bandwidth and higher transmission speeds. One such band is the THz (terahertz) frequency band, which covers approximately 100 GHz to 10 THz. The THz frequency band is a band that has both the transparency of radio waves and the straightness of light waves, and communications using the THz frequency band are expected to play a transitional role from existing radio-centered communications to lightwave-based communications.

[0108] 6G systems utilizing the THz frequency band are aimed at i) very high data rates per device, ii) a very large number of connected devices, iii) global connectivity, iv) very low latency, v) reducing energy consumption of battery-free IoT devices, vi) ultra-reliable connectivity, and vii) connected intelligence with machine learning capabilities. The vision of the 6G system can be divided into four aspects: “intelligent connectivity,” “deep connectivity,” “holographic connectivity,” and “ubiquitous connectivity,” and the 6G system can be designed to satisfy the requirements as shown in [Table 1] below.

[0109] Per device peak data rate1 TbpsE2E latency1 msMaximum spectral efficiency100 bps / HzMobility supportup to 1000 km / hrSatellite integrationFullyAIFullyAutonomous vehicleFullyXRFullyHaptic CommunicationFully

[0110] At this time, the 6G system may have key factors such as enhanced mobile broadband (eMBB), ultra-reliable low latency communications (URLLC), massive machine type communications (mMTC), AI integrated communication, tactile internet, high throughput, high network capacity, high energy efficiency, low backhaul and access network congestion, and enhanced data security.

[0111] Figure 5 illustrates an example of a communication structure that can be provided in a 6G system applicable to the present disclosure. Referring to Figure 5, a 6G system is expected to have 50 times higher simultaneous wireless communication connectivity than a 5G wireless communication system. URLLC, a key feature of 5G, is expected to become an even more important technology in 6G communications by providing end-to-end latency of less than 1 ms. Furthermore, 6G systems will have significantly superior volumetric spectral efficiency, unlike the frequently used area spectral efficiency. 6G systems can provide extremely long battery life and advanced battery technologies for energy harvesting, so mobile devices in 6G systems may not need to be separately charged.

[0112] As core implementation technologies of the 6G system, technologies such as artificial intelligence (AI), THz (terahertz) communication, optical wireless technology, FSO backhaul network, massive MIMO technology, blockchain, 3D networking, quantum communication, unmanned aerial vehicles, cell-free communication, wireless information and energy transfer (WIET), integration of sensing and communication, integration of access backhaul networks, holographic beamforming, big data analysis, and large intelligent surface (LIS) can be adopted.

[0113] For example, THz communication is a communication that utilizes a spectrum in a frequency band between 0.1 THz and 10 THz with a corresponding wavelength in the range of 0.03 mm to 3 mm as shown in Fig. 6, and can be implemented using circuit elements having a structure as shown in Fig. 7. In addition, optical wireless technology is a technology that generates and modulates THz signals using optical elements, and can be implemented based on devices having structures as shown in Figs. 8, 9, 10, and 11.

[0114] In addition, artificial intelligence can be implemented based on various models such as neural networks and machine learning (machine models). For example, an artificial intelligence model of a neural network structure can be based on the structure of a perceptron as in Fig. 12. Referring to Fig. 12, an artificial neural network can be composed of multiple perceptrons. According to the structure of the perceptron, when an input vector x={x1, x2,... xd} is input, each component is multiplied by a weight {W1, W2,... Wd}, and all the results are added, and then the activation function σ(·) is applied. A large artificial neural network structure can be formed by extending the simplified perceptron structure illustrated in Fig. 12, and the input vector can be applied to perceptrons of different dimensions. When perceptrons are stacked, a neural network having an input layer, a hidden layer, and an output layer as in Fig. 13 can be configured.

[0115] Specific embodiments of the present invention

[0116] The present disclosure relates to semantic communication in a wireless communication system, and more particularly, to a technique for detecting and correcting errors occurring in a semantic representation (SR) transmitted and received for semantic communication. More specifically, the present disclosure proposes various embodiments for detecting and correcting semantic errors at a destination based on multiple attention maps generated and transmitted at a source in a system supporting semantic communication, and a union attention map utilizing the multiple attention maps.

[0117] Problems related to communication can be divided into three levels, as illustrated in Figure 14, according to the philosophy of Shannon and Weaver. Figure 14 illustrates a communication model applicable to the present disclosure. The problem at Level A (1410) is a technical problem related to how accurately symbols in communication can be conveyed. The problem at Level B (1420) is a semantic problem related to how accurately the conveyed symbols convey the desired meaning. The problem at Level C (1430) is an effectiveness problem related to how effectively the received meaning influences the operation in the desired manner.

[0118] Shannon's information theory focuses only on technical problems at Level A (1410). However, Weaver explains that Shannon's information theory is general enough to consider problems at Level B (1420) and Level C (1430) by adding semantic transmitters, semantic receivers, and semantic noise to Shannon's communication model.

[0119] Meanwhile, one of the various goals of 6G communication is to enable various new services that interconnect people and machines. Therefore, a semantic communication method that considers the semantic issues of Level B (1420) beyond the technical issues of Level A (1410) in communication systems needs to be provided. Semantic communication refers to the efficient transmission and reception of semantic information by utilizing common background knowledge between a first device and a second device, respectively corresponding to the source and destination. Referring to the communication model of Figure 14, if the meaning of the intended message sent by the first device, which corresponds to the source, is accurately interpreted by the second device, which corresponds to the destination, it can be said that correct semantic communication has been performed.

[0120] For semantic communication, a source can generate a semantic representation based on given or collected raw data and transmit the generated semantic representation to a destination. The destination interprets and reasons the received semantic representation to align with the source's intent. Semantic communication requires an approach not from the perspective of reducing reconstruction errors that occur during the process of restoring the received semantic representation to the original raw data, but from the perspective of whether the downstream task performed by the destination can operate according to the source's intent using the received semantic representation. Thus, the semantic representation generated by the source and transmitted to the destination must be generated with consideration for the downstream task performed by the destination. That is, a task-oriented semantic communication system is needed that can generate semantic representations based on whether a task operation is performed well at the destination, and the task-oriented semantic communication system can preserve task-relevant information while introducing useful invariances to downstream tasks.

[0121] As described above, to perform semantic communication, a new layer for semantic communication, such as a semantic layer, can be defined that defines the overall operation of semantic expressions and messages. For example, a semantic layer can be located at each of the source and destination, reflecting a task-oriented semantic communication system. To perform communication between the semantic layers located at each of the source and destination, a protocol, which is a convention between the layers, and a series of operation processes need to be defined.

[0122] In semantic communication, messages received at the destination may contain errors due to channel noise between the source and destination. Errors can occur at either the technical or semantic level. The technical level seeks to preserve the syntactic integrity of the transmitted and received messages. Therefore, errors at the technical level can be defined as syntactic differences between the transmitted and received messages. In contrast, the semantic level does not seek to preserve the syntactic integrity of the transmitted and received messages, but rather seeks to infer similarities between the semantics of the transmitted and received messages. Therefore, errors at the semantic level can be defined as semantic differences between the transmitted and received messages.

[0123] An example of a semantic error is shown in Figure 15. Figure 15 illustrates an example of a semantic communication error. Figure 15 exemplifies a case where the source is a person and conveys a message through speech. Referring to Figure 15, the source attempts to transmit the message "copy machine" to the destination using sound. However, since the pronunciation of "p" is similar to the pronunciation of "ff," the destination may misinterpret the message as "coffee machine," contrary to the source's intention. This can be viewed as a semantic error that occurs when the semantic information transmitted from the source is misinterpreted at the destination. To prevent this type of semantic error, the source can transmit a different word, "Xerox," which has the same meaning, instead of the word "copy machine." In this case, the additionally received information and the previously received information are used together for semantic interpretation, so the semantic error can be prevented.

[0124] As mentioned above, semantic communication can result in semantic mismatches between the source and destination due to semantic errors. To ensure reliable semantic communication, the destination must detect semantic mismatches using semantic error checking, correct semantic errors, or request retransmission of semantic information from the source, thereby obtaining the exact semantic information intended by the source.

[0125] To detect semantic errors, attention techniques, first introduced in the field of neural machine translation (NMT), can be used. According to attention techniques, the decoder can refer back to the encoder's input sequence at each step of predicting the output word. In other words, the decoder does not consider all words in the input sequence with equal weight, but rather focuses on words highly related to the predicted word. This technique is called attention, and the information indicating the focused portion can be determined using an attention function.

[0126] Figure 16 illustrates an example of visualizing the association between words in a translation task using the attention technique. The attention technique can be applied to sequence modeling between input English words and output French words. Here, the sequence modeling between input English words and output French words can include weight values ​​expressed as a correlation matrix. In Figure 16, the closer the pixel color is to white, the higher the association between words. Referring to Figure 16, it can be confirmed that even if the order in which English and French sentences are written differs, the weight is higher for parts that are to be translated using words that have an association regardless of the position.

[0127] A transformer has been introduced, designed to focus on all important parts of an input, regardless of its length, by applying attention techniques to any network architecture. Using transformers, techniques capable of processing various data modalities, such as text, images, and graphs, are being studied. An example of a vision transformer for situations where the data modality is image is shown in Figure 17. Figure 17 illustrates an example of a vision transformer model. Referring to Fig. 4, a class token (1710) is used to perform learning in the direction of matching the true label of the input image with the input value of the encoder of the transformer, tokens (1720) that tokenize the input image in units of patches, and a distillation token (1730) that plays a role in learning the knowledge of the teacher model by reducing the error with the inference value of the teacher model. Here, the class token (1710) and the distillation token (1730) are randomly initialized before being input to the transformer structure, and are changed to optimized values ​​while passing through multiple layers.

[0128] As mentioned above, attention techniques focus on the parts of input data that are highly relevant to the expected output for data processing. When applying attention techniques to semantic communication, the source can use attention techniques to generate and convey semantic representations to express the intended message. Based on the semantic representations conveyed using attention techniques, the destination can determine which parts of the original data to focus on to better interpret the meaning the source intends to convey. Therefore, utilizing attention techniques in semantic communication is desirable both for expressing the intended meaning and for interpreting the conveyed meaning.

[0129] Looking at previous research, systems are typically built using the Transformer architecture, focusing on reconstruction of input data for training, and the trained model is typically used for inference. Furthermore, existing research ignores the attention map (e.g., attention matrix), which is the output of attention techniques, and does not utilize attention maps to detect and correct semantic errors.

[0130] As mentioned above, in a task-oriented semantic communication system, it is necessary to ensure that the tasks located at the destination perform their tasks properly in accordance with the intent conveyed by the source. Here, the tasks are performed to provide the service intended by the source. In the case of standards, metrics such as QoS, which indicates the quality of service provided by the service provider, and QoE, which indicates the quality experienced and perceived by the user, can be set differently for each service. In other words, different metrics can be set depending on the task. Therefore, a task located at the destination can be considered equivalent to performing a single service operation. Furthermore, since a single system can provide not only a single service but also multiple services, it is reasonable to consider multi-tasks in a task-oriented semantic communication system.

[0131] In a task-oriented semantic communication system that performs multiple tasks, introducing semantic error detection and correction based on an attention map can extract a single attention map related to the area to be focused on and the meaning to be conveyed from the input data, and perform semantic error detection and correction based on the single attention map. However, since the service requirements of each multi-task targeted by the system are different, the area to be focused on from the input data required by each task may differ. However, since the data area extracted from the single attention map received at the destination does not contain the information required by each task, if each task is performed based on the data obtained through semantic error detection and correction based on a single attention map, the metrics related to the requirements of some tasks may not be satisfied.

[0132] Therefore, in order to perform semantic error detection and correction while performing multi-task operations similar to providing operations of multiple services in semantic communication, the present disclosure proposes a technique for performing semantic error detection and correction based on a union attention map that includes all semantic information required by each task by using multiple attention maps obtained by extracting attention maps for each task, and utilizing the same data in multiple tasks. In other words, the present disclosure proposes a technique for performing semantic error detection and correction based on a union attention map by using multiple attention maps generated through an attention technique for each task supported in a system supporting semantic communication. The proposed technique is related to a semantic level corresponding to Level B in FIG. 14, and can be applied to bidirectional communication.

[0133] For example, in a semantic communication system, multi-tasks can be performed as follows. A source generates and transmits a semantic representation, a destination reconstructs the received semantic representation at the semantic level, and uses the reconstructed data at the semantic level as input to multiple downstream tasks located at the destination to obtain the task's output. Here, the reconstructed data at the semantic level does not mean restoring the input data to a perfect form, but rather means performing the operations of the downstream tasks normally or in a manner consistent with the intent, thereby obtaining an output corresponding to the operation intended by the source at the destination. For example, when the modality of the input data is an image, an example of performing multiple downstream tasks at the destination is as shown in FIG. 18.

[0134] Figure 18 illustrates an example of performing multiple downstream tasks. Referring to Figure 18, the source and destination may possess labeled datasets necessary for multi-task learning. For example, the source may be an entity transmitting data, such as a terminal, base station, or other device. Furthermore, the destination may be an entity receiving data and performing tasks appropriate to its purpose, such as a terminal, base station, or other device. In other words, the source and destination may be each entity performing communication and may not be limited to a specific type of device. For convenience of explanation, the following description will be based on the source and destination. For example, the source and destination may possess labeled datasets necessary for task learning, and the destination may perform multiple tasks. Here, each of the multiple tasks may be an operation based on a different purpose, and the semantic representation may be transmitted from the source, and the reconstructed semantic level data may be transmitted as input to each of the multiple tasks. For example, each of the multiple tasks can operate for different purposes based on its input.

[0135] Referring to FIG. 18, a source can encode (e.g., compress) data given as an input image (1810) to generate a semantic representation (e.g., a latent representation) (1820). The semantic representation (1820) is transmitted to a destination through a channel. The destination can receive the semantic representation (1830) through the channel and obtain reconstructed data (1840) at a semantic level by decoding (e.g., decompression) the semantic representation (1830). Thereafter, the destination can use the reconstructed data (1840) at a semantic level as input for each of multiple tasks, perform an operation appropriate for the purpose of each task, and then confirm the output.

[0136] The system according to various embodiments is based on a structure as shown in FIG. 18, and the source and destination perform inference operations using an encoder and decoder to which an attention technique is applied and on which training has been completed, and the destination can operate various downstream tasks. In particular, the present disclosure considers a system that treats a data region to be focused on, related to the meaning to be conveyed from the entire data, as a semantic element, and performs the operation in accordance with the intention of a downstream task using the corresponding semantic information. Here, the reconstructed error at the semantic level can be understood as a reconstructed error for the data region to be focused on in relation to the meaning to be conveyed, rather than a reconstructed error for the entire data to be conveyed from the source. Therefore, when measuring the reconstructed error at the semantic level, the reconstructed error in a part other than the region to be focused may not be considered. For example, the error rate for the reconstructed error at the semantic level can be calculated as shown in [Mathematical Equation 1] below.

[0137]

[0138] In [Equation 1], is the error rate for restoration errors at the semantic level, refers to the ratio of the area of ​​the union attention map based on the restored data at the semantic level to the area of ​​the union attention map based on the data to be transmitted.

[0139]

[0140] FIG. 19 illustrates the functional structure of a source for performing union attention map-based semantic error detection and correction according to one embodiment of the present disclosure. FIG. 19 illustrates the structure of a device that functions as a source using multiple encoders to perform union attention map-based semantic error detection and correction utilizing multiple attention maps.

[0141] Referring to FIG. 19, the source includes a semantic data retransmission count checker (1902) for checking a retransmission count value, a bitwise attention map receipt checker (1904) for checking the reception of an attention map, a plurality of encoders (1906-1 to 1906-N) for generating semantic representations for different tasks, a semantic representation generator (1908) for combining a plurality of semantic representations, an attention map generator (1910) for generating an attention map for each semantic representation, a semantic packet generator (1912) for generating at least one packet for transmitting at least one semantic representation and at least one attention map, a saving data block (1914) for storing information required for semantic error correction, and an original It includes a residual area data extractor (1916) that extracts residual area data from data, and a system failure operator (1918) that determines the failure of semantic error correction. Here, the semantic representation generator (1908) can combine multiple semantic representations according to a predefined algorithm (e.g., mathematical operation such as averaging) or combine multiple semantic representations using a trained neural network.

[0142] In the example of FIG. 19, multiple encoders (1906-1 to 1906-N) are included, and one encoder can be mapped to each task, or one encoder can be mapped to multiple tasks. In addition, the attention map generator (1910) can generate an attention map that sets a semantic region in the input data so that the multiple encoders (1906-1 to 1906-N) included in the source perform tasks at the destination at a semantic level in accordance with the intent of the source, thereby providing a desired service. That is, if K services are provided at the destination, K tasks can be performed at the destination, and the number of encoders (1906-1 to 1906-N) used to generate the multiple attention map can be greater than 1 and less than or equal to K. This is because, although the services provided by the tasks can be different, the region to be focused on in the data used as input to the tasks can be set to be the same. Therefore, N attention maps can be generated.

[0143] If the source lacks the computational power or resources required to operate multiple encoders, or if the destination lacks the computational power or resources required to operate multiple encoders, a structure including a unified encoder (2006) that supports processing for multiple tasks, as shown in FIG. 20, may be applied. If the unified encoder (2006) as shown in FIG. 20 is used at the source, in order to generate multiple attention maps, metrics required to generate attention maps tailored to the semantic domains required for each service operation may be set differently so that the service operations provided by the tasks can be performed in accordance with the intent of the source. In addition, different attention maps may be generated using the metrics.

[0144] At this time, if K services are provided at the destination, K tasks can be performed at the destination, and the number of metrics M used to generate multiple attention maps can be set to be greater than 1 and less than or equal to K. This is because, similar to the explanation referring to Fig. 19, although the services provided by the tasks may be different, the areas to be focused on in the data used as input to the tasks can be set to be the same. Therefore, M attention maps can be generated.

[0145] The process of creating a union attention map using a multi-attention map through a structure such as Fig. 19 or Fig. 20 is expressed in a formula as follows: [Mathematical Formula 2], [Mathematical Formula 3], and [Mathematical Formula 4].

[0146]

[0147] In [Equation 2], is a set of multiple attention matrices, Extraction of multi-head attention-based attention matrix obtained from each layer to which the encoder's attention technique is applied / Operation on attention matrix (e.g., attention rollout, gradient attention rollout) / Multiplication of attention matrices that have gone through a normalization process to make the rows of the attention matrix 1, which is a set of final attention aware matrices A function that generates Is Input data to be passed from the source to the destination as an argument, silver The number of semantic regions required to perform semantic error detection and correction so that each task corresponding to the service provided by the destination as an argument can be performed in accordance with the source's intent. silver It refers to the attention matrix related metric set to extract the attention map as an argument. In the following description, silver can be referred to as

[0148] In relation to this, each task may require its own semantic domain, and multiple tasks may require a common semantic domain. In relation to this, when the structure of Fig. 19 is used, since different encoders are used for each task, the same metric can be applied or different metrics can be set according to the requirements of the task. In addition, In relation to this, when the structure of Fig. 20 is used, an integrated encoder is used, Different metrics can be set for different values.

[0149] Here, examples of metrics are as follows. For example, Max, Min, and Mean can be used as fusion metrics. Since each of the multiple attention heads associated with the attention matrix extracted for each layer of the encoder can have different areas of interest or focus in the input sequence, the fusion metric can be used to make the attention map point to a more local area by applying different metrics to multiple heads. As another example, the discard ratio can be used. Since it is necessary to focus on high-order attention scores rather than each attention score or value that constitutes the attention matrix, the discard ratio can be set to discard low-order attention scores at a certain rate. By using the discard ratio, the area to be focused on in the data can be better distinguished.

[0150]

[0151] In [Equation 3], is a set of multiple attention maps, is a set of bitwise attention maps from a set of multiple attention matrices. A function that generates is a set of multiple attention matrices, Is Arguments that control the behavior of refers to a reference value that considers the part that determines whether the data for which restoration at the semantic level is performed at the destination is normal. In the following description, silver as, Is can be referred to as

[0152] If is set to 1, Is Generate a bit-wise attention map based on If is set to 0, By acting as an identity function, Prints out. In relation to, The value of each element of is, The corresponding element of each matrix in If it is less than the value of , it is set to 0, otherwise it is set to 1.

[0153]

[0154] In [Equation 4], is a union attention map, Is Obtained through A function that generates a single union attention map by combining refers to a set of multiple attention maps. In the following description, Is can be referred to as

[0155] FIG. 21 illustrates the functional structure of a destination for performing union attention map-based semantic error detection and correction according to one embodiment of the present disclosure. FIG. 21 illustrates the structure of a device functioning as a destination that uses multiple encoders to perform union attention map-based semantic error detection and correction utilizing multiple attention maps. The structure of FIG. 21 can be applied when a source adopts the structure of FIG. 19.

[0156] Referring to FIG. 21, the destination comprises a semantic packet decomposer (2102) that obtains data from at least one semantic packet received from a source, a decoder (2104) that reconstructs a semantic expression obtained from at least one semantic packet at a semantic level, a reconstructed data type checker (2106) that verifies the type of data reconstructed at a semantic level (e.g., original data, residual region data), a checker for performing semantic error checking (2108) that determines whether to perform semantic error checking based on settings, a data type configurator (2110) that sets the type of data and whether to correct errors, a plurality of encoders (2112-1 to 2112-N) that generate semantic expressions for error checking and / or error correction, and attention maps that generate attention maps based on attention matrices. An attention map generator (2114), a semantic error checker (2116) for determining a semantic error based on information related to an attention map provided from a data storage block (2120) that stores information related to an attention map received from a source or data for checking and correcting semantic errors and attention map information generated by the attention map generator (2114), an early stopping checker (2118) for determining whether to stop early in response to the occurrence of a semantic error, a data storage block (2120) for storing data for checking and correcting errors, an attention map based packet generator (2122) for generating at least one packet including information related to an attention map fed back to a source, a system failure operator (2124) for determining a failure in correcting a semantic error,It includes a reconstructed data type checker (2126) that checks the type of data restored at a semantic level (e.g., whether it is residual region data), a data compositor (2128) that synthesizes residual region data restored at a semantic level into previously stored data, and a plurality of downstream task blocks (213-1 to 2130-N) that perform tasks according to settings.

[0157] If the source includes N encoders, as in Fig. 19, the destination may also include N encoders. The decoder uses the decoder to restore data at a semantic level from the received semantic representation. The data restored at a semantic level is provided as input data to multiple encoders, and the destination may use the attention map generator to extract N attention maps representing a region, use the extracted attention maps to create a union attention map, and use the union attention map for semantic error detection and correction.

[0158] If the source adopts a structure as in FIG. 20, an integrated encoder (2212) as in FIG. 22 may be used to support multi-task processing based on data restored to a semantic level at the destination. At this time, if the source uses M metrics as in FIG. 20, the destination can also obtain M attention maps using the M metrics. In addition, if the destination uses an integrated encoder (2212) as in the structure of FIG. 20, the destination can generate different attention maps using metrics related to attention map generation according to the number of semantic regions required for each service operation used in the source, in order to generate multiple attention maps. Based on this, the destination can generate a union attention map and perform semantic error detection and correction based on the comparison result with the union attention map for information received from the source. For example, if the destination lacks the computational power or resources required to operate multiple encoders, the source may adopt a structure like FIG. 20, and the destination may adopt a structure like FIG. 22.

[0159] Figure 23 illustrates an example of a procedure for performing semantic communication by a first device according to one embodiment of the present disclosure. Figure 23 illustrates a method performed by the first device. The first device acts as a source or destination for performing semantic communication, and may be a terminal, a base station, or any entity defined in a wireless communication system.

[0160] Referring to FIG. 23, at step S2301, the first device establishes a connection with the second device. To this end, the first device may receive a synchronization signal from the second device, acquire synchronization with the second device based on the synchronization signal, and perform signaling to establish the connection. Alternatively, the first device may transmit a synchronization signal and perform signaling to establish the connection.

[0161] In step S2303, the first device transmits capability information to the second device. In other words, the first device may transmit a message including capability information to the second device. Prior to this, although not illustrated in FIG. 23, the first device may receive a request for capability information from the second device. Here, the capability information may include information regarding functional and hardware capabilities related to communication of the first device. In one embodiment, the capability information may include capability information related to semantic communication, particularly semantic error detection and correction.

[0162] In step S2305, the first device receives configuration information related to communication from the second device. The configuration information may be received via at least one message. The configuration information may indicate resources for communication, data processing related to communication, values ​​of variables related to communication, etc. Accordingly, the first device may store the configuration information and set variables, parameters, etc. related to a receiver or transmitter for communication. According to various embodiments, the configuration information may include various information, variables, parameters, etc. for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to a plurality of tasks.

[0163] In step S2307, the first device performs semantic communication based on the configuration information. In addition, the first device may support or perform semantic error detection and correction based on the configuration information. If the first device is a source, the first device may transmit at least one semantic expression for multi-tasks, and further, according to one embodiment, may transmit attention map information corresponding to the semantic expression, and, if necessary, may perform retransmission based on a residual region used for semantic error correction. If the first device is a destination, the first device may perform data restoration at a semantic level from the received at least one semantic expression for multi-tasks, detect semantic errors in the data restored at the semantic level, and, depending on whether a semantic error exists, may request retransmission based on a residual region used for semantic error correction so as to perform a semantic error correction operation.

[0164] Figure 24 illustrates an example of a procedure for performing semantic communication by a second device according to one embodiment of the present disclosure. Figure 24 illustrates a method performed by the second device. The second device acts as a destination or source for performing semantic communication, and may be a base station, a terminal, or any entity defined in a wireless communication system.

[0165] Referring to FIG. 24, at step S2401, the second device establishes a connection with the first device. To this end, the second device may transmit a synchronization signal and perform signaling to establish the connection. Alternatively, the second device may receive a synchronization signal from the first device, acquire synchronization with the first device based on the synchronization signal, and perform signaling to establish the connection.

[0166] In step S2403, the second device receives capability information from the first device. In other words, the second device may receive a message including capability information from the first device. Prior to this, although not illustrated in FIG. 24, the second device may transmit a request for capability information to the first device. Here, the capability information may include information regarding functional and hardware capabilities related to communication of the second device. In one embodiment, the capability information may include capability information related to semantic communication, particularly semantic error detection and correction.

[0167] In step S2405, the second device transmits configuration information related to communication to the first device. The configuration information may be transmitted via at least one message. The configuration information may indicate resources for communication, data processing related to communication, values ​​of variables related to communication, etc. Accordingly, the second device may store the configuration information and set variables, parameters, etc. related to a receiver or transmitter for communication. According to various embodiments, the configuration information may include various information, variables, parameters, etc. for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to a plurality of tasks.

[0168] In step S2407, the second device performs semantic communication based on the configuration information. In addition, the second device may perform or support semantic error detection and correction based on the configuration information. If the second device is a destination, the second device may perform data restoration at the semantic level from at least one semantic expression for the received multi-task, detect semantic errors in the data restored at the semantic level, and, depending on whether an error exists, request retransmission based on the residual region used for semantic error correction so as to perform a semantic error correction operation. If the second device is a source, the second device may transmit at least one semantic expression for the multi-task, and further, according to one embodiment, transmit attention map information corresponding to the semantic expression, and, if necessary, perform retransmission based on the residual region used for semantic error correction.

[0169] Figure 25 illustrates an example of a procedure for supporting semantic error correction in a device operating as a source according to one embodiment of the present disclosure. Figure 25 illustrates a method performed by a device operating as a source. Here, the device operating as a source may be a terminal or a base station.

[0170] Referring to FIG. 25, in step S2501, the source generates and transmits at least one packet containing semantic information for multi-tasks. Here, the semantic information may include at least one semantic representation generated from data. When multi-tasks are intended, the semantic information may include a unified semantic representation corresponding to a plurality of semantic representations generated by a plurality of encoders or a unified semantic representation generated by a unified encoder. In addition, the semantic information may include attention information for semantic error detection and / or correction. The attention information is related to an attention matrix and / or an attention map. According to various embodiments, the attention information may include a task-specific attention map or a union attention map. Specifically, the source may perform semantic encoding based on an attention technique on the data to generate semantic information. The output of semantic encoding includes at least one semantic representation, and attention information can be derived during the process of semantic encoding.

[0171] In step S2503, the source receives feedback information for semantic error correction. The feedback information includes information related to a mask generated at the destination, indicating a residual region within the entire data region required for semantic error correction. Specifically, the feedback information may include the mask or information required to generate the mask. For example, the information required to generate the mask may include attention information generated at the destination (e.g., a union attention map or a task-specific attention map).

[0172] In step S2505, the source generates and transmits at least one retransmission packet containing semantic information related to the residual region data. In other words, the source generates and transmits a retransmission packet related to semantic error correction. Based on the feedback information, the source can identify a residual region, which is a portion of data required for semantic error correction. Then, the source can generate semantic information for the residual region and transmit at least one retransmission packet containing a semantic expression and corresponding attention information. In other words, the source can transmit a packet related to semantic error correction. Specifically, the source can perform semantic encoding based on an attention technique on a portion of data included in the residual region for semantic error correction to generate semantic information. The output of the semantic encoding includes at least one semantic expression, and the attention information can be derived during the semantic encoding process.

[0173]

[0174] Figure 26 illustrates an example of a procedure for performing semantic error correction on a device operating as a destination according to one embodiment of the present disclosure. Figure 26 illustrates a method performed by a device operating as a destination. Here, the device operating as a destination may be a terminal or a base station.

[0175] Referring to FIG. 26, in step S2601, the destination receives at least one packet containing semantic information for a multi-task. Here, the semantic information may include an integrated semantic representation corresponding to a plurality of semantic representations generated by a plurality of encoders or an integrated semantic representation generated by an integrated encoder. In addition, the semantic information may include attention information for semantic error detection and / or correction. The attention information is related to an attention matrix and / or an attention map generated by the source. According to various embodiments, the attention information may include a task-specific attention map or a union attention map. Specifically, the destination may perform semantic encoding based on an attention technique on data restored to a semantic level to generate semantic information. The output of the semantic encoding includes at least one semantic representation, and the attention information may be derived during the semantic encoding process.

[0176] In step S2603, the destination generates and transmits feedback information for semantic error correction based on attention information for the multi-task. To this end, the destination performs semantic decoding on the semantic representation, and determines a semantic error rate based on the matching information between the attention information derived from the semantic encoding process for the data restored to the semantic level obtained through semantic decoding and the attention information received from the source. At this time, if the semantic error rate exceeds a threshold, the destination may determine to perform a semantic error correction procedure and transmit feedback information. The feedback information includes information related to a mask indicating a residual region required for semantic level error correction, which is part of the data restored to the semantic level. Specifically, the feedback information may include the mask or information necessary for generating the mask. For example, the information necessary for generating the mask may include attention information generated at the destination (e.g., a union attention map or a task-specific attention map).

[0177] In step S2605, the destination receives at least one retransmission packet containing semantic information related to the residual region data. In other words, the destination receives a retransmission packet related to semantic error correction. Here, the received retransmission packet contains semantic information about the residual region generated by the source. Specifically, the destination may perform semantic encoding based on an attention technique on a portion of the data included in the residual region to generate the semantic information. The output of the semantic encoding includes at least one semantic expression, and the attention information may be derived during the semantic encoding process.

[0178] In step S2607, the destination performs semantic error correction based on at least one retransmission packet. In other words, the destination performs semantic error correction based on the retransmission packet related to semantic error correction. The destination obtains data restored to the semantic level for the remaining region by performing semantic decoding on at least one semantic expression included in at least one retransmission packet related to semantic error correction. Then, the destination synthesizes the data restored to the semantic level for the remaining region and the data restored to the semantic level obtained in step S2601, and performs semantic encoding on the synthesized data.

[0179] Subsequently, although not illustrated in Figure 26, the destination can determine a semantic error rate based on the semantic encoding results for the synthesized data. Depending on the semantic error rate, the destination can perform downstream tasks or transmit feedback information for additional semantic error correction. At this time, the additional semantic error correction can be selectively performed based on the semantic error rate as well as the maximum number of allowed retransmissions.

[0180] Figure 27 illustrates an example of a procedure supporting error detection and correction based on a union attention map according to one embodiment of the present disclosure. Figure 27 illustrates a method performed by a device operating as a source. At least one of the operations illustrated in Figure 27 may be omitted depending on circumstances and / or settings.

[0181] Referring to Figure 27, in step S2701, the source checks whether there is data to be transmitted. If there is data to be transmitted, in step S2703, the source sets a variable indicating the number of times the semantic data is retransmitted. Check whether is less than the threshold. Here, the threshold is It could be. If the threshold is greater than the threshold, in step S2705, the source performs a system failure action. For example, the source may perform a reconfiguration for semantic error correction in response to the system failure or transmit information related to new data. As another example, the source may reset the semantic data retransmission count to 0 and flush the stored attention map information.

[0182] If the value is below the threshold, in step S2705, the source checks whether attention map information has been received. Here, attention map information is transmitted by the destination to request retransmission of data for correction of semantic errors. If attention map information has not been received, in step S2707, the source determines whether the variable is set as new transmission data. If attention map information is received, in step S2709, the source is a variable Increases by 1. Then, in step S2711, the source extracts residual region-based data based on attention map information. Here, the attention map information may include at least one of attention map information received from the destination or attention map information extracted based on original data. In step S2713, the source extracts a variable Set to residual area-based data.

[0183] Afterwards, in step S2715, the source checks whether a multi-encoder is used. Whether a multi-encoder is used can be determined based on the initial settings. If a multi-encoder is not used, in step S2717, the source performs encoding using the integrated encoder. That is, the source For the data set in , semantic encoding based on the attention technique is performed. At this time, at least one of the metrics for obtaining the attention matrix, the attention head fusion metric, and the discard ratio can be used for encoding. On the other hand, if multiple encoders are used, the source performs encoding using multiple encoders in step S2719. That is, the source For the data set in , semantic encoding based on the attention technique is performed. Then, in step S2721, the source generates an integrated semantic representation. That is, the source integrates multiple semantic representations generated by multiple encoders into a single semantic representation.

[0184] In step S2723, the source generates multiple attention maps. Then, in step S2725, the source determines whether bitwise operations are performed. If bitwise operations are performed, in step S2727, the source converts each attention map into a bitwise attention map. For this purpose, a threshold This can be utilized. Then, in step S2729, the source determines whether to generate a union attention map. If a union attention map is generated, in step S2731, the source performs a union operation using multiple attention maps. Then, in step S2733, the source transmits at least one semantic packet. The semantic packet may include at least one of a semantic expression, a union semantic map, or multiple semantic maps. Then, in step S2735, the source stores the union attention map or multiple attention map transmitted to the destination.

[0185] Figure 28 illustrates an example of a procedure for performing union attention map-based error detection and correction according to one embodiment of the present disclosure. Figure 28 illustrates a method performed by a device operating as a destination. At least one of the operations illustrated in Figure 28 may be omitted depending on circumstances and / or settings.

[0186] Referring to FIG. 28, in step S2801, the destination can receive at least one semantic packet from the source. Here, the semantic packet can include at least one semantic expression, at least one attention map set, or at least one union attention map. In step S2803, the destination decomposes at least one semantic packet. Through this, the destination can obtain at least one semantic expression, at least one attention map set, or at least one union attention map. In step S2805, the destination performs decoding on at least one semantic expression. Through this, the destination can obtain data restored to a semantic level.

[0187] Afterwards, in step S2807, the destination verifies the type of data restored to the semantic level. If the data restored to the semantic level is data restored to the semantic level based on the original data, in step S2809, the destination checks the type of data restored to the semantic level. Set the data restored to the semantic level based on the original data. If the data restored to the semantic level is data restored to the semantic level based on the residual region for semantic error correction, in step S2811, the destination determines whether to perform semantic error checking. If semantic error checking is performed, in step S2813, the destination determines whether to perform a variable is set as restored data based on the residual region. If semantic error checking is not performed, in step S2815, the destination synthesizes data. That is, the destination performs data synthesis as a process of semantic error correction. For example, the destination can synthesize data restored to a semantic level based on the residual region and data restored to a stored semantic level (e.g., data subject to semantic error correction). Then, in step S2817, the destination sets the variable Set the data restored to the semantic level by going through a semantic error correction process based on synthetic data.

[0188] Afterwards, in step S2819, the destination checks whether a multiple encoder is used. If a multiple encoder is used, in step S2821, the destination performs encoding using the multiple encoder. If a multiple encoder is not used, in step S2823, the destination performs encoding using the integrated encoder. That is, the destination For the data set in , semantic encoding based on the attention technique is performed using a multi-encoder or an integrated encoder. Then, in step S2825, the destination generates a multi-attention map. In step S2827, the destination determines whether bit-wise conversion of the attention map is necessary. If bit-wise conversion is necessary, in step S2829, the destination converts each attention map into a bit-wise attention map. For this purpose, a threshold This can be utilized. And, in step S2831, the destination generates a union attention map.

[0189] In step S2833, the destination checks whether multiple attention maps have been received. If multiple attention maps have been received, in step S2835, the destination determines whether bit-wise transformation of the received attention maps is required. If bit-wise transformation is required, in step S2837, the destination transforms each received attention map into a bit-wise attention map. For this purpose, a threshold value is used. This can be utilized. Then, in step S2839, the destination performs a union operation using multiple attention maps. Through this, the destination can obtain the union attention map from the source. Then, in step S2841, the destination attempts to detect semantic errors. A semantic error is detected by combining the union attention map from the source, the union attention map from the destination, and the threshold. can be detected using . In step S2843, the destination checks whether a semantic error has occurred.

[0190] If no semantic error is detected, in step S2845, the destination verifies the input data type of the encoder. If the data type is data restored to a semantic level based on the residual region, the destination proceeds to step S2815 and performs data synthesis, which is the semantic error correction process described above. On the other hand, if the data type is not data restored to a semantic level based on the residual region, that is, if the data type is data restored to a semantic level based on the original data or synthesized data at a semantic level that has undergone a semantic error correction process, in step S2847, the destination performs a downstream task.

[0191] On the other hand, if a semantic error is detected, the destination determines whether to perform an early termination in step S2849. If it is determined that an early termination will be performed, the destination performs a system failure operation in step S2851. If it is determined that an early termination will not be performed, the destination stores data (e.g., data restored to the semantic level, synthesized data at the semantic level that has gone through a semantic error correction process, attention map information, etc.) in step S2853. Then, in step S2855, the destination transmits attention map information. That is, by transmitting the attention map information, the destination requests transmission of a semantic packet including additional information for semantic error correction.

[0192] FIG. 29 illustrates an example of an initial setup procedure for semantic communication according to one embodiment of the present disclosure. FIG. 29 illustrates signaling for setup related to semantic communication between a first device (2910) and a second device (2920). Here, the first device (2910) may be one of a source and a destination, and the second device (2920) may be the other of the source and the destination. Furthermore, the first device (2910) may be a terminal and the second device (2920) may be a base station, or the first device (2910) may be a base station and the second device (2920) may be a terminal, or both the first device (2910) and the second device (2920) may be terminals.

[0193] Referring to FIG. 29, in step S2901, the first device (2910) and the second device (2920) perform synchronization and connection establishment procedures. To this end, the first device (2910) and the second device (2920) may perform synchronization using at least one synchronization signal and perform signaling for connection establishment.

[0194] In step S2903, the second device (2920) transmits a capability information request message to the first device (2910). For example, the second device (2920) transmits a downlink-dedicated control channel (DL-DCCH) message including a UE capability inquiry indicator. In particular, according to one embodiment, the capability information request message includes information inquiring whether semantic communication can be performed.

[0195] In step S2905, the first device (2910) transmits a capability information message to the second device (2920). For example, the first device (2910) transmits an uplink-dedicated control channel (UL-DCCH) message including UE capability information. In particular, according to one embodiment, the capability information message includes capability information related to semantic communication. According to various embodiments, the capability information may include information on whether the first device (2910) is equipped with semantic communication performance capability. In addition, the capability information may further include information on the types of raw data that can be generated, collected, or processed by the first device (2910), the computational capabilities of the first device (2910), etc.

[0196] In step S2907, the second device (2920) may determine whether to perform semantic communication. For example, the second device (2920) may recognize that the first device (2910) has the capability to perform semantic communication based on capability information received from the first device (2910). In addition, the second device (2920) may further consider other acquired information to determine whether to perform semantic communication. Specifically, the second device (2920) may determine whether the semantic communication-related capability of the first device (2910) satisfies the requirements for semantic communication. In the example of FIG. 29, it is determined whether to perform semantic communication.

[0197] In step S2909, the second device (2920) transmits information related to semantic communication to the first device (2910). Here, the configuration information may include information instructing the performance of semantic communication and configuration information for performing semantic communication. The information related to semantic communication may be transmitted via at least one of downlink control information (DCI), medium access control (MAC) control element (CE), or radio resource control (RRC) message. Specifically, the information instructing the performance of semantic communication may be transmitted via upper layer signaling, and at least a portion of the configuration information may be transmitted via DCI or MAC CE. In step S2911, the first device (2910) stores the information related to semantic communication. According to one embodiment, the configuration information may indicate parameters, variables, etc. related to error detection and correction of semantic communication. Specifically, the configuration information may include at least one of information related to whether multiple encoders are used, information related to a semantic encoder / decoder model, information related to a set of metrics related to the operation of an attention matrix, information related to a threshold related to a bit-wise attention map, information related to a function for generating a union attention map, information related to a threshold related to error detection, information related to a maximum semantic data retransmission count, and information related to an early termination threshold.

[0198] Thereafter, although not illustrated in FIG. 29, the first device (2910) and the second device (2920) may perform semantic communication according to various embodiments of the present disclosure. At this time, the first device (2910) and the second device (2920) may perform semantic error detection and semantic error correction based on the information transmitted in step S2911 and stored in step S2911.

[0199] The present disclosure below describes exemplary procedures according to one embodiment using more specific examples. In the exemplary procedures below, an environment as shown in [Table 2] is assumed.

[0200] #Contents 1 The source adopts the structure of Fig. 20, and the destination adopts the structure of Fig. 22. That is, both the source and the destination use an integrated encoder. 2 Three tasks are performed at the destination, and the metrics for obtaining the attention map are set differently for each task. Specifically, the discard ratios are all set to 0.9, and the fusion metrics are set differently to maximum (max), average (mean), and minimum (min). In addition, [Mathematical Formula 3] is used, that is, =1. And, the threshold for determining each attention matrix is set to 0.4. 3 At least one semantic packet transmitted from a source to a destination includes a union attention map. 4 The error rate threshold for determining whether a semantic error has occurred is set to 20%. If a semantic error has occurred below the threshold, the task at the destination can be determined to have operated as intended by the source using data restored to the semantic level. 5 Semantic error detection of residual area-based data used for semantic error correction may not be performed.

[0201] In an environment such as [Table 2], an example of an exemplary procedure in which semantic error detection and correction are performed is as follows.

[0202] Step-1) The source does not receive a bit-wise attention map from the destination, and the counter for current semantic error correction is set to the maximum number of times to prevent indiscriminate retransmission for semantic error correction. If the original data, which is the input data, exists, an operation is performed to transmit information including the intention that the source wants to convey by using it. Through the encoder operation, the semantic expression and the function of [Mathematical Formula 2] are performed. A set of multiple attention matrices containing three attention matrices with different metrics set using is extracted. Afterwards, the source is a function of [Mathematical Formula 3] By using A set of multiple attention maps by converting the attention matrices belonging to into a bit-wise attention map format. and generates the function of [Mathematical Formula 4] By using By combining the attention maps belonging to the union bit-wise attention map obtain. For example, silver may contain the union of .

[0203] FIG. 30A illustrates examples of bit-wise attention maps and union attention maps determined based on original data according to one embodiment of the present disclosure. FIG. 30A illustrates a result of representing multiple bit-wise attention maps and union bit-wise attention maps extracted from input data according to set contents in the form of a heatmap on the input data. In the example of FIG. 30A, three metrics are set, which means that each task focuses on different semantic areas of data used for service operation. Referring to FIG. 30A, three attention maps (3012, 3013, 3014) using different metrics can be generated from original data (3011), and a union attention map (3015) can be generated based on the three attention maps (3012, 3013, 3014). That is, three bit-wise attention maps (3012, 3013, 3014) can be generated using different metrics from the same encoder.

[0204] As another example, when a single metric is set for a single encoder, a semantic region related to the source's intended operation for the input data is set through a single attention map. In this case, the data corresponding to the semantic region required to indicate the source's intent through the single attention map may not include the data corresponding to the semantic region required to perform the service operation provided by the task. This is confirmed by the fact that the regions indicated by the attention maps are different even though only the fusion metric for generating the attention maps in Fig. 30a has been changed. Therefore, when the source and destination are configured with a single encoder, multiple attention maps can be generated through metrics set for each source and destination to indicate the semantic region used by the task for the service operation, a final union attention map can be determined, and semantic error detection and correction can be performed on the data corresponding to the semantic region that matches the intent considering the operations of all tasks in the input data.

[0205] If the structure of Fig. 19 is adopted instead of the structure of Fig. 20, and an attention map for each bit of the input data is generated for each encoder using an encoder related to the task providing the target service, the same result as generating an attention map for each bit by adopting the structure of Fig. 20 and applying a task-specific metric to a single encoder can be obtained. At this time, if the structure of Fig. 19 is adopted, since a semantic expression is generated from each encoder, the final semantic expression can be generated through a semantic expression generator that combines multiple semantic expressions into a single semantic expression.

[0206] Step-2) The source generates at least one semantic packet including a union attention map (e.g., a union attention map (3015) of FIG. 30a) obtained from the source and at least one semantic expression, and transmits the generated semantic packet to the destination. Here, the at least one semantic packet may include a set of attention maps (e.g., attention maps (3012, 3013, 3014) of FIG. 30a) instead of the union attention map according to an initial setting. In other words, the at least one semantic packet includes attention map information as information related to attention, and the attention map information may include at least one of a union attention map or an individual attention map set. The source stores the transmitted attention map information and uses the stored attention map information for subsequent semantic error correction operations.

[0207] Step 3) The destination receives at least one semantic packet, and obtains a packet with a union attention map and at least one semantic representation from the at least one semantic packet. The destination uses the received at least one semantic representation as input to a decoder to perform semantic level reconstruction.

[0208] Step-4) The destination uses the data restored to the semantic level (e.g., the input data (3021) of Fig. 30b) as the input data of the encoder, and uses the union attention map according to [Mathematical Formula 2], [Mathematical Formula 3], and [Mathematical Formula 4]. (e.g., union attention map (3025) of Fig. 30b) is generated. For example, the destination can generate a final union attention map (e.g., union attention map (3025) of Fig. 30b) by generating a multi-attention map including multiple attention maps (e.g., attention maps (3022, 3023, 3024) of Fig. 30b) using the same task-specific metrics used in the source.

[0209] Step-5) The destination receives the union attention map received from the source (e.g., the union attention map (3015) of Fig. 30a) and the union attention map generated at the destination. (e.g., the union attention map (3025) of Fig. 30b) is used to measure the semantic error rate based on [Mathematical Formula 1]. If a multi-attention map including a set of attention maps other than the union attention map (e.g., the attention maps (3012, 3013, 3014) of Fig. 30a) is received from the source, the destination uses the multi-attention map of the source to generate the union attention map. and create, and can be compared. In the example of Fig. 30b, the error rate of the measured semantic level is 37%, and 37% is the threshold. is higher than the 20% set. Therefore, the destination can determine the occurrence of a semantic error.

[0210] Step 6) The destination performs an early stop checking operation to determine whether early stop can be performed. If the semantic error rate is lower than the threshold compared to the previously measured semantic error rate, If the decrease is less than the threshold, the destination determines that the semantic error rate cannot be improved by retransmission due to semantic errors and reports a system failure. The destination can then request a new configuration or transmission of new data. On the other hand, if the semantic error rate is lower than the threshold compared to the previously measured semantic error rate, the destination can determine that the semantic error rate is lower than the threshold. If the threshold value is reduced to this value, the destination does not perform an early stop operation and proceeds to the next step. In this example, the semantic error is reduced by approximately 63% compared to the initial state of 100%. In this exemplary procedure, the threshold value is When this is set to 5%, the semantic error rate is reduced by more than 5%, so the next step can proceed without premature termination.

[0211] Step-7) The destination stores the data restored to the semantic level and the union attention map obtained based on the data, and transmits information necessary for semantic error correction to the source. For example, the destination can directly transmit the union attention map extracted from the data restored to the semantic level at the semantic level (e.g., the union attention map (3032) of FIG. 30c). As another example, the destination can perform an operation to extract a portion corresponding to the remaining area excluding the portion occupied by the union attention map generated by the destination from the union attention map received from the source (e.g., the residual area mask (3033) of FIG. 30c), i.e., information indicating the target area to which semantic error correction is applied.

[0212] Step-8) The source has a threshold number of retransmissions for semantic error correction. Check whether the current number of retransmissions is If this is the case, the source determines a system failure, sets the current counter to 0, and flushes the stored attention map information to prevent retransmissions for indiscriminate semantic error correction. If a system failure is determined, a new configuration or transmission of new data is requested. On the other hand, the retransmission count counter If it is less than that, the source determines that retransmission is possible for semantic error correction, increments the counter by one, and performs the next step.

[0213] Step 9) The source extracts the residual area data required to perform semantic error correction at the destination based on the type of attention map received from the destination (e.g., the attention map information transmitted in Step 7). At this time, the source extracts a set of multiple attention maps. If you were sending it to the destination and storing that information, By using The action of acquiring can be performed first.

[0214] Next, the source performs two actions based on the received attention map information: A union attention map is created based on the data restored from the semantic level to the semantic level based on the attention map information received from the destination. (e.g., union attention map (3032) in Fig. 30c), as in Fig. 30c, the source is a union attention map based on the original data. (e.g., union attention map (3031) of Fig. 30c) compared to the received union attention map A mask indicating the remaining portion excluding the included portion (e.g., the residual region mask (3033) of FIG. 30c) is generated, and residual region-based data (e.g., the residual region-based data (3034) of FIG. 30c) is generated by applying the generated mask to the original data. On the other hand, if the attention map information received from the destination is a residual region mask (e.g., the residual region mask (3033) of FIG. 30c), the source generates residual region-based data (e.g., the residual region-based data (3034) of FIG. 30c) by applying the residual region mask to the original data.

[0215] Step-10) If the received attention map exists, the source uses residual region-based data (e.g., residual region-based data (3034) of FIG. 30c) as input to the encoder to generate a semantic representation, and uses the set task-related metric information to generate a multi-attention map and a union attention map according to [Mathematical Formula 2], [Mathematical Formula 3], and [Mathematical Formula 4]. This can be understood as a similar operation to performing the process of Step-1) using residual region-based data.

[0216] An example of generating a union attention map for residual region-based data extracted from original data for semantic error correction is shown in FIG. 30d. Referring to FIG. 30d, attention maps (3042, 3043, 3044) based on three different metrics for residual region-based data (3041) are generated, and a union attention map (3045) corresponding to residual region-based data (3041) can be generated based on the attention maps (3042, 3043, 3044).

[0217] Step-11) The source generates and transmits at least one semantic packet including semantic representations for residual region-based data and a union attention map or a multi-attention map. If, during initial setup, semantic error detection is performed on data restored to a semantic level corresponding to the residual region at the destination, a semantic error correction operation is performed when a semantic error occurs for the data, so the source stores a multi-attention map or a union attention map corresponding to the residual region-based data according to the type of attention map transmitted to the destination. Otherwise, the source may not store a multi-attention map or a union attention map for the residual region-based data.

[0218] Step-12) Similar to Step-3 described above, the destination receives at least one semantic packet, obtains a union attention map and semantic representations from the received at least one packet, and performs semantic level restoration using the semantic representations as input to the decoder to obtain residual region-based data. Then, depending on whether semantic error detection is performed on the residual region-based data, the operation of Step-13 or Step-14 is performed.

[0219] Step-13) When performing semantic error checking on residual region-based data, the residual region-based data restored to the semantic level is used as input data of the encoder, similar to Step-4, and a union attention map is generated according to [Mathematical Equation 2], [Mathematical Equation 3], and [Mathematical Equation 4]. At this time, the destination generates multiple attention maps using the same task-specific metrics used in the source, and the final union attention map, i.e., a union attention map based on residual region data restored to the semantic level. Obtain .

[0220] An example of a process for extracting multi-bit-wise attention maps and union bit-wise attention maps from residual region-based data restored to a semantic level is shown in FIG. 30e. Referring to FIG. 30e, attention maps (3052, 3053, 3054) based on three different metrics for residual region-based data (3051) restored to a semantic level are generated, and a union attention map (3055) corresponding to the residual region-based data (3051) can be generated based on the attention maps (3052, 3053, 3054).

[0221] Union attention map for residual region-based data (e.g., if a union attention map (3045) in Fig. 30d is passed, the destination transmits the union attention map for the residual region-based data generated at the destination. (e.g., union attention map (3055) in Fig. 30e) and The error rate can be calculated based on the comparison between the attention maps. In contrast, if the source transmits multiple attention maps including a set of attention maps (e.g., attention maps (3042, 3043, 3044) in FIG. 30e), the destination can generate a union attention map for the multiple attention maps of the source according to Equations 3 and 4. and create, and The semantic error rate can be calculated based on the comparison between the two. The comparison results are based on the threshold. If a lower semantic error rate is confirmed, the destination determines that semantic error correction can be performed using residual region-based data corresponding to the overlapping portion of the two union attention maps, and synthesizes residual region-based data corresponding to the overlapping portion between the two union attention maps after semantic error checking to the existing semantic error correction target data (e.g., input data (3021) in FIG. 30b or noise data (3061) in FIG. 30f), and checks whether semantic error correction is successful. If semantic error correction fails, the destination can perform additional operations to perform semantic error correction on the residual region-based data.

[0222] Step-14) If semantic error checking is not performed on the residual region-based data, the destination synthesizes the residual region-based data restored to the semantic level into data restored to the semantic level that has not undergone semantic error checking in the previous process that is the target of semantic error correction. Fig. 30f shows an example of data synthesis. Referring to Fig. 30f, synthesized data (3063) is generated by synthesizing noise data (3061) and residual region data (3062). If semantic error checking is not performed on the residual region-based data restored to the semantic level, this step may be performed after Step-13.

[0223] Step-15) The destination is a union attention map from the composited data (e.g., the composited data (3063) in Fig. 30f). and create, and destinations have The semantic error rate is measured based on the comparison between the attention maps (3072, 3073, 3074) generated from the synthesized data (3071), and the semantic error rate is determined based on the comparison between the union attention map (3075) determined based on the attention maps (3072, 3073, 3074) and the union attention map (3015) corresponding to the original data. In this exemplary procedure, the error rate is about 8.3%, and 8.3% is the threshold. Since the synthesized data is lower than the input of the task, the intended downstream task operation can be performed at the source. In contrast, and The error rate measured through comparison is In higher cases, the destination can extract residual regions for additional error correction through comparison with the two union attention maps, and perform semantic error correction on the extracted residual regions.

[0224] As described above, the present disclosure relates to a technology for performing semantic error detection and semantic error correction at a destination based on a multiple attention map generated and transmitted from a source in a system supporting semantic communication and for transferring information between a source and a destination, and for considering the operation of the source and the destination. When a data area that a source wants to focus on in relation to the meaning that it wants to convey in input data is set as a semantic element, the semantic area required for the operation of tasks related to service provision located at the destination is considered, and each task extracts the semantic area required for service operation from the input data using a multiple attention map set, and by performing semantic error detection and correction on a single data restored at the semantic level using a union attention map that combines the multiple attention maps, the amount of semantic information for the data used as input of the task is increased, thereby utilizing it in the task operation, thereby improving the performance of the task service operation. In addition, by performing operations based on the union attention map, the number of times semantic error detection and correction is performed is reduced by performing semantic error detection and correction on data restored to a single semantic level compared to performing semantic error detection and correction on the semantic region from the data utilized by each task, and by reducing the amount of information exchanged between the source and destination, additional end-to-end latency will be reduced.

[0225] It is clear that the examples of the proposed methods described above can also be considered as a type of proposed methods, as they can be included as one of the implementation methods of the present disclosure. Furthermore, the proposed methods described above can be implemented independently, but they can also be implemented in the form of a combination (or merge) of some of the proposed methods. Information regarding the applicability of the proposed methods (or information regarding the rules of the proposed methods) can be defined by a rule such that the base station notifies the terminal of the application of the proposed methods through a predefined signal (e.g., a physical layer signal or a higher layer signal).

[0226] The present disclosure may be embodied in other specific forms without departing from the technical ideas and essential features described herein. Therefore, the above detailed description should not be construed as limiting in all respects but rather as illustrative. The scope of the present disclosure should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present disclosure are intended to be included within the scope of the present disclosure. Furthermore, claims that do not explicitly cite each other in the claims may be combined to form embodiments or incorporated into new claims through post-filing amendments.

[0227] Embodiments of the present disclosure can be applied to various wireless access systems. As an example of various wireless access systems, 3GPP (3 rd Generation Partnership Project) or 3GPP2 system.

[0228] The embodiments of the present disclosure can be applied not only to the various wireless access systems described above, but also to all technical fields that utilize these various wireless access systems. Furthermore, the proposed method can also be applied to mmWave and THz communication systems utilizing ultra-high frequency bands.

[0229] Additionally, embodiments of the present disclosure can be applied to various applications such as autonomous vehicles and drones.

Claims

1. A method performed by a first device in a wireless communication system, Step of establishing a connection with a second device; A step of receiving a first message requesting capability information from the second device; A step of transmitting a second message including the above capability information to the second device; A step of receiving setting information for communication from the second device; and A step of performing semantic communication that supports multiple tasks based on the above setting information is included. A method in which the above setting information includes information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

2. In claim 1, A method wherein the above configuration information includes at least one of information related to whether a multiple encoder is used, information related to a semantic encoder / decoder model, information related to a set of metrics related to the operation of an attention matrix, information related to a threshold related to a bitwise attention map, information related to a function for generating a union attention map, information related to a threshold related to error detection, information related to a maximum semantic data retransmission count, and information related to an early stopping threshold.

3. In claim 1, A method in which the above union attention map includes a union of the plurality of attention maps.

4. In claim 1, A method in which the above multiple tasks are performed based on the same input data.

5. In claim 1, The steps for performing the above semantic communication are: A step of transmitting a first packet including first semantic information for the plurality of tasks using the same input data; A step of receiving feedback information for semantic error correction, which is related to a residual area which is part of the above input data; and A method comprising the step of transmitting a second packet including second semantic information related to the remaining area.

6. In claim 5, The step of transmitting the first packet is: generating a first semantic representation for the plurality of tasks from input data; and A step of transmitting a packet including first attention information corresponding to the first semantic expression and the first semantic expression to the second device, A method wherein the first attention information comprises at least one of a first plurality of attention maps or a first union attention map corresponding to the first semantic expression.

7. In claim 5, The step of transmitting the second packet is: generating a second semantic representation for the plurality of tasks from a portion of the input data indicated by the residual region; and A step of transmitting a packet including second attention information corresponding to the second semantic expression and the second semantic expression to the second device, A method wherein the second attention information includes at least one of a second plurality of attention maps or a second union attention map corresponding to the second semantic expression.

8. In a method performed by a second device in a wireless communication system, Step of establishing a connection with the first device; A step of transmitting a first message requesting capability information to the first device; A step of receiving a second message including the above capability information from the first device; A step of transmitting setting information for communication to the first device; and A step of performing semantic communication that supports multiple tasks based on the above setting information is included. A method in which the above setting information includes information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

9. In claim 8, The steps for performing the above semantic communication are: A step of receiving a first packet including first semantic information for the plurality of tasks; A step of obtaining data restored to a first semantic level by performing semantic decoding on a first semantic expression included in the first packet; A step of obtaining second attention information by performing semantic encoding on data restored to the first semantic level; A step of transmitting feedback information for semantic error correction related to a residual area, which is part of the input data, based on the second attention information; A step of receiving a second packet including second semantic information related to the remaining area; and A method comprising a step of performing semantic error correction based on the second semantic information included in the second packet.

10. In claim 9, The step of transmitting the above feedback information is: If the difference between the first attention information and the second attention information included in the first packet is greater than a threshold, a step of checking the remaining area corresponding to the difference; and A method comprising the step of transmitting the feedback information indicating at least one of a mask indicating the residual area or the second attention information.

11. In claim 10, A method in which the residual area includes the remainder of the first union attention map generated in the first device, excluding the overlapping area of ​​the first union attention map generated in the first device and the second union attention map generated in the second device.

12. In claim 10, The steps for performing the above semantic error correction are: A step of obtaining data restored to a second semantic level for the residual region by performing semantic decoding on a second semantic expression included in the second packet; A method comprising a step of synthesizing data restored to the first semantic level and data restored to the second semantic level.

13. In claim 12, The step of synthesizing data restored to the first semantic level and data restored to the second semantic level is; A step of determining a semantic error rate for data restored to the second semantic level; If the semantic error rate is greater than a threshold, a step of performing semantic error correction for the remaining region; and A method comprising the step of synthesizing data restored to the first semantic level and data restored to the second semantic level if the semantic error rate is less than a threshold.

14. In claim 12, The steps for performing the above semantic error correction are: A method comprising a step of determining a semantic error rate for synthesized data obtained by synthesizing data restored to the first semantic level and data restored to the second semantic level.

15. In a first device in a wireless communication system, Transmitter and receiver; and A processor connected to the above transmitter and receiver is included, The above processor, Establish a connection with the second device, Receive a first message requesting capability information from the second device, Transmitting a second message including the above capability information to the second device, Receive setup information for communication from the second device, Controls semantic communication to support multiple tasks based on the above setting information, A first device, wherein the above setting information includes information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

16. In a second device in a wireless communication system, Transmitter and receiver; and A processor connected to the above transmitter and receiver is included, The above processor, Establish a connection with the first device, Transmitting a first message requesting capability information to the first device, Receive a second message including the above capability information from the first device, Transmitting setting information for communication to the first device, Controls semantic communication to support multiple tasks based on the above setting information, A second device, wherein the above setting information includes information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

17. In communication devices, At least one processor; At least one computer memory connected to said at least one processor and storing instructions that direct operations when executed by said at least one processor, The above actions are, A step of establishing a connection with another communication device; A step of receiving a first message requesting capability information from said other communication device; A step of transmitting a second message including the above capability information to the other communication device; A step of receiving setting information for communication from the other communication device; and A step of performing semantic communication that supports multiple tasks based on the above setting information is included. A method in which the above setting information includes information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks.

18. In a non-transitory computer-readable medium storing at least one instruction, comprising at least one instruction executable by the processor, At least one of the above commands causes the device to: Establish a connection with another device, Receive a first message requesting capability information from said other device, Transmitting a second message including the above capability information to the other device, Receive setup information for communication from the other device, Controls semantic communication to support multiple tasks based on the above setting information, A non-transitory computer-readable medium comprising information for performing semantic error detection and correction based on a union attention map generated by combining a plurality of attention maps corresponding to the plurality of tasks, wherein the above setting information is a combination of a plurality of attention maps corresponding to the plurality of tasks.

Images