Semantic Communication

Abstract

The present disclosure relates to semantic communication. An apparatus is configured to: receive data input corresponding to a plurality of data objects to be sent; process the data input to generate a semantic representation of each of the data objects, wherein the semantic representation includes a semantic distance of each of the data objects, wherein the semantic distance relates each of the data objects to each other; and prepare the semantic representation of the data objects for sending.

Description

[0001] Priority / Incorporated by Reference

[0002] This patent application claims priority to U.S. Provisional Application Serial No. 63 / 586,528, entitled “Semantic Communication,” filed on September 29, 2023, the entire contents of which are incorporated herein by reference. Background Art

[0003] The use of wireless communication systems is growing rapidly and is constantly evolving. Users expect a high-quality experience on wireless networks, including when executing real-time applications with large traffic, such as virtual reality (VR) / augmented reality (AR) applications including holographic conferencing, ultra-high-resolution video on demand, video conferencing under poor channel conditions, cloud gaming, etc. Traditional communication modes that use error-free deterministic communication have heavy computational requirements and high processing delays. These traditional communication modes may not meet the needs of future users. Summary of the invention

[0004] Some example embodiments relate to a device having processing circuitry configured to: receive data input corresponding to a plurality of data objects to be sent; process the data input to generate a semantic representation of each of the data objects, wherein the semantic representation includes a semantic distance for each of the data objects, wherein the semantic distance relates each of the data objects to one another; and prepare the semantic representation of the data objects for sending.

[0005] Other example embodiments relate to a device having processing circuitry configured to jointly train a variational autoencoder (VAE) and a VAE decoder based on a training dataset including data related to semantic distances, add a scaling layer to the VAE to generate a semantic processing model, eliminate a noise dimension from the VAE, and train a semantic processing reconstruction model based on the training dataset, the semantic processing model, and data generated by eliminating the noise dimension.

[0006] Another example embodiment relates to an apparatus having a processing circuit configured to receive a data input corresponding to a data object to be sent, process the data input to generate a semantic representation of the data object, generate an Internet Protocol (IP) packet including the semantic representation and a header indicating a data type of the semantic representation, and generate a transport block (TB) including a soft delivery portion and a precise delivery portion, wherein the soft delivery portion includes the semantic representation of the data object. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 Example network arrangements are shown according to various example embodiments.

[0008] Figure 2 An example user equipment (UE) is shown according to various example embodiments.

[0009] Figure 3 An example base station is shown according to various example embodiments.

[0010] Figure 4 An example system architecture for semantic communication is shown according to various example embodiments.

[0011] Figure 5 An example flow of a semantic processing neural network (NN) or a semantic recovery NN using an application specific integrated circuit (ASIC) according to various example embodiments is shown.

[0012] Figure 6 An overview of an example training process for a semantic processing NN and a semantic recovery NN according to various example embodiments is shown.

[0013] Figure 7 An example parameter model for β-Variational Autoencoder (β-VAE) training is shown according to various example embodiments.

[0014] Figure 8 Examples of data vectors of semantic representation at the medium access control (MAC) layer of a transmitter are shown according to various example embodiments.

[0015] Fig. 9 Example Internet Protocol (IP) / Transport Level Protocol packets for soft communications are shown according to various example embodiments.

[0016] Fig.10 Examples of radio link control (PLC) protocol data units (PDUs) including MAC PDUs for soft communication are shown according to various example embodiments.

[0017] Fig.11 An example soft delivery portion of a transport block (TB) for soft communication is shown according to various example embodiments. DETAILED DESCRIPTION

[0018] Example embodiments may be further understood with reference to the following description and associated drawings, wherein similar elements have the same reference numerals. Example embodiments relate to a semantic communication system including functional extensions of the application layer, layer 2 and layer 3, and the physical (PHY) layer for both transmitters and receivers.

[0019] The example embodiments are described with reference to user equipment (UE). However, reference to UE is provided for illustrative purposes only. The example embodiments may be used with any electronic component that can establish a connection with a network and is configured with hardware, software and / or firmware for exchanging information and data with the network. Therefore, UE as described herein is used to represent any suitable type of electronic component.

[0020] The example embodiments are also described with reference to sixth generation (6G) networks. However, reference to 6G networks is provided for illustrative purposes only. The example embodiments may be used with any suitable type of network that supports semantic communication as described herein, including future evolutions of cellular standards beyond 6G.

[0021] Example embodiments relate to semantic communication. Semantic communication is different from traditional or classical communication. Specifically, traditional communication modes use error-free deterministic communication, such as data to be sent by a sender and received by a receiver is the actual data to be exchanged, such as data representing pixel values ​​of an image. In contrast, semantic communication is not limited to the sending of actual data. Instead, semantic communication sends semantic representations of data, such as semantic data or metadata about the data to be sent. The receiver can use the semantic data or metadata to reconstruct the actual data intended by the sender without the sender sending the actual data.

[0022] One advantage of semantic communication is that the semantic representation of data (e.g., semantic data or metadata about the data to be sent) can be smaller than the actual data to be sent. To provide an example, the semantic representation of an image can be significantly smaller than the actual data of the image. This reduction in the amount of data required to exchange information between a sender and a receiver can allow for higher throughput to accommodate scenarios with larger traffic volumes.

[0023] Semantic communication is not data compression. That is, conventional communication modes may use various compression algorithms to compress the size of the actual data, for example, there are various forms of MPEG compression that reduce the size of video files used for conventional communication. These forms of compression typically remove some of the actual data from the data being sent, and the receiver may use interpolation or other methods to decode the compressed data. Semantic communication differs from compression (or encoding) in that the reduced amount of data used for semantic communication is not a subset of the actual data as in conventional communication compression.

[0024] A simple example of sending an image that includes a tree is provided. Traditional communication modes require representation of each pixel of the image that includes the tree, and those representations of each pixel are sent to the receiver. The receiver can decode the representation of each pixel and display the image. In contrast, in semantic communication, the semantic representation can be as simple as indicating "tree". The receiver can then place the tree in the image. The semantic representation can be more complex, such as "tree with green leaves", "tree with autumn leaves", "oak tree", etc. As can be seen from this simple example, the amount of data used to transmit the same information using semantic communication is significantly less.

[0025] However, since the semantic representation is not the actual data, there may be errors introduced by the semantic communication. Example embodiments use various techniques to ensure that the semantic representation of data conveys the meaning of the data from the sender to the receiver.

[0026] Throughout the specification, there are different terms used to describe semantic data. The term semantic representation is as used above and describes data or metadata that represents actual data, e.g., information that provides the meaning of the actual data. In some cases, the semantic representation is represented as a data vector, e.g., the input data is a data vector of actual data and is processed to obtain the semantic representation.

[0027] Data may also be described as belonging to a specific data category. Data categories may be thought of as kinds of data that are processed in a specific way by an application. Examples are video frames, audio frames, text, 3-D video, sensor cubes, etc. As will be described in more detail below, the application layer of the sender and receiver may have a specific model for processing each data category.

[0028] There may also be different data types, which may be considered as numerical data commonly used for digital representation in digital systems. For example, an unsigned 8-bit integer is a typical example of a data type. Thus, a data category may be represented by one or more data types.

[0029] The example embodiments describe application layer functional extensions for semantic communication. These functional extensions may include, but are not limited to, semantic distance definition, structure and implementation of semantic processing, and semantic reconstruction neural network models, including training procedures for isometric semantic processing and semantic reconstruction models, motion compensation in the context of semantic communication, and data statistics collection.

[0030] The example embodiments also describe medium access control (MAC) layer extensions for semantic communication. These functional extensions may include, but are not limited to, rate matching and modulation selection, downlink control information (DCI) extensions containing modulation scheme index, data vector index, and data end marker, soft control data structures containing soft integrity checks, new MAC scheduler functions, and adjustments to PHY parameters.

[0031] The example embodiments also describe PHY layer extensions for semantic communication. These functional extensions may include, but are not limited to, potential feature multiplexing of the PHY layer and superposition of soft-coded PHY transmissions.

[0032] Figure 1 An example network arrangement 100 according to various example embodiments is shown. The example network arrangement 100 includes a UE 110. The UE 110 may be any type of electronic component configured to communicate via a network, such as a mobile phone, a tablet computer, a desktop computer, a smart phone, a phablet, an embedded device, a wearable device, an Internet of Things (IoT) device, etc. A practical network arrangement may include any number of UEs used by any number of users. Therefore, for illustrative purposes, only an example with a single UE 110 is provided.

[0033] UE 110 may be configured to communicate with one or more networks. In the example of network configuration 100, the network with which UE 110 may wirelessly communicate is a 6G radio access network (RAN) 120. However, UE 110 may also communicate with other types of networks (e.g., fifth generation (5G) RAN, 5G cloud RAN, next generation RAN (NG-RAN), long term evolution (LTE) RAN, traditional cellular networks, wireless local area networks (WLAN), etc.), and UE 110 may also communicate with the network via a wired connection. Referring to the example implementation, UE 110 may establish a connection with 6G RAN 120. Therefore, UE 110 may have at least a 6G chipset to communicate with 6G RAN 120.

[0034] The 6G RAN 120 may be part of a cellular network that may be deployed by a network operator, such as Verizon, AT&T, T-Mobile, etc. The 6G RAN 120 may include base stations or access nodes (Node Bs, eNodeBs, HeNBs, eNBSs, gNBs, gNodeBs, macrocell base stations, microcell base stations, small cell base stations, femtocell base stations, etc.) configured to transmit and receive traffic from UEs equipped with appropriate cellular chipsets.

[0035] Any association process may be performed to connect the UE 110 to the 6G RAN 120. For example, as described above, the 6G RAN 120 may be associated with a particular cellular provider where the UE 110 and / or its user has protocol and credential information (e.g., stored on a SIM card). Upon detecting the presence of the 6G RAN 120, the UE 110 may send corresponding credential information to associate with the 6G RAN 120. More specifically, the UE 110 may be associated with a particular base station (e.g., gNB 120A).

[0036] The network arrangement 100 also includes a cellular core network 130, the Internet 140, an IP multimedia subsystem (IMS) 150, and a network service backbone 160. The cellular core network 130 may refer to an interconnected set of components that manage the operation and traffic of a cellular network. It may include an evolved packet core (EPC), a 5G core (5GC), and a 6G core (6GC). The cellular core network 130 also manages traffic flowing between the cellular network and the Internet 140. The IMS 150 may generally be described as an architecture for delivering multimedia services to the UE 110 using the IP protocol. The IMS 150 may communicate with the cellular core network 130 and the Internet 140 to provide multimedia services to the UE 110. The network service backbone 160 communicates directly or indirectly with the Internet 140 and the cellular core network 130. The network service backbone 160 may generally be described as a set of components (e.g., servers, network storage arrangements, etc.) that implement a set of services that can be used to extend the functionality of the UE 110 to communicate with various networks.

[0037] Figure 2 An exemplary UE 110 is shown according to various exemplary embodiments. Figure 1 100 is used to describe the UE 110 with respect to the network arrangement 100 of FIG. UE 110 may include a processor 205, a memory arrangement 210, a display device 215, an input / output (I / O) device 220, a transceiver 225, and other components 230. Other components 230 may include, for example, an audio input device, an audio output device, a power source, a data acquisition device, a port for electrically connecting the UE 110 to other electronic devices, and the like.

[0038] Processor 205 may be configured to execute multiple engines of UE 110. For example, the engines may include semantic communication engine 235. Semantic communication engine 235 may perform various operations, such as, but not limited to, semantically encoding data, rate matching semantic data for transmission, applying PHY layer parameters for semantic transmission, and decoding semantic data to reconstruct original input data. Each of these example operations will be described in more detail below.

[0039] The engine 235 described above is provided as an application (e.g., program) executed by the processor 205 for illustrative purposes only. The functions associated with the engine 235 may also be represented as independent combined components of the UE 110, or may be modular components coupled to the UE 110, for example, an integrated circuit with or without firmware. For example, the integrated circuit may include an input circuit for receiving a signal and a processing circuit for processing signals and other information. The engine may also be embodied as an application or multiple independent applications. In addition, in some UEs, the functionality described for the processor 205 is split between two or more processors such as a baseband processor and an application processor. Specifically, in some examples, when the UE 110 is operated in a low battery mode, the capabilities of the UE 110 that are usually processed by the baseband processor may be reduced. The exemplary embodiments may be implemented in any of these or other configurations of the UE.

[0040] The memory arrangement 210 may be a hardware component configured to store data related to operations performed by the UE 110. The display device 215 may be a hardware component configured to display data to a user, and the I / O device 220 may be a hardware component that enables user input. The display device 215 and the I / O device 220 may be separate components or may be integrated together (such as a touch screen).

[0041] The transceiver 225 may be a hardware component configured to establish a connection with the 6G-RAN 120, LTE-RAN (not shown in the figure), a traditional RAN (not shown in the figure), a WLAN (not shown in the figure), etc. Therefore, the transceiver 225 can operate on a variety of different frequencies or channels (e.g., a set of continuous frequencies). The transceiver 225 includes a circuit configured to send and / or receive signals (e.g., control signals, data signals). Such signals may be encoded with information for implementing any of the methods described herein. The processor 205 is operably coupled to the transceiver 225 and is configured to receive signals from the transceiver 225 and / or send signals to the transceiver. The processor 205 may be configured to encode and / or decode signals (e.g., signaling from a base station of the network) for implementing any of the methods described herein.

[0042] Figure 3 An example base station 300 is shown in accordance with various example embodiments. Base station 300 may represent any base station included within a network, such as base station 120A.

[0043] The base station 300 may include a processor 305, a memory arrangement 310, an input / output (I / O) device 315, a transceiver 320, and other components 325. These other components 325 may include, for example, an audio input device, an audio output device, a battery, a data acquisition device, a port for electrically connecting the base station 300 to other electronic devices and / or a power source, a TxRU, a transceiver chain, an antenna element, an antenna panel, etc.

[0044] The processor 305 may be configured to execute multiple engines of the base station 300. For example, the engines may include a semantic communication engine 330. The semantic communication engine 330 may perform various operations related to semantic communication for the base station 300.

[0045] The above-mentioned engine 330 as an application (e.g., a program) executed by the processor 305 is merely an example. The functions associated with the engine 330 may also be represented as a separate combined component of the base station 300, or may be a modular component coupled to the base station 300, such as an integrated circuit with or without firmware. For example, the integrated circuit may include an input circuit for receiving a signal and a processing circuit for processing the signal and other information. In addition, in some servers, the functions described for the processor 305 are split between multiple processors (e.g., a baseband processor, an application processor, etc.). Specifically, in some examples, when the UE 110 is operating in a low battery mode, the operations for communicating with the UE 110 that are typically processed by the baseband processor may be reduced. The example embodiments may be implemented in accordance with any of these or other configurations of the server.

[0046] The memory 310 may be a hardware component configured to store data related to operations performed by the base station 300. The I / O device 315 may be a hardware component or port that enables a user to interact with the base station 300. The transceiver 320 may be a hardware component configured to exchange data with the UE 110 and any other UE in the network arrangement 100.

[0047] The transceiver 320 can operate on a variety of different frequencies or channels (e.g., a set of continuous frequencies). Therefore, the transceiver 320 may include one or more components to enable data exchange with various networks and UEs. The transceiver 320 includes a circuit configured to send and / or receive signals (e.g., control signals, data signals). Such signals can be encoded with information that implements any of the methods described herein. The processor 305 can be operably coupled to the transceiver 320 and is configured to receive signals from the transceiver 320 and / or send signals to the transceiver. The processor 305 can be configured to encode and / or decode signals (e.g., signaling from a UE) for implementing any of the methods described herein.

[0048] Figure 4An example system architecture 400 for semantic communication according to various example embodiments is shown. The system architecture 400 shows a transmitter 410 and a receiver 450. The transmitter 410 may be a device that transmits data using semantic communication, and the receiver 450 may be a device that receives data via semantic communication and reconstructs the data. Figure 4 In the example of , the transmitter 410 and the receiver 450 may be considered to be UEs (e.g., UE 110), but it is not required that the transmitter 410 and the receiver 450 are both UEs. One of the transmitter 410 and the receiver 450 may be a different type of device, such as a network server or other component that communicates with the UE, such as a streaming server that streams content to the UE.

[0049] The system architecture 400 shows multiple communication layers for each of the transmitter 410 and the receiver 450, including the application layer, layer 2 (L2) and layer 3 (L3), and the physical (PHY) layer. These are shown as the application layer 420, the L2 and L3 layers 430, and the PHY layer 440 for the transmitter 410, and the application layer 460, the L2 and L3 layers 470, and the PHY layer 480 for the receiver 450. Various processes for semantic communication can be performed in each of these communication layers at the transmitter 410 and the receiver 450. These processes will be described in more detail below. However, the transmitter 410 and the receiver 450 do not need to perform each of these processes for all semantic communications, for example, different types of data or different data categories may use different semantic communication processes.

[0050] Before describing the specific processes implemented at the various layers of the transmitter 410 and the receiver 450, a general overview of semantic communication will be described with reference to the system architecture 400. The transmitter 410 may have input data 415 to be sent to the receiver 450. The input data 415 undergoes semantic processing 425 at the application layer 420, which may involve processing the input data 415 using an artificial intelligence (AI) / machine learning (ML) model. Examples of training AI / ML models for semantic processing will be described in more detail below. The semantically processed input data can then be processed at the application layer 420 for motion differentiation 428. As described above, the motion differentiation processing 428 may only be applicable to certain data types or data categories.

[0051] The input data is then passed to the L2 and L3 layers 430 to perform L2 and L3 processes 435 on the input data. The L2 and L3 layers 430 may also perform rate matching 438 on the input data. The input data is then passed to the PHY layer 440 for whitening 445, which may also be applicable to only certain data types or data categories. Finally, the input data undergoes baseband (BB) and radio frequency (RF) processes 448 at the PHY layer 440 to be sent to the receiver 450 via a channel.

[0052] The data received at the receiver 450 is then processed to reconstruct the input data at the receiver 450. The reconstruction includes BB and RF processes 488 and de-whitening 485 at the PHY layer 480 of the receiver 450, data vector assembly 478 and L2 and L3 processes 475 at the L2 and L3 layers 470 of the receiver 450, motion integration 468 and semantic reconstruction 465 at the application layer 460 of the receiver 450 to generate output data 455. Similar to the semantic processing 425 of the transmitter 410, the semantic reconstruction 465 may include the use of an AI / ML model for reconstruction purposes. Therefore, at the end of the process, the receiver 450 should reconstruct the input data 415 into output data 455.

[0053] Turn to the specific operations at each layer of the transmitter 410 and the receiver 450. Initially, the operations at the corresponding application layers 420 and 450 will be described. When semantic processing 425 or semantic reconstruction 465 is performed on data, the semantic distance (D sem ), for example D sem :

[0054] The semantic distance definition captures the perceptual differences between data objects. For example, the data can be represented as an N-dimensional vector space The points in this N-dimensional vector space are called the "observation space" or "input space". Can be defined on any domain (e.g. or {0,1}).

[0055] In some example embodiments, semantic distance may be defined as geodesic distance. For example, the ground truth data distribution may be concentrated on the manifold (supported by a manifold). In this case, the semantic distance can be defined as the data manifold The geodesic distance at , which can then be extended to space as follows

[0056] (p) where h(·) is: is a monotone non-increasing function. Therefore,

[0057]

[0058] In other example embodiments, the semantic distance (D sem ) can be a distance defined by domain knowledge. For example, for some categories of data, domain knowledge can be used to define a distance based on perception. For example, the structured similarity of images (SSIM).

[0059] Therefore, two neural network (NN) models may be defined for each data category, namely, a semantic processing 425NN model for the transmitter 410 and a semantic recovery 465NN model for the receiver 450. The semantic processing 425NN and the semantic recovery 465NN may be pre-trained NN models shared with the transmitter 410 and the receiver 450. An example manner of training the semantic processing 425NN and the semantic recovery 465NN is described below.

[0060] Semantic processing 425NN and semantic restoration 465NN can convert the semantic distance (D sem ) is mapped to the Euclidean distance in the output space (also called the latent space):

[0061]

[0062] In some example embodiments, the transmitter and / or receiver may implement semantic processing 425NN and semantic recovery 465NN using an application specific integrated circuit (ASIC) or using acceleration with the help of a coprocessor. In these example embodiments, the same ASIC model may be used for multiple data categories with category specific weights that are pre-trained for a specific category.

[0063] Figure 5 An example flow of semantic processing 425NN or semantic recovery 465NN using an application specific integrated circuit (ASIC) according to various example embodiments is shown. Input 510 may be, for example, an N-dimensional vector. As described above, the ASIC NN model 520 may have data category specific weight data so that the same ASIC NN model 520 may be used for multiple data categories. Output 540 may be an M-dimensional vector of high precision values ​​(e.g., composed of 2 b bits to represent a fixed-point real value).

[0064] In some example embodiments, semantic processing 425NN and semantic restoration 465NN may be of the diagonal recurrent neural network type. This form of neural network may allow for low complexity implementations, since multiplication with a diagonal matrix or a recurrent matrix is ​​an O(N) or O(N logN) operation, respectively.

[0065] NN,f is true if and only if: Can be thought of as a diagonal recurrent neural network:

[0066]

[0067] in:

[0068] D i is a diagonal matrix,

[0069] C i is a circulant matrix,

[0070] b i is a vector, and

[0071] is the rectified linear unit (ReLU) or the identity function.

[0072] In other example embodiments, semantic processing 425NN and semantic recovery 465NN can be a combination of several layers of a specific predefined structure, followed by a diagonal recurrent neural network, followed by several layers of a specific predefined structure. This flexible structure allows the combination of processes such as discrete cosine transform (DCT), image slicing, nonlinear scaling, etc., which can be used for some data categories.

[0073] The equidistant property of the model implemented in the above example embodiments means that the Euclidean distortion (also called mean square error (MSE) distortion) of the encoded data is equal to the semantic distortion of the row data after decoding, up to a certain global coefficient. If this is the case, then minimization of the Euclidean (MSE) distortion of the communication system directly leads to minimization of the semantic distortion of the decoded data at the receiver. Therefore, the training and use of an equidistant autoencoder model for any data class results in a minimal amount of semantic distortion.

[0074] Figure 6 An overview of an example training process for semantic processing 425NN and semantic recovery 465NN according to various example embodiments is shown. Figure 6 The training process described is just one example of a training process, and other training processes may be used to train the semantic processing 425NN and the semantic recovery 465NN.

[0075] In 610, a β-Variational Autoencoder (β-VAE) is trained using a semantic distance training dataset including meta-parameters β. In this training 610, the β-VAE encoder and the β-VAE decoder are jointly trained, but only the β-VAE encoder is retained.

[0076] Figure 7 An example parameter model for β-Variational Autoencoder (β-VAE) training according to various example embodiments is shown. Figure 7 As shown, the parameter model Pθ(z) of the latent space 710 can be derived from the data distribution 720, where for the encoder and decoder respectively, Typical choices for prior and posterior distributions can be as follows:

[0077] That is, the latent space 710 variables are standard Gaussian distributions; and That is, the encoder maps the data points to a Gaussian distribution on the latent space 710 with a non-trivial mean and variance.

[0078] If β is considered to be a fixed regularization parameter, the optimization objective in β-VAE training can be the following evidence lower bound (ELBO):

[0079]

[0080] The ELBO is optimized using a gradient-based back-propagation method.

[0081] Under these assumptions,

[0082]

[0083] where x φθ is the output of the encoded sample, then the decoding of x is:

[0084]

[0085] return Figure 6 , after β-VAE training 610, there is a β-VAE encoder model. This β-VAE encoder model can be input to a non-linear scaling module 620 to add a scaling layer to the trained β-VAE encoder. This expanded encoder can be considered as a semantic processing model.

[0086] Additional details about the operations performed by the scaling module 620 are provided below. The β-VAE encoder model is not isometric, for example, geodesic distance does not map to Euclidean distance. That is, after training the β-VAE encoder model, the following encoder mapping is obtained:

[0087]

[0088] f(x): = (μ(x; φ opt ),σ(x;φ opt )), where μ(x; φ opt ) and σ(x; φ opt ) is an M-dimensional vector.

[0089] To obtain an isometric representation in the latent space 710, the following transformation (non-linear scaling) may be applied:

[0090] For each m=1, ..., M,

[0091] Therefore, the isometric semantic processing mapping can be defined as:

[0092] For each m=1, ..., M,

[0093] The normalized semantic processing mapping can be defined as:

[0094] For each m=1, ..., M,

[0095] In some example embodiments, the power control transformation may also be applied to the optimal Gaussian source transmission as follows:

[0096] For each m=1, ..., M,

[0097] When no data class is provided to the communication system, power control scaling should be performed at the application layer and the normalized semantic processing mapping is used as the semantic processing mapping, e.g. f:=f norm .

[0098] On the other hand, when the communication system is informed of the data class, power scaling can be skipped at the application layer and can be performed at the MAC and / or PHY layer. In this case, data statistics can be provided to those communication layers and isometric mapping is used as semantic processing mapping, e.g. f:=f iso .

[0099] Return again Figure 6 , after β-VAE training 610, there is a β-VAE encoder model. The β-VAE encoder model can also be input to the noise dimension removal module 630. In this example, M' dimensions (of M) are eliminated and the value M-M' is provided to the lower layers of the system.

[0100] Additional details about the operations performed by the noise dimension removal module 630 are provided below. For example, let X comprise a random variable corresponding to the "ground truth" distribution of the input data set. X^ is encoded, sampled, and then decoded: X^:=f -1 (f(X)+ε). Y includes random variables corresponding to the semantic representation of X, that is, Y:=f(X), and is defined as a sampled object in the representation space (i.e.,

[0101] The isometric property between the initial data space and the latent representation space implies the following equation:

[0102]

[0103] Therefore, after semantic processing, the entropy H(y) of each dimension m=1, ..., M of the representation vector can be estimated based on the data set. m ) and compare it with The dimensions can then be sorted in descending order based on the estimated entropy. Any dimension of can be eliminated. These operations can be performed by multiplying the representation vector y by a rectangular M×M-M′ permutation matrix (which can be denoted by P).

[0104] Return again Figure 6 , the final operation is the training of the semantic restoration model 640. Figure 6 As shown, the input of the training may include the semantic processing model output from the nonlinear scaling module 620, the rectangular M×M-M' permutation matrix output by the noise dimension elimination module 630, and the semantic distance training data set including the meta-parameters β and c. The meta-parameter c may depend on the meta-parameter β, but this is not necessary. The training of the semantic restoration model can provide a trade-off between reconstruction accuracy and robustness.

[0105] After training the semantic processing model f, a second model, the semantic reconstruction model, can be trained to approximate the inverse mapping (f) -1 Since the system will send the semantic representation y through a non-deterministic channel that allows for distortion, the reconstruction model should be robust enough to account for the distortion.

[0106] The reconstruction model can be trained to minimize the following function:

[0107]

[0108] The first summand listed below corresponds to the accuracy of the reconstruction, while the second summand listed below is related to robustness and generalization ability.

[0109]

[0110] in

[0111]

[0112] Because the noise is generated as a normal variable with variance that corresponds to the theoretical variance of the semantic representation conditioned on the corresponding input vector.

[0113] Although in Figure 6Although not shown in the figure, the training process may also include a quality control loop to check the final quality of the training system of the model from the perspective of accuracy and resilience. For example, the value of the objective function may be determined (or estimated based on the validation set), such as the following value:

[0114]

[0115] The determined or estimated value can be compared with a predetermined threshold. If the determined or estimated value changes by more than the predetermined threshold, the model can be retrained with more stringent requirements for optimization quality. This process can be repeated until the target of model quality is met.

[0116] Re-reference Figure 4 , another process that may be performed at the application layer 420 or 460 of the respective transmitter 410 or receiver 450 is motion compensation. On the transmitter 410 side, this is shown as motion differential 428, while on the receiver 450 side, this is shown as motion integral 468. As described above, motion compensation may not be applicable to all data types or categories of data. Example details of motion compensation that may be performed by the motion differential 428 of the transmitter 410 or the corresponding motion integral 468 of the receiver 450 are provided below.

[0117] In the potential output space of semantic processing, it can be considered that there are several continuous semantic representations of the data input, such as y t-2 ,y t-1 ,y t Several parameters related to motion can be defined based on the semantic representation of the data input. In a first example, a motion vector (MV) can be defined as the difference between two consecutive representations, for example MV(t): = y t -y t-1 In a second example, the motion vector difference (MVD) may be defined as the difference between two consecutive representations: MVD(t): = MV(t) - MV(t-1) = y t -2y t-1 +y t-2 .

[0118] It can be considered that the communication system can send three types of messages at any time (t), such as y t , MV(t) or MVD(t). t The messages can be considered analogous to the "I frames" in standard video transmission. The MV(t) and MVD(t) messages can be analogous to the different kinds of "P frames" in standard video transmission. As mentioned above, semantic communication is different from standard communication, and this analogy is not meant to indicate that semantic communication messages (y t, MV(t) or MVD(t)) are the same as for standard video transmission. Instead, this is just meant to explain how motion compensation in semantic communication is achieved by using the context of standard video transmission.

[0119] At the receiver 450, rMV(t) and rMVD(t) are the received semantic representation, the received motion vector and the received motion vector difference, respectively. Because the message is sent using a semantic communication method, the received semantic representation may be different from the sent semantic representation.

[0120] If the receiver 450 has a buffer of size 1 for the received representation, the receiver 450 may use the following formula: and rMV(t)

[0121]

[0122] Similarly, if the receiver 450 has a buffer of size 2 for the received representation, the receiver 450 may use and rMVD(t)

[0123]

[0124] Sending motion vector differences instead of semantic representations may be preferred because it reduces time-dependent redundancy. However, in the case of using the received approximate representation to obtain the next approximation, the approximation errors accumulate. Compared to traditional methods (i.e., video codecs), the motion vector differences depend linearly on the semantic representation, which makes motion estimation and compensation computationally simple.

[0125] To implement motion compensation, the transmitter 410 and the receiver 450 may agree on a message type pattern. For example, if the message types are considered to be semantic representation (R), motion vector (MV), and motion vector difference (MVD), the transmitter 410 and the receiver 450 may agree on a pattern for each data type. An example pattern may be RR MVD MVD MVD MVD MVD MVD R R... This means that the transmitter 410 will send two consecutive semantic representations, then send five consecutive motion vector differences, and then repeat the pattern.

[0126] In some example embodiments, a mode can be described by three natural values ​​(A, B, C), where the first value corresponds to the number of consecutive semantic representations (R) in the mode, the second value corresponds to the number of consecutive motion vectors (MV), and the third value corresponds to the value of consecutive motion vector differences (MVD).

[0127] The UE or the network may have a cross-layer mechanism to provide the application with the desired transmission rate. The application may then select a mode taking into account the provided rate expectation.

[0128] The process of replacing the initial semantic representation with MV or MVD at the transmitter 410 is called "motion differentiation". Figure 4 The process of restoring the original semantic representation at the receiver 450 using the buffer data and the newly received MV or MVD is called "motion integration", as shown in FIG. Figure 4 The motion integral 468 is shown in FIG.

[0129] As described above, in some example embodiments of notifying the communication system of the data class, power scaling is skipped at the application layer and power scaling is performed at the MAC and / or PHY layer. Also as described above, in these example embodiments, an isometric mapping is used as a semantic processing mapping, e.g., f:=f iso In these example embodiments, data statistics specific to a particular category may be used to provide to a communication layer that performs power scaling, such as to a MAC layer for rate matching / data adaptation processes and / or to a BB PHY for digital signal processing algorithms. Some examples of data statistics that may be provided to a MAC or PHY layer are provided below.

[0130] In a first example, the covariance matrix of data vectors represented isometrically in the latent space is provided as follows:

[0131] For each k, m=1, ..., M,

[0132] In a second example, if the motion vectors of the latent representation are to be sent, the covariance matrix of the MV can be provided as follows:

[0133] For each k, m=1, ..., M,

[0134]

[0135] In a third example, if the motion vector differences of the latent representation are to be sent, the covariance matrix of the MVD may be provided as follows:

[0136] For each k, m=1, ..., M,

[0137] These example matrices can be calculated after model training and then shared between the transmitter 410 and the receiver 450. The matrices can be sparse, where small elements can be replaced with zeros. Sparse matrix formats can be used for transmission and storage.

[0138] return Figure 4 , describes the operation of the PHY layers 440 and 480 with respect to semantic communication. For the purposes of this specification, the PHY channels used for semantic communication are referred to as best effort (BE) PHY channels. This term is used to distinguish semantic communication PHY channels from legacy channels. BE PHY channels (e.g., BE-Physical Downlink Shared Channel (BE-PDSCH) and BE-Physical Uplink Shared Channel (BE-PUSCH)) are shared (data) channels that exist along with the legacy PDSCH / PUSCH. In some example embodiments, transmissions from / to a device may be scheduled in both the BE channel and the legacy channel within the same time slot.

[0139] The BB / RF PHY processes 448 and 488 for the BE-PHY channels may be different from those of the legacy PHY channels. For example, the modulation and channel coding processes in the BE PHY channels may use pseudo-analog modulation or may not use forward error correction (FEC) channel coding / decoding. In some example embodiments, the BE PHY channels may use QAM, M-PSK, or OOK modulation. In other example embodiments, the BE PHY channels may still use the lightweight FEC channel coding / decoding option.

[0140] In some example embodiments, the BE PHY channel can support a single user, such as the superposition of different modulation orders related to the importance of the potential feature / bit. This is similar to the LTE-AMUST feature except that it is used for a single user and a single stream. For example, QPSK, 16QAM, up to the highest possible order of QAM can be used, so that the most effective potential feature / bit is mapped to QPSK, the next relevant potential feature bit (adding some details) is mapped to 16QAM, and so on, until the highest order QAM that will be used for the least effective potential feature / bit. UEs near the edge of the cell may not be able to successfully demodulate higher modulation orders, while UEs near the center of the cell may be able to successfully demodulate all modulation orders and therefore will have the maximum details for reconstruction. Some decoding can be introduced, such as orthogonal cover codes (OCC), so that the superimposed potential feature / bit information can be decoupled / decoded.

[0141] In some example embodiments, the downlink control information (DCI) for the BE data block may contain a modulation scheme index instead of the modulation and coding scheme. The same data block may use multiple modulation schemes within a single transmission. In this case, the DCI indicates all modulation schemes used and the physical resources using each modulation scheme.

[0142] As described above, in some example embodiments, a data category specific semantic model and a motion compensation process are applied only at the application layer (e.g., application layer 420), and the communication layers (e.g., L2 and L3 layers 430 and PHY layer 440) may not be informed about the data category of the transmitted data.

[0143] In some example embodiments, the header sent via the legacy channel (shared or control channel) may include a low-level ID of the data vector being sent to help the receiver 450 assemble several data vectors that are sent in an overlapping manner. In addition, the header sent via the legacy channel (shared or control channel) may include an end-of-data vector indication that notifies the receiver 450 that no further data for the data vector will be sent. When the receiver 450 receives the end-of-data indicator, this indicates to the receiver 450 that it should attempt to reconstruct the data vector from the received data.

[0144] As mentioned above, semantic communication can use pseudo-analog modulation. This is not a requirement for semantic communication, but an option for modulation. The modulation can be defined as a mapping: m,n∈N. In this case, the m-dimensional vector is modulated by n resource elements (e.g., orthogonal frequency division multiplexing (OFDM) subcarriers), which can be jointly demodulated. Modulation / demodulation can be implemented with digital circuits; therefore, all "real" values ​​are replaced by high-precision bit representations.

[0145] In one example, direct pseudo-analog modulation (1:1 mapping) may be used as follows:

[0146]

[0147] (x,y)→x+iy

[0148] In another example, a 1:2 Shannon-Kotelnikov map (Archimedes spiral) may be used as follows:

[0149]

[0150] where Δ depends on the signal-to-interference ratio (SINR) estimate.

[0151] In another example, for n>1, a 1:2n Shannon-Kotelnikov mapping may be used.

[0152] The OFDM waveform may be used in combination with a predefined series of pseudo-analog modulations ordered by the dimension n. In these examples, higher order modulations are able to modulate larger values ​​with lower power at the expense of additional resource consumption.

[0153] In a further example embodiment, an additional process of isometrically mapping the semantic representation vector into a Hamming cube is applied. This can be used to allow conventional modulation schemes (QAM, PSK, OOK, etc.) to be used for semantic communication, such as the superposition concept described above. The Hamming cube can be defined as the space of fixed-length binary sequences with Hamming distances. The isometric mapping i satisfies the following properties:

[0154] ||yy′|| 2 =d Ham (i(y),i(y′))

[0155] O(n log n) algorithms are known for Hamming embedding based on random cyclic projection.

[0156] Likewise Figure 4 As shown, PHY layers 440 and 480 may perform whitening 445 and de-whitening 485 operations, respectively. Again, it is not required to apply whitening 445 and de-whitening 485 operations. Whitening operation 445 may be applied to make the data distribution closer to a normal distribution and achieve a more balanced power allocation. Whitening operation 445 may be applied to a single data vector assigned with the same modulation and equidistant (orthogonal) linear transformation, and may be applied before actual modulation.

[0157] The whitening operation 445 may include centering of the data, application of a Hadamard transform, etc. The Hadamard transform is unitary and therefore isometric and has a fast implementation. These transforms are reversible, and an inverse transform may be performed at the receiver 450, such as a de-whitening operation 485.

[0158] Turning to the L2 and L3 layers 430 and 470 of the transmitter 410 and the receiver 450, respectively, the MAC layer may be responsible for resource allocation in the BE-PDSCH and / or BE-PUSCH. For UEs with BE transmission requirements, the MAC provides radio resources (in frequency, time, and spatial domains) and power resource budgets.

[0159] Therefore, at the transmitter 410 side, after semantic processing 425 (and motion differentiation 428, if applicable), each semantic representation is a vector with appropriately scaled components due to the equidistance property described above. Therefore, the value of a vector component can be positively correlated with the transmit power used to transmit the value.

[0160] Figure 8An example of a data vector of a semantic representation at the MAC layer of the transmitter 410 according to various example embodiments is shown. The MAC layer may include a local system of IDs for data vectors of a particular UE. With this mechanism, the MAC is able to distinguish data of different data vectors (e.g., data vector #1, data vector #2, and data vector #3).

[0161] Based on the power and radio resources provided, and also considering hardware (RF) power limitations, the MAC layer can choose Figure 8 The order of the modulation scheme of the data shown, such as 1:4 modulation, 1:2 modulation, 1:1 modulation. Figure 4 As shown, the MAC layer is responsible for SINR estimation and parameter Δ selection for 1:n modulation (where n>1). The use of parameter Δ in modulation is described above with reference to the PHY layer. Figure 4 As shown, the selected value of the parameter Δ may be provided to the receiver 450 via a control channel DCI.

[0162] The data of a particular data vector (eg, data vector #1) may be mapped onto the radio resources such that the component order is preserved. If the data is sent using a different modulation scheme, the order may be compatible with the highest to lowest modulation scheme order.

[0163] In some example embodiments, if the MAC layer decides to discard the remainder of the data vector, the MAC layer of the transmitter 410 may have a protocol opportunity to explicitly inform the receiver 450 of the decision.

[0164] The above description provides the system architecture ( Figure 4 ) and a general way of performing semantic communication. A more detailed example of a specific type of semantic communication, called soft communication, is provided below. Soft communication is a communication mode for wireless links that is considered to deliver data that allows some distortion. Typically, soft communication can be used for digital data types of data categories, such as audio data, video data, etc. There may be other types of semantic communication that are different from soft communication, such as sub-stream communication, programmable protocol semantic communication, etc. Different types of semantic communication are not mutually exclusive, and they can be combined in different scenarios and settings.

[0165] In the general example of semantic communication described above, the PHY layer is described to implement pseudo-analog modulation for semantic communication. In the example of soft communication, the MAC layer can be agnostic to the PHY layer modulation, for example, the example operation of the MAC layer can be used with any type of selected PHY layer modulation. In addition, as will be described in more detail below, the example MAC layer design for soft communication can allow both numerical data and metadata (including packet headers) to be sent within a single MAC PDU (TB).

[0166] As mentioned above, soft communication is often used to represent numerical values ​​of typical outputs of machine learning or other types of data processing algorithms, such as integer or rational data types. The goal of the system is to reduce distortion, such as the sum of distances between numerical values, as follows:

[0167] Where x is the initial value, is the received value, and

[0168] p is the configured system parameter

[0169] Based on this approach, delay and distortion requirements can be configured by the system to derive retransmission strategies and upper layer processing of the data.Application layer source decoding can be designed so that the vector normal distortion of the encoded data corresponds to the perceptual (semantic) distortion of the original data.

[0170] Fig. 9 An example Internet Protocol (IP) / Transport Level Protocol packet 900 for soft communication according to various example embodiments is shown. In this example, a lightweight User Datagram Protocol (UDP-Lite) may be applied to the data for soft delivery. UDP-Lite may be used so that damaged packets will not be discarded by IP routers. Since those skilled in the art will understand the purpose of each field and the type of data that may be included in each field, data packet 900 is not described in its entirety.

[0171] Instead, the present description will focus on fields that can be configured for soft communication (e.g., data type field 910). The data type field 910 can be configured for each packet and can include a description of the bit precision and how to interpret the bits, such as 8-bit unsigned integer (uint8), 16-bit signed integer (int16), etc.

[0172] Other examples of data type options that reduce real-time DCI overhead by providing only an index of the options used may include indicating a finite field operation Z or Z q The sign of an integer, where q is determined by the bit precision q: = 2 b , distortion power (p), parameters related to the distortion estimation method and the preferred method (for example, the soft checksum method), the threshold of acceptable distortion, parameters related to the encryption strategy and the preferred method, etc.

[0173] Fig.10An example of a radio link control (RLC) protocol data unit (PDU) 1000 including a MAC PDU 1010 for soft communication according to various example embodiments is shown. As described above, a semantic communication system can schedule both legacy communication (e.g., PDSCH) and semantic communication (e.g., BE-PDSCH) simultaneously. Thus, the RLC PDU 1000 includes a legacy portion 1020 (or precise portion) and a soft delivery portion 1030. The MAC layer may be responsible for mapping MAC service data units (SDUs) (or portions thereof) into the legacy portion 1020 and the soft delivery portion 1030 of the RLC PDU 1000. As described in more detail below, for the soft delivery portion 1030 of data, the MAC module may be used to adjust soft PHY parameters, and in some examples, send only a portion of the allocated soft data.

[0174] exist Fig.10 In an example of , the legacy portion 1020 may include an extended MAC subheader (e.g., SH1 1021), a payload for precise delivery (e.g., SDU1 1022), a soft checksum for a soft delivery portion (e.g., SD-SC1 1023), and subsequent consecutive triplets of soft delivery control information. The extended MAC subheader may be, for example, a 5G NR MAC subheader with an additional field indicating the length of the accompanying soft delivery SDU portion (SD-SDU) and indicating the bit format (related to the data type) of the soft delivery data in the SDU.

[0175] The soft checksum of a soft delivered SDU may be a value of a function of the corresponding SD-SDU, which is used for soft integrity checking purposes and retransmission control strategy. There may also be a soft checksum for soft delivery of the entire MAC PDU (eg, SD-SC MAC PDU 1024).

[0176] In an example implementation of soft communication, the MAC scheduler provides resource allocation for both traditional (precision delivery) PDSCH and soft PDSCH. For example, the MAC scheduler can select the modulation and coding scheme (MCS) for traditional PDSCH and the multiple input / multiple output (MIMO) rank for both traditional PDSCH and soft PDSCH. In order to complete the scheduling, the MAC scheduler can receive a buffer report including the data size of the precise communication and the data size of the soft communication. The DCI can be correspondingly extended to include two rank indicators and allocation information for the soft part of the transport block (TB).

[0177] In addition, the selection of PHY parameters for soft PDSCH and soft PUSCH (e.g., modulation, FEC decoding, etc.) can be performed by the MAC entity of the transmitter, such as UE MAC for UL and network MAC for DL.

[0178] On the receiver side, the MAC layer can implement various processes for soft communication in both the downlink (DL) and the uplink (UL). In the DL, the MAC layer can receive the physical downlink control channel (PDCCH) and decode the DCI to identify the allocation of the traditional PDSCH and the soft PDSCH. Then, the MAC layer can receive the traditional PDSCH and the soft PDSCH, and decode the traditional PDSCH to obtain the location of the soft part (if any) of the data SDU. Then, the MAC layer can obtain the soft control information and soft integrity check value of each soft MAC SDU and the soft integrity check of the entire soft PDSCH. Then, the MAC layer can decode the soft PDSCH after the soft control information, as shown above. Then, the MAC layer can calculate the soft integrity check function based on the decoded soft data and compare it with the soft integrity check value delivered in the PDSCH. If necessary, this may include requesting retransmission. Then, the MAC layer can reconstruct the MAC SDU from the precise and soft delivery parts and provide them to the upper layer.

[0179] In the UL, the MAC layer may receive the traditional PUSCH and the soft PUSCH, decode the traditional PUSCH to obtain the location of the soft part of the data SDU (if any), the soft control information and soft integrity check value of each soft MAC SDU, and the soft integrity check of the entire soft PUSCH. The MAC layer may then decode the soft PUSCH after the soft control information. The MAC layer may then calculate the soft integrity check function based on the decoded soft data and compare it with the soft integrity check value delivered in the PUSCH, including requesting retransmission (if necessary). The MAC layer may then reconstruct the MAC SDU from the precise and soft delivery parts and provide them to the upper layer.

[0180] Turning to the PHY layer for soft communication, the input to the PHY layer of the transmitter may include a bit group G0, G1, ..., G B-1 The data is organized as follows, where G0 contains the least significant bit, G1 contains the second least significant bit, etc. For each bit group, the MAC layer provides the modulation order Q to the PHY layer. b , decoding rate R b and allocating radio resources to the group.

[0181] The PHY layer may also provide an abstraction of the bit error rate (BER) function of the soft channel code. The BER function may be obtained via link-level simulation and used for PHY abstraction in system-level simulation. The BER may be viewed as a function of various parameters such as modulation order, effective SINR, code block size, coding rate, code type, decoder options, etc. The effective SINR model may be an exponential effective SINR mapping (EESM), where the averaged parameters are appropriate for coding rate, modulation order, code type, etc. For example, for a fixed code type, decoder option, code block size:

[0182]

[0183] For the norm, where x k For a value represented as an unsigned int using B bits, the error contribution in the bth bit is 2 p·b . Therefore, the total distortion of the received soft data will be:

[0184] Where k is the index of the value sent.

[0185] Therefore, as described above, the input from the MAC scheduler to the PHY layer includes PHY resource allocation (grant) for both the legacy part and the soft delivery part, the MCS for the legacy part, and the MIMO rank for both the legacy part and the soft delivery part. Then, based on this input, the RLC PDU (or part of the RLC PDU) can be selected for transmission in the TB.

[0186] As also mentioned above, the inputs from the PHY layer include SINR estimates for the provided radio resources, In addition, from the upper layer configuration, the PHY layer can receive a distortion norm option (e.g., the exponent (p) as described above), and from the RLC PDU, including bit format information.

[0187] Fig.11 An example soft delivery portion 1100 of a transport block (TB) for soft communication according to various example embodiments is shown. As described above, the MAC layer may identify PHY layer parameters for the soft delivery portion so that the soft delivery portion of the TB may be transmitted according to the following example. Fig.11 The following contents are constructed as shown.

[0188] The data may be represented by a set of S blocks (which may correspond to an SD-SDU), where each block is characterized by a specific bit format B that it contains s and multiple values ​​K s These blocks can be opened by pressing B s In descending order, B:=max{B1,......,BS} is the maximum bit format.

[0189] As described above, bit groups G0, G1, . . . , G B-1 , where G0 contains the least significant bit, etc. For each bit group G b , the MAC layer can select the QAM modulation scheme Q b and decoding rate R b To minimize the desired distortion, for example:

[0190]

[0191]

[0192] Soft delivery control information may include selected

[0193] The case where a single MCS is used for all soft transmitted bits within a TB can be a special case of PHY parameter adjustment. Specifically, the system can select bits to be transmitted and discard these bits. For example, with knowledge of the data structure (e.g., data type), the MAC can select only an appropriate number of significant bits for transmission. The soft control information may include a value indicating the number of transmitted bits for each value (bit precision). The receiver can then construct the decoded data based on the soft control information and update the bit format (precision) in the packet data type attribute.

[0194] Example

[0195] In a first embodiment, a method includes: receiving data input corresponding to a plurality of data objects to be sent by a UE; processing the data input to generate a semantic representation of each of the data objects, wherein the semantic representation includes a semantic distance for each of the data objects, wherein the semantic distance relates each of the data objects to each other; and preparing the semantic representation of the data objects for sending.

[0196] In a second embodiment, according to the method of the first embodiment, the semantic distance comprises a geodesic distance or a distance defined by domain knowledge.

[0197] In a third embodiment, according to the method of the first embodiment, the data includes one or more data categories, wherein the semantic distance is defined separately for each data category.

[0198] In a fourth embodiment, the method according to the third embodiment, wherein the data is processed by a semantic processing machine learning (ML) model, wherein the ML model includes weights specific to each data category.

[0199] In a fifth embodiment, the method according to the fourth embodiment, wherein the ML model comprises a diagonal recurrent neural network model.

[0200] In a sixth embodiment, the method according to the first embodiment further includes determining a first motion vector between a first semantic representation of a first data input and a second semantic representation of a second data input, wherein the first data input and the second data input are continuous data inputs.

[0201] In a seventh embodiment, the method according to the sixth embodiment further comprises preparing the motion vector instead of the first semantic representation for transmission.

[0202] In an eighth embodiment, according to the method of the seventh embodiment, the motion vector is sent based on a pattern, wherein the pattern is exchanged between the UE and another UE to which the UE is sending the motion vector.

[0203] In a ninth embodiment, the method according to the sixth embodiment further includes: determining a second motion vector between the second semantic representation of the second data input and the third semantic representation of the third data input, wherein the second data input and the third data input are continuous data inputs; determining a motion vector difference between the first semantic representation and the third semantic representation based on the first motion vector and the second motion vector; and preparing the motion vector instead of the first semantic representation for sending.

[0204] In a tenth embodiment, according to the method of the ninth embodiment, the motion vector difference is sent based on a pattern, wherein the pattern is exchanged between the UE and another UE to which the UE is sending the motion vector difference.

[0205] In an eleventh embodiment, the method according to the first embodiment further includes processing each of the semantic representations by a medium access control (MAC) layer or a physical (PHY) layer based on a covariance matrix of a data vector related to each of the semantic representations.

[0206] In a twelfth embodiment, the method according to the eleventh embodiment, wherein the MAC layer processes each of the semantic representations to perform rate matching or data adaptation.

[0207] In a thirteenth embodiment, the method according to the first embodiment, wherein the semantic representation is transmitted using a shared channel different from a shared channel used for conventional data transmission.

[0208] In a fourteenth embodiment, the method according to the first embodiment further comprises modulating, by a physical (PHY) layer, the semantic representations, wherein the modulation is based on the importance of underlying features or bits in each of the semantic representations.

[0209] In a fifteenth embodiment, the method according to the first embodiment further includes determining, by a medium access control (MAC) layer, a modulation to be applied to the semantic representation for transmission, wherein the modulation is determined based on radio resources and a power budget allocated to the semantic representation.

[0210] In a sixteenth embodiment, the method according to the fifteenth embodiment further comprises preparing the determined modulation to be applied to the semantic representation for transmission to a receiver via downlink control information (DCI).

[0211] In a seventeenth embodiment, the method according to the first embodiment further includes generating an identifier of the transmitted semantic representation or a data end vector to be sent to a receiver via downlink control information (DCI), wherein the data end vector indicates that no additional data will be sent for the transmitted semantic representation.

[0212] In an eighteenth embodiment, a processor is provided, wherein the processor is configured to execute any one of the methods described in accordance with the first to seventeenth embodiments.

[0213] In a nineteenth embodiment, a user equipment (UE) is provided, wherein the user equipment (UE) is configured to perform any one of the methods described in accordance with the first to seventeenth embodiments.

[0214] In a twentieth embodiment, a method includes: jointly training a variational autoencoder (VAE) and a VAE decoder based on a training dataset including data related to semantic distance; adding a scaling layer to the VAE to generate a semantic processing model; eliminating a noise dimension from the VAE; and training a semantic processing reconstruction model based on the training dataset, the semantic processing model, and data generated by eliminating the noise dimension.

[0215] In a twenty-first embodiment, the method according to the twentieth embodiment further includes discarding the VAE decoder after joint training.

[0216] In a twenty-second embodiment, according to the method of the twentieth embodiment, the optimization target of the joint training of the VAE and the VAE decoder is an evidence lower bound.

[0217] In a twenty-third embodiment, according to the method of the twentieth embodiment, the scaling layer is added to generate an isometric semantic processing model.

[0218] In a twenty-fourth embodiment, the method of the twentieth embodiment, wherein the processing circuit eliminates the noise dimensions based on comparing the entropy of each noise dimension to an entropy threshold.

[0219] In a twenty-fifth embodiment, a processor is provided, wherein the processor is configured to execute any one of the methods described in accordance with the twentieth to twenty-fourth embodiments.

[0220] In a twenty-sixth embodiment, a user equipment (UE) is provided, wherein the user equipment (UE) is configured to perform any one of the methods described in accordance with the twentieth to twenty-fourth embodiments.

[0221] In a twenty-seventh embodiment, a method comprises: receiving a data input corresponding to a data object to be sent; processing the data input to generate a semantic representation of the data object; generating an Internet Protocol (IP) packet including the semantic representation and a header indicating a data type of the semantic representation; and generating a transport block (TB) including a soft delivery part and a precise delivery part, wherein the soft delivery part includes the semantic representation of the data object.

[0222] In a twenty-eighth embodiment, the method according to the twenty-seventh embodiment, wherein the header also includes (i) a tag indicating that the data object is an integer of a finite field operation, (ii) a distortion power parameter, (iii) a parameter related to distortion estimation, (iv) a threshold of acceptable distortion, or (v) a parameter of an encryption strategy.

[0223] In a twenty-ninth embodiment, the method according to the twenty-seventh embodiment, wherein the precise delivery part includes an extended MAC subheader, a precise delivery payload, a soft checksum of the soft delivery part, and soft delivery control information.

[0224] In a thirtieth embodiment, the method according to the twenty-ninth embodiment, wherein the extended MAC subheader includes fields indicating the length of the soft delivery part and the bit format of the semantic representation.

[0225] In a thirty-first embodiment, the method according to the twenty-ninth embodiment, wherein the precise delivery part also includes a triplet of the extended MAC subheader, the precise delivery payload, the soft checksum of the soft delivery part, and the soft delivery control information.

[0226] In a thirty-second embodiment, the method according to the thirty-first embodiment, wherein the precision delivery portion also includes a soft checksum of the entire TB.

[0227] In a thirty-third embodiment, the method according to the twenty-seventh embodiment further includes receiving, by a medium access control (MAC) layer, a buffer report including data sizes of the precise delivery part and the soft delivery part.

[0228] In the thirty-fourth embodiment, the method according to the thirty-third embodiment further includes: determining, by the MAC layer, resource allocation of the precise delivery part and the soft delivery part based at least on the data size used for the precise delivery part and the soft delivery part; selecting, by the MAC layer, a modulation and coding scheme (MCS) for the precise delivery part based at least on the data size of the precise delivery part; and selecting, by the MAC layer, a multiple input / multiple output (MIMO) rank for each of the precise delivery part and the soft delivery part based at least on the data size of the precise delivery part and the soft delivery part.

[0229] In a thirty-fifth embodiment, the method according to the thirty-third embodiment further includes determining, by the MAC layer, selection of physical (PHY) layer parameters for sending the soft delivery part.

[0230] In a thirty-sixth embodiment, the method according to the thirty-fifth embodiment, wherein the PHY layer parameters include modulation or forward error correction (FEC) decoding.

[0231] In a thirty-seventh embodiment, the method according to the thirty-fourth embodiment further includes configuring downlink control information (DCI), wherein the downlink control information (DCI) includes a MIMO rank indicator for each of the precise delivery part and the soft delivery part and allocation information for the soft delivery part.

[0232] In a thirty-eighth embodiment, a processor is configured to execute any one of the methods described in embodiments twenty-seven to thirty-seven.

[0233] In a thirty-ninth embodiment, a user equipment (UE) is configured to perform any one of the methods described in accordance with the twenty-seventh to thirty-seventh embodiments.

[0234] Those skilled in the art will appreciate that the example embodiments described above may be implemented with any suitable software configuration or hardware configuration or combination thereof. Example hardware platforms for implementing the example embodiments may include, for example, Intel x86-based platforms with compatible operating systems, Windows OS, Mac platforms and MAC OS, mobile devices with operating systems such as iOS, Android, etc. The above embodiments may be embodied as a program comprising lines of code stored on a non-transitory computer-readable storage medium, which, when compiled, may be executed on a processor or microprocessor.

[0235] In some embodiments, a non-transitory computer-readable storage medium (e.g., a non-transitory storage element) may be configured to store program instructions and / or data, wherein if the program instructions are executed by a computer system, the computer system is caused to perform a method, such as any method implementation of the method implementations described herein, or any combination of the method implementations described herein, or any subset of any method implementation of the method implementations described herein, or any combination of such subsets.

[0236] In some embodiments, a device (e.g., a UE) may be configured to include a processor (or a group of processors) and a memory medium (or a memory element), wherein the memory medium stores program instructions, wherein the processor is configured to read and execute these program instructions from the memory medium, wherein these program instructions are executable to implement any method implementation of the various method implementations described herein (or any combination of the method implementations described herein, or any subset of any method implementation of the method implementations described herein, or any combination of such subsets). The device may be implemented in any of various forms.

[0237] Embodiments of the present invention may be implemented in any of a variety of forms. For example, in some embodiments, the present invention may be implemented as a computer-implemented method, a computer-readable storage medium, or a computer system. In other embodiments, one or more custom-designed hardware devices such as ASICs may be used to implement the present invention. In other embodiments, one or more programmable hardware elements such as FPGAs may be used to implement the present invention.

[0238] Although this patent application describes various combinations of various embodiments each having different features, those skilled in the art will understand that any feature of an embodiment may be combined with features of other embodiments in any manner not publicly denied or with features that are not functionally or logically inconsistent with the operation or function of the device of the embodiments disclosed in the present invention.

[0239] As described above, one aspect of the present technology is to collect and use data obtained from specific and legitimate sources to improve the delivery of inspirational content or any other content that may be of interest to users. The present disclosure contemplates that, in some instances, the collected data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data may include demographic data, location-based data, online identifiers, phone numbers, email addresses, home addresses, data or records related to the user's health or fitness level (e.g., vital signs measurements, medication information, exercise information), date of birth, or any other personal information.

[0240] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit users. For example, personal information data can be used to deliver targeted content that users may be more interested in based on their preferences. Therefore, the use of such personal information data enables users to have greater control over the content delivered.

[0241] The disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, it is expected that such entities will implement and consistently apply privacy practices that are generally recognized as meeting or exceeding the industry or government requirements for maintaining user privacy. Such information about the use of personal data should be highlighted and easily accessible to users, and should be updated as the collection and / or use of data changes. The user's personal information should be collected only for legal use. In addition, such collection / sharing should only occur after receiving user consent or other legal basis specified in applicable law. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data, and ensure that other entities with access to personal information data comply with their privacy policies and procedures. In addition, such entities can subject themselves to third-party assessments to prove that they comply with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted for specific types of personal information data collected and / or accessed, and applied to applicable laws and standards, including considerations that are exclusive to jurisdictions that can be used to impose higher standards. For example, in the United States, collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly.

[0242] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates hardware elements and / or software elements to prevent or block access to such personal information data. For example, such as with respect to an advertising delivery service, the technology of the present invention may be configured to allow a user to choose to "opt in" or "opt out" at any time during or after registration for the service to participate in the collection of personal information data. In another example, a user may choose not to provide emotion-related data for a target content delivery service. As another example, a user may choose to limit the length of time that emotion-related data is retained, or to completely block the development of basic emotional conditions. In addition to providing "opt-in" and "opt-out" options, the present disclosure also contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application, and then the user may be reminded again just before the personal information data is accessed by the application.

[0243] In addition, it is the intention of the present disclosure that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once the data is no longer needed, the risk can be minimized by limiting the collection of data and deleting the data. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be promoted when appropriate by removing identifiers, controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods such as differential privacy.

[0244] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, content may be selected and delivered to a user based on aggregated non-personal information data or an absolute minimum amount of personal information, such as content processed only on the user's device or other non-personal information that may be used for content delivery services.

[0245] It will be apparent to those skilled in the art that various modifications may be made to the present disclosure without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is intended to cover modifications and variations of the present disclosure, provided that these modifications and variations are within the scope of the appended claims and their equivalents.

Claims

1. An apparatus comprising a processing circuit, the processing circuit being configured to: receiving data input corresponding to a plurality of data objects to be sent; processing the data input to generate a semantic representation of each of the data objects, wherein the semantic representation includes a semantic distance for each of the data objects, wherein the semantic distance relates each of the data objects to one another; and The semantic representation of the data object is prepared for transmission. 2 . The apparatus according to claim 1 , wherein the semantic distance comprises a geodesic distance or a distance defined by domain knowledge. 3 . The apparatus of claim 1 , wherein the data comprises one or more data categories, wherein the semantic distance is defined separately for each data category.

4. The apparatus of claim 3, wherein the data is processed by a semantic processing machine learning (ML) model, wherein the ML model includes weights specific to each data category.

5. The apparatus of claim 1 , wherein the processing circuit is further configured to: A first motion vector is determined between a first semantic representation of a first data input and a second semantic representation of a second data input, wherein the first data input and the second data input are consecutive data inputs.

6. The apparatus of claim 5, wherein the processing circuit is further configured to: The motion vector is prepared for transmission instead of the first semantic representation.

7. The apparatus of claim 5, wherein the processing circuit is further configured to: determining a second motion vector between the second semantic representation of the second data input and a third semantic representation of a third data input, wherein the second data input and the third data input are consecutive data inputs; determining a motion vector difference between the first semantic representation and the third semantic representation based on the first motion vector and the second motion vector; and The motion vector is prepared for transmission instead of the first semantic representation.

8. The apparatus of claim 1, wherein the processing circuit is further configured to: Each of the semantic representations is processed by a medium access control (MAC) layer or a physical (PHY) layer based on a covariance matrix of data vectors associated with each of the semantic representations.

9. The apparatus of claim 1 , wherein the processing circuit is further configured to: The semantic representations are modulated by a physical (PHY) layer, wherein the modulation is based on the importance of underlying features or bits in each of the semantic representations.

10. The apparatus of claim 1, wherein the processing circuit is further configured to: A modulation to be applied to the semantic representation for transmission is determined by a medium access control (MAC) layer, wherein the modulation is determined based on radio resources and a power budget allocated to the semantic representation.

11. The apparatus of claim 1 , wherein the processing circuit is further configured to: An identification of the transmitted semantic representation or an end-of-data vector is generated for transmission to a receiver via downlink control information (DCI), the end-of-data vector indicating that no additional data will be transmitted for the transmitted semantic representation.

12. An apparatus comprising a processing circuit, the processing circuit being configured to: Jointly training a variational autoencoder (VAE) and a VAE decoder based on a training dataset including data related to semantic distances; Adding a scaling layer to the VAE to generate a semantic processing model; Eliminating the noise dimension from the VAE; and A semantic processing reconstruction model is trained based on the training data set, the semantic processing model and data generated by eliminating the noise dimension.

13. The apparatus of claim 12, wherein the processing circuit is further configured to: The VAE decoder is discarded after joint training. The apparatus of claim 12 , wherein adding the scaling layer generates an isometric semantic processing model.

15. The apparatus of claim 12, wherein the processing circuit removes noise dimensions based on comparing the entropy of each noise dimension to an entropy threshold.

16. An apparatus comprising a processing circuit, the processing circuit being configured to: receiving data input corresponding to a data object to be sent; processing the data input to generate a semantic representation of the data object; generating an Internet Protocol (IP) packet including the semantic representation and a header indicating a data type of the semantic representation; as well as A transport block (TB) is generated that includes a soft delivery portion and a precise delivery portion, wherein the soft delivery portion includes the semantic representation of the data object.

17. The apparatus of claim 16, wherein the header further comprises (i) a tag indicating that the data object is an integer of finite field operations, (ii) a distortion power parameter, (iii) a parameter related to distortion estimation, (iv) a threshold of acceptable distortion, or (v) a parameter of an encryption strategy.

18. The apparatus of claim 16, wherein the precise delivery portion comprises an extended MAC subheader, a precise delivery payload, a soft checksum of the soft delivery portion, and soft delivery control information.

19. The apparatus of claim 18, wherein the extended MAC subheader includes fields indicating a length of the soft delivery part and a bit format of the semantic representation.

20. The apparatus of claim 18, wherein the precise delivery portion further comprises a triplet of the extended MAC subheader, the precise delivery payload, the soft checksum of the soft delivery portion, and the soft delivery control information.

Images