A multi-user source channel adaptive semantic coding method based on deep learning

Abstract

The present invention discloses a multi-user source channel adaptive semantic coding method based on deep learning, comprising the following steps: S1. Constructing a multi-user semantic communication system based on a deep neural network: at the transmitting end, the signal to be transmitted is processed by a deep source encoder and a digital channel encoder, and then superimposed, numbered, and sent; at the receiving end, the received signal is processed by a digital channel decoder, and then processed by a data user decoder and a semantic user decoder to obtain source data and semantic information; S2. End-to-end distortion modeling of data users and semantic users is performed based on data regression; S3. Joint optimization of source channel coding rate, power allocation, and beamforming is performed. The present invention takes into account the multi-user broadcast channel and designs a joint optimization algorithm for source channel rate, power allocation, and beamforming to minimize the weighting and end-to-end distortion of the multi-user system.

Description

Technical Field

[0001] The present invention relates to the field of semantic communication, and in particular to a multi-user source channel adaptive semantic coding method based on deep learning. Background Art

[0002] To meet the growing demand for data transmission in future communication systems, sixth-generation mobile networks must achieve higher transmission efficiency than fifth-generation networks within limited spectrum resources. Semantic communication, by using artificial intelligence to extract semantic features from raw data, can significantly reduce communication resource overhead while achieving the same performance.

[0003] While existing semantic communication technologies offer superior performance compared to traditional communication technologies, they typically directly transmit analog semantic features extracted by neural networks as wireless signals, making them incompatible with digital communication systems. Furthermore, most semantic communication solutions only consider point-to-point transmission scenarios, with customized designs based on source data and channel parameters, making them difficult to scale to multi-user scenarios. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a multi-user source channel adaptive semantic coding method based on deep learning. Taking into account the multi-user broadcast channel, a joint optimization algorithm for source channel rate, power allocation and beamforming is designed to minimize the multi-user system weighting and end-to-end distortion.

[0005] The objective of the present invention is achieved through the following technical solution: a multi-user source channel adaptive semantic coding method based on deep learning, comprising the following steps:

[0006] S1. Build a multi-user semantic communication system based on deep neural network:

[0007] At the transmitting end, the signal to be transmitted is processed by the deep source encoder and the digital channel encoder, and then superimposed, numbered, and transmitted.

[0008] At the receiving end, the received signal is processed by the digital channel decoder and then processed by the data user decoder and semantic user decoder to obtain the source data and semantic information;

[0009] S2. End-to-end distortion modeling for data users and semantic users based on data regression;

[0010] S3. Perform joint optimization of source channel coding rate, power allocation, and beamforming.

[0011] The beneficial effects of the present invention are as follows: the present invention considers multi-user broadcast channels and designs a joint optimization algorithm for source channel rate, power allocation and beamforming to minimize multi-user system weighting and end-to-end distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the principle of the present invention;

[0013] Figure 2 Schematic diagram of the source codec principle based on deep neural network;

[0014] Figure 3 Schematic diagram of information recovery for data users;

[0015] Figure 4 Schematic diagram of information recovery for semantic users;

[0016] Figure 5 Schematic diagram comparing weighted and end-to-end distortion under different average bandwidth rates;

[0017] Figure 6 Schematic diagram comparing weighted and end-to-end distortion under different transmission powers. DETAILED DESCRIPTION

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0019] A multi-user source channel adaptive semantic coding method based on deep learning, comprising the following steps:

[0020] S1. Build a multi-user semantic communication system based on deep neural network:

[0021] At the transmitting end, the signal to be transmitted is processed by the deep source encoder and the digital channel encoder, and then superimposed, numbered, and transmitted.

[0022] At the receiving end, the received signal is processed by the digital channel decoder and then processed by the data user decoder and semantic user decoder to obtain the source data and semantic information;

[0023] First, we introduce the deep source encoder. Figure 2 The structure of the source encoder based on deep neural network is shown. , consider an independent and identically distributed image source semantic pair ,in Represents the original image data, Represents the semantic information corresponding to the image data, and Represent the dimensions of image and semantic information respectively. Image data Through the function Extracted dimensional continuous vector . is the function of the neural network, and the neural network parameters are Continuous eigenvectors Further through the quantization module, the discrete semantic feature vector is obtained , and then compressed using arithmetic coding to a length of Bitstream The average source coding rate of the source encoder is expressed as .

[0024] Introducing the design of the deep source decoder:

[0025] Consider two types of users: data users whose purpose is to recover source data, whose index is , is the number of data users, and the semantic users who perform downstream machine tasks through semantic information, whose index is , is the number of semantic users, Indicates the total number of users in the system, represents the set of all users. The following introduces the design schemes of the source decoder for two types of users. First, for the data user, that is, ,like Figure 3 As shown, the received bit stream First, the recovered semantic feature vector is obtained through the arithmetic decoder , and then use the source data recovery function according to Get the recovered source data ,in is the function of the neural network, is the neural network parameter. For semantic users, , similarly, if Figure 4 As shown, the received bit stream Use arithmetic decoding to get semantic feature vector , Unlike data users, semantic users use semantic information to recover functions according to Performing semantic tasks users recover semantic information ,in is the function of the neural network, are the neural network parameters.

[0026] Channel Coding:

[0027] For users , consider any Block code, consisting of a channel encoder and a channel decoder Composition, of which represents a complex set, Indicates the data bit length, represents the block length. The channel coding rate can be expressed as .

[0028] Based on the above coding scheme, the overall transmission process of the considered multi-user semantic communication system is as follows.

[0029] On the sending side, for users , input data pass Figure 1 Deep Source Encoder in The encoding length is Bitstream .Then, Divided into indivual Length of the data packet, where Indicates rounding up operation. data packets, using a channel coding rate of Channel encoder Encode it as a length The symbol sequence The transmission symbol meets the unit power constraint .

[0030] use Represents a user The channel transmission symbol sequence is The symbols transmitted during secondary channel transmission use the unit beamforming vector and power All user transmission symbols are superimposed and coded, which can be expressed as

[0031] (1)

[0032] in . The sender broadcasts the message to all receivers for transmission.

[0033] On the receiving end, for the user , in The symbol received during secondary channel transmission is

[0034] (2)

[0035] in From the sender to the user The channel parameters, Indicates the number of antennas at the transmitting end, The mean is 0 and the variance is Independent and identically distributed circularly symmetric complex Gaussian (CSCG) noise. So the user The signal-to-interference-and-noise ratio (SINR) can be expressed as

[0036] (3)

[0037] The receiving end continues to receive channel symbols according to formula (2) until the complete received symbol sequence is obtained. , then use the channel decoder Will Decoded into the recovered bit sequence According to the finite code length random coding rate formula, the average packet error probability is

[0038] (4)

[0039] in represents the Gaussian Q function.

[0040] In getting Afterwards, Figure 1 As shown, for data users , using the data user decoder Will Decoding to obtain the recovered source data For semantic users , using semantic user decoder Will Decoding to recover semantic information The end-to-end distortion of data users during transmission is expressed as ,in Represents a user All channel noise experienced, represents the data distortion metric. The end-to-end distortion of semantic users is expressed as , represents the semantic distortion measure.

[0041] S2. End-to-end distortion modeling for data users and semantic users based on data regression;

[0042] Now let’s introduce the end-to-end distortion between data users and semantic users. and The introduction is divided into end-to-end distortion modeling based on data regression and bit error rate analysis.

[0043] End-to-end distortion modeling based on data regression

[0044] First, we introduce the end-to-end distortion modeling method for data users. For the image source, the source coding rate is arranged from small to large as follows: Next training Data compression neural network models, each model consists of an encoder and a decoder, the encoder corresponds to Figure 1 The deep source encoder in , the decoder corresponds to Figure 1 The data in the user decoder. Then for the A pre-trained model, , the error probability of the bit stream compressed by the encoder is The binary symmetric channel is then input into the decoder for image recovery. , bit error probability End-to-end distortion of data users under Finally, we use the logistic function to fit and and The relationship is expressed as

[0045] (5)

[0046] in , , and Represents the logistic function fitting parameters of the data user.

[0047] The end-to-end distortion modeling of semantic users adopts the same approach. First, for the image source, consider the image classification task, using the image label as the semantic information and the pre-trained Based on a data compression neural network model, the encoder network parameters of the model are frozen, and a semantic information decoder is constructed to perform the image restoration task. The semantic decoder corresponds to Figure 1 Similarly, for the semantic user decoder A pre-trained model, , the error probability of the bit stream compressed by the encoder is The binary symmetric channel is then fed into the semantic information decoder for image classification. , bit error probability Semantic user end-to-end distortion under Finally, we use the logistic function to fit and and The relationship is expressed as

[0048] (6)

[0049] in , , and Represents the logistic function fitting parameters of semantic users.

[0050] Bit Error Rate Analysis

[0051] Now we introduce the bit error rate analysis of finite code length random coding. From formula (4), we can see that for a Finite code length random coding, the block error probability is determined by the channel coding rate, signal to interference noise ratio and code length. When a block error occurs, assuming that the bit error probability in each data packet is independent and identically distributed, the bit error rate can be estimated as

[0052] (7)

[0053] in represents the channel coding rate, Indicates the signal-to-noise ratio (SNR) / signal-to-interference-and-noise ratio (SI

[0054] S3. Perform joint optimization of source channel coding rate, power allocation, and beamforming.

[0055] Problem Construction

[0056] Based on the multi-user semantic communication system and end-to-end distortion modeling constructed above, we formulate the following optimization problem:

[0057] (8)

[0058] in Represents a user The end-to-end distortion weight of Represents a user Source compression rate; Represents a user The channel coding rate, Represents a user Power; Represents a user The beamforming vector of Indicates the total power of the system; Represents a user The maximum transmission delay of Indicates data users The end-to-end data distortion is calculated by substituting (7) and Replace with the corresponding user of and From this we get , and replace the formula (5) and Replace with and ; Representing semantic users The end-to-end semantic distortion of is calculated by transforming (7) and Replace with the corresponding user of and From this we get , and replace the formula (6) and Replace with and .

[0059] Solving optimization problems

[0060] To solve the optimization problem (8), we first relax the discrete constraint on the source rate in problem (8) and transform the problem into

[0061] (9)

[0062] For problem (9), the solution is to divide it into two sub-problems: rate allocation and power and beamforming joint optimization. By alternately solving these two sub-problems until convergence, the solution of problem (9) is completed. After obtaining the continuous source coding rate, the continuous result is quantized down to The closest source coding rate is obtained, and the solution of problem (8) is completed. The following describes the solution steps of these two sub-problems respectively.

[0063] Among them, rate allocation:

[0064] For a fixed set of power and beamforming , Represents the number of iterations of alternating solutions, and the signal to noise ratio of each user is also fixed. Since the source channel coding rate of each user has no effect on the end-to-end distortion of other users at this time, the problem of minimizing the weighted sum of end-to-end distortion is equivalent to the problem of independently allocating the source channel rate to each user. By further equating the delay constraint of each user, the rate allocation problem is simplified to an optimization problem of a one-dimensional continuous variable, which can be solved by the subgradient descent method to obtain the source channel rate result. .

[0065] Joint optimization of power and beamforming:

[0066] When the source channel coding rate of each user is fixed to hour, represents the number of iterations of alternating solutions. The joint optimization problem of power and beamforming is decomposed into a power allocation subproblem and a beamforming optimization subproblem. For the beamforming optimization problem, when the user power allocation is fixed, the uplink and downlink duality theory can prove that the optimal beamforming is the MMSE beamforming method. By converting the original downlink system into an equivalent uplink system, the optimal beamforming can be expressed as

[0067] (10)

[0068] in represents the MMSE beamforming vector, express The identity matrix, Indicates the upstream system to the user The power distribution, It is the channel parameter of the uplink system.

[0069] When the beamforming vector of each user is fixed, the original optimization problem (9) is simplified to:

[0070] (11)

[0071] in Indicates users in the upstream system By introducing intermediate variables, problem (11) is transformed into:

[0072] (12)

[0073] Among them hour, otherwise, ; ; ; ; is the intermediate variable introduced. By making a first-order Taylor approximation to the non-convex terms in problem (12), (12) can be transformed into a convex problem, and then the expansion point is iteratively updated to complete the solution of problem (12), and the power allocation result of the uplink system is obtained. .Will Substituting into formula (10) we can get the beamforming result of the uplink system: The power and beamforming results of the uplink system are converted into downlink system results through the uplink and downlink duality theory, and the source channel coding rate is obtained as The joint optimization results of power and beamforming when , and the weighted and end-to-end distortion of the system at this time .

[0074] The joint optimization of the source channel coding rate, power allocation, and beamforming adopts an alternating solution method, including:

[0075] (1) First, initialize the power allocation to uniform power allocation, that is, the power of all users is the same, initialize the beamforming to zero-forcing beamforming, initialize the source coding rate of all users to the minimum source coding rate among all pre-trained models, and initialize the channel coding rate to the channel capacity of each user at this time;

[0076] (2) In the nth iteration, the source channel coding rate of all users is fixed, and the power allocation and beamforming optimization methods are used to solve the power and beamforming joint optimization results. , and the weighted and end-to-end distortion of the system at this time ;

[0077] (3) Then based on , according to the channel coding rate optimization, the source channel coding rate used in the next iteration is obtained ;

[0078] (4) Let n = n + 1, and repeat steps (2) to (3); when the weighted sum end-to-end distortion The change value is less than the preset convergence threshold ,Right now , terminate the alternating optimization algorithm, and the convergence result at this time is the final source channel coding rate, power allocation and beamforming result.

[0079] In this embodiment, we use a classic hyper-prior model as the encoder and decoder for the data user, and a classic ResNet architecture for the semantic information recovery network for the semantic user. We tested the proposed method on the CUB-200-2011 image dataset. The source coding model was trained on the CUB-200-2011 dataset.

[0080] When making the lookup table, we use the Adam optimizer and to train the neural network model. The batch size is 16, for a total of 200 epochs, and the initial learning rate is set to , and decays by a factor of 0.1 while the loss function remains unchanged. The established lookup table contains 16 models, with an average compression bit value per pixel ranging from 0.012 to 1.36.

[0081] exist Figure 5In the experiments, the horizontal axis represents the average bandwidth rate, which is calculated by dividing the number of symbols sent after channel coding by the image dimensions, and the vertical axis represents the weighted and end-to-end distortion. As can be seen, compared with the zero-forcing beamforming + water filling + BPG + 2.0, 1.5 channel coding rate scheme and the zero-forcing beamforming + water filling + traditional deep source channel scheme, our scheme shows significant performance gains at different average bandwidth rates, indicating that our scheme can save more bandwidth while maintaining the same weighted and end-to-end distortion.

[0082] exist Figure 6 In the experiment, the horizontal axis is the total transmission power, and the vertical axis is the weighted sum of end-to-end distortion. From the figure, we can see that our method has significant performance gains compared with traditional deep source-channel coding methods and source-channel separation coding methods at different total transmission powers.

Claims

1. A multi-user source channel adaptive semantic coding method based on deep learning, characterized by: The following steps are involved: S1. Build a multi-user semantic communication system based on deep neural network: At the transmitting end, the signal to be transmitted is processed by the deep source encoder and the digital channel encoder, and then superimposed, numbered, and transmitted. At the receiving end, the received signal is processed by the digital channel decoder and then processed by the data user decoder and semantic user decoder to obtain the source data and semantic information; S2. End-to-end distortion modeling for data users and semantic users based on data regression; S3. Jointly optimize the source channel coding rate, power allocation, and beamforming; The step S1 comprises: S101. At the sending end, for the user , input data The length of the encoded signal is Bitstream ; Then, Divided into indivual Length of the data packet, where Indicates rounding up operation; For the data packets, using a channel coding rate of Channel encoder Encode it as a length The symbol sequence ,in Represents a set of complex numbers, and the transmitted symbols satisfy the unit power constraint ; use Represents a user The channel transmission symbol sequence is The symbols transmitted during secondary channel transmission use the unit beamforming vector and power All user transmission symbols are superimposed and coded, which can be expressed as: （1） in , Broadcast by the sender to all receivers for transmission; S302. At the receiving end, for the user , in The symbol received during secondary channel transmission is (2) in From the sender to the user The channel parameters, Indicates the number of antennas at the transmitting end, The mean is 0 and the variance is Independent and identically distributed circularly symmetric complex Gaussian noise, user The signal-to-interference-noise ratio is expressed as (3) The receiving end continues to receive channel symbols according to formula (2) until the complete received symbol sequence is obtained. , then use the channel decoder Will Decoded into the recovered bit sequence ; According to the finite code length random coding rate formula, the average packet error probability is (4) in represents the Gaussian Q function; In getting After that, for data users , use the data user decoder to Decoding to obtain the recovered source data ; For semantic users , using the semantic user decoder to Decoding to recover semantic information ; The end-to-end distortion of data users during transmission is expressed as ,in Represents a user All channel noise experienced, Representing the data distortion metric, the end-to-end distortion of semantic users is expressed as , represents the semantic distortion measure.

2. The deep learning-based multi-user source channel adaptive semantic coding method according to claim 1, characterized in that: In step S1, for any user , consider an independent and identically distributed image source semantic pair ,in Represents the original image data, Represents the semantic information corresponding to the image data, and Represents the dimensions of image and semantic information respectively, image data Through the function Extracted dimensional continuous vector ; is the function of the neural network, and the neural network parameters are ; Continuous eigenvectors Further through the quantization module, the discrete semantic feature vector is obtained , and then compressed using arithmetic coding to a length of Bitstream , the average source coding rate of the source encoder is expressed as .

3. The deep learning-based multi-user source channel adaptive semantic coding method according to claim 1, characterized in that: In step S1, two types of users are considered: data users whose purpose is to restore source data, whose index is , is the number of data users, and the semantic users who perform downstream machine tasks through semantic information, whose index is , is the number of semantic users, Indicates the total number of users in the system, Represents the set of all users; Design solutions for constructing source decoders for two types of users: First, for data users, that is, , the received bit stream First, the recovered semantic feature vector is obtained through the arithmetic decoder , and then use the source data recovery function according to Get the recovered source data ,in is the function of the neural network, are neural network parameters; For semantic users, i.e. , similarly, the received bit stream Use arithmetic decoding to get semantic feature vector , Unlike data users, semantic users use semantic information to recover functions according to Performing semantic tasks users recover semantic information ,in is the function of the neural network, are the neural network parameters.

4. The method for multi-user source channel adaptive semantic coding based on deep learning according to claim 1, characterized in that: The step S2 comprises: S201. End-to-end distortion modeling for data users: For the image source, the source coding rate is arranged from small to large as follows: Next training A data compression neural network model, each model consists of an encoder and a decoder, the encoder is the depth source encoder, and the decoder is the data user decoder; For the A pre-trained model, , the error probability of the bit stream compressed by the encoder is The binary symmetric channel is then input into the decoder for image restoration. The source coding rate is obtained from this statistic. , bit error probability End-to-end distortion of data users under ,Finally, we use the logistic function to fit and and The relationship is expressed as (5) in , , and Represents the logistic function fitting parameters of the data user; S202. End-to-end distortion modeling for semantic users: For image sources, consider image classification tasks, use image labels as semantic information, and use pre-trained Based on a data compression neural network model, the encoder network parameters of the model are frozen, and a semantic information decoder is constructed to perform the image restoration task. The semantic decoder is also called the semantic user decoder. For the A pre-trained model, , the error probability of the bit stream compressed by the encoder is The binary symmetric channel is then fed into the semantic information decoder for image classification. The source coding rate is obtained from this statistic. , bit error probability Semantic user end-to-end distortion under ; Use logistic function fitting and and The relationship is expressed as (6) in , , and Represents the logistic function fitting parameters of semantic users.

5. The multi-user source channel adaptive semantic coding method based on deep learning according to claim 1, characterized in that: In step S2, for a In the case of finite code length random coding, the block error probability is determined by the channel coding rate, signal-to-interference-and-noise ratio, and code length. When a block error occurs, assuming that the bit error probability in each data packet is independent and identically distributed, the bit error rate is estimated as: (7) in represents the channel coding rate, Indicates the signal-to-noise ratio (SNR) / signal-to-interference-and-noise ratio (SINR).

6. The multi-user source channel adaptive semantic coding method based on deep learning according to claim 1, characterized in that: The step S3 comprises: S301. Based on the multi-user semantic communication system and end-to-end distortion modeling, the optimization problem is formulated as follows: (8) in Represents a user The end-to-end distortion weight of Represents a user Source compression rate; Represents a user The channel coding rate, Represents a user Power; Represents a user The beamforming vector of Indicates the total power of the system; Represents a user The maximum transmission delay of Indicates data users The end-to-end data distortion is calculated by substituting (7) and Replace with the corresponding user of and From this we get , and replace the formula (5) and Replace with and ; Representing semantic users The end-to-end semantic distortion of is calculated by transforming (7) and Replace with the corresponding user of and From this we get , and replace the formula (6) and Replace with and ; S302. When solving the optimization problem (8), first relax the discrete constraint on the source rate in problem (8) and transform the problem into (9) For problem (9), the solution is as follows: divide it into two sub-problems: rate allocation and power and beamforming joint optimization; solve these two sub-problems alternately until convergence, and then complete the solution of problem (9); after obtaining the continuous source coding rate, quantize the continuous result down to The closest source coding rate is obtained, and the solution to problem (8) is completed.

7. The multi-user source channel adaptive semantic coding method based on deep learning according to claim 6, characterized in that: The channel coding rate optimization includes: For a fixed set of power and beamforming , Represents the number of iterations of alternating solutions, and the signal to noise ratio of each user is also fixed. Since the source channel coding rate of each user has no effect on the end-to-end distortion of other users at this time, the problem of minimizing the weighted sum and end-to-end distortion is equivalent to the problem of independently allocating the source channel rate to each user. By further equating the delay constraint of each user, the rate allocation problem is simplified to an optimization problem of a one-dimensional continuous variable, which is solved by the subgradient descent method to obtain the source channel rate result. .

8. The multi-user source channel adaptive semantic coding method based on deep learning according to claim 7, characterized in that: The power allocation and beamforming optimization include: When the source channel coding rate of each user is fixed to hour, represents the number of iterations of alternating solutions. The joint optimization problem of power and beamforming is decomposed into a power allocation subproblem and a beamforming optimization subproblem; For the beamforming optimization problem, when the user power allocation is fixed, the optimal beamforming method is the MMSE beamforming method based on the uplink and downlink duality theory. By converting the original downlink system into an equivalent uplink system, the optimal beamforming is expressed as: (10) in represents the MMSE beamforming vector, express The identity matrix, Indicates the upstream system to the user The power distribution, is the channel parameter of the uplink system; When the beamforming vector of each user is fixed, the original optimization problem (9) is simplified to: (11) in Indicates users in the upstream system By introducing the intermediate variable, problem (11) is transformed into: (12) Among them hour, otherwise, ; ; ; ; As the intermediate variable introduced, the non-convex terms in problem (12) are converted into a convex problem by making a first-order Taylor approximation, and then the expansion point is iteratively updated to complete the solution of problem (12), and the power allocation result of the uplink system is obtained. ,Will Substituting into formula (10) we can get the beamforming result of the uplink system: , the power and beamforming results of the uplink system are converted into downlink system results through the uplink and downlink duality theory, and the source channel coding rate is obtained as The joint optimization results of power and beamforming when , and the weighted and end-to-end distortion of the system at this time .

9. The multi-user source channel adaptive semantic coding method based on deep learning according to claim 8, characterized in that: The joint optimization of source channel coding rate, power allocation, and beamforming includes: (1) First, initialize the power allocation to uniform power allocation, that is, the power of all users is the same, initialize the beamforming to zero-forcing beamforming, initialize the source coding rate of all users to the minimum source coding rate among all pre-trained models, and initialize the channel coding rate to the channel capacity of each user at this time; (2) In the nth iteration, the source channel coding rate of all users is fixed, and the power allocation and beamforming optimization methods are used to solve the power and beamforming joint optimization results. , and the weighted and end-to-end distortion of the system at this time ; (3) Then based on , according to the channel coding rate optimization, the source channel coding rate used in the next iteration is obtained ; (4) Let n = n + 1, and repeat steps (2) to (3); when the weighted sum end-to-end distortion The change value is less than the preset convergence threshold ,Right now , terminate the alternating optimization algorithm, and the convergence result at this time is the final source channel coding rate, power allocation and beamforming result.

Images