Multi-user source channel adaptive semantic coding method based on deep learning

Abstract

The invention discloses a multi-user source channel adaptive semantic coding method based on deep learning, and the method comprises the following steps: S1, constructing a multi-user semantic communication system based on a deep neural network: at a transmitting end, processing a to-be-transmitted signal through a deep source encoder and a digital channel encoder, and transmitting the to-be-transmitted signal through a digital channel encoder; carrying out superposition numbering and signal transmission; at a receiving end, the received signal is processed by a digital channel decoder and then is processed by a data user decoder and a semantic user decoder to obtain information source data and semantic information; s2, performing end-to-end distortion modeling of the data user and the semantic user based on data regression; and S3, carrying out joint optimization on the source channel coding rate, the power distribution and the beam forming. According to the method, a multi-user broadcast channel is considered, and an information source channel rate, power distribution and beamforming joint optimization algorithm is designed, so that weighting and end-to-end distortion of a multi-user system are reduced to the maximum extent.

Description

Technical Field

[0001] The present invention relates to the field of semantic communication, and particularly to a multi-user source-channel adaptive semantic coding method based on deep learning. Background Art

[0002] Facing the increasing data transmission requirements in future communication systems, compared with the fifth-generation mobile network, the sixth-generation mobile network needs to achieve higher transmission efficiency under limited spectrum resources. Semantic communication can significantly reduce the communication resource overhead while achieving the same performance by using artificial intelligence technology to extract the semantic features of the original data.

[0003] Although the existing semantic communication technologies have better performance than traditional communication technologies, they usually directly send the analog semantic features extracted by the neural network as wireless signals and cannot be compatible with digital communication systems. At the same time, most semantic communication schemes only consider the point-to-point transmission scenario and are customized according to the source data and channel parameters, making it difficult to expand to multi-user scenarios. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a multi-user source-channel adaptive semantic coding method based on deep learning, which considers the multi-user broadcast channel and designs a joint optimization algorithm for source-channel rate, power allocation, and beamforming to minimize the weighted sum end-to-end distortion of the multi-user system to the greatest extent.

[0005] The purpose of the present invention is achieved by the following technical solutions: A multi-user source-channel adaptive semantic coding method based on deep learning, comprising the following steps: S1. Construct a multi-user semantic communication system based on a deep neural network: At the sending end, the signal to be sent is processed by a deep source encoder and a digital channel encoder, and then superposition coding and signal transmission are performed; At the receiving end, the received signal is processed by a digital channel decoder, and then the source data and semantic information are obtained by processing with a data user decoder and a semantic user decoder; S2. Perform end-to-end distortion modeling for data users and semantic users based on data regression; S3. Perform joint optimization of source-channel coding rate, power allocation, and beamforming.

[0006] The beneficial effects of the present invention are: The present invention considers the multi-user broadcast channel and designs a joint optimization algorithm for source-channel rate, power allocation, and beamforming to minimize the weighted sum end-to-end distortion of the multi-user system to the greatest extent. Description of the Drawings

[0007] Figure 1Schematic diagram of the principle of the present invention; Figure 2 Schematic diagram of the principle of the source codec based on the deep neural network; Figure 3 Schematic diagram of information recovery of the data user; Figure 4 Schematic diagram of information recovery of the semantic user; Figure 5 Schematic diagram of comparison of weighted sum end-to-end distortion under different average bandwidth rates; Figure 6 Schematic diagram of comparison of weighted sum end-to-end distortion under different transmission powers. Detailed implementation manners

[0008] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0009] A multi-user source-channel adaptive semantic coding method based on deep learning includes the following steps: S1. Construct a multi-user semantic communication system based on a deep neural network: At the sending end, after the signal to be sent is processed by the deep source encoder and the digital channel encoder, superposition coding and signal transmission are performed; At the receiving end, after the received signal is processed by the digital channel decoder, the source data and semantic information are obtained by processing with the data user decoder and the semantic user decoder; First, the deep source encoder is introduced. Figure 2 The structure of the source encoder based on the deep neural network is shown. For any user , consider an independent and identically distributed image source semantic pair , where represents the original image data, represents the semantic information corresponding to the image data, and represent the dimensions of the image and semantic information respectively. The image data is processed through the function to obtain -dimensional continuous vector . is a function composed of a neural network, and the neural network parameters are . The continuous feature vector is further processed through a quantization module to obtain a discrete semantic feature vector , and then it is compressed into a bit stream of length using arithmetic coding. The average source coding rate of the source encoder is expressed as .

[0010] Introducing the design of the deep source decoder: Consider two types of users: data users who aim to restore source data, whose index is , is the number of data users, and the semantic users who perform downstream machine tasks through semantic information, whose index is , is the number of semantic users, Represents the total number of users in the system, represents the set of all users. The following introduces the design schemes of the source decoder for two types of users. First, for the data user, that is, ,like Figure 3 As shown, the received bit stream First, the semantic feature vector is restored by the arithmetic decoder , and then use the source data recovery function according to Get the recovered source data ,in is the function of the neural network. is the neural network parameter. For semantic users, , similarly, if Figure 4 As shown, the received bit stream Use arithmetic decoding to get semantic feature vector , unlike data users, semantic users use semantic information to recover functions according to Performing semantic tasks users recover semantic information ,in is the function of the neural network. are neural network parameters.

[0011] Channel Coding: For Users , consider any Block code, composed of a channel encoder and a channel decoder Composition, of which represents a complex set, Indicates the data bit length, represents the block length. The channel coding rate can be expressed as .

[0012] Based on the above coding scheme, the overall transmission process of the considered multi-user semantic communication system is as follows.

[0013] On the sending side, for users , input data Through Figure 1 the deep source encoder in it is encoded into a bitstream of length ... . Then, it is divided into packets of length where denotes the ceiling operation. For the -th packet, a channel encoder with a channel coding rate of is used to encode it into a symbol sequence of length ... The transmitted symbols satisfy the unit power constraint . .

[0014] Using to represent the symbol transmitted during the -th channel transmission of the channel transmission symbol sequence of user , and using the unit beamforming vector and power to perform superposition coding on all user transmission symbols, which is expressed as (1) where . It is broadcast from the transmitter to all receivers for transmission.

[0015] At the receiver, for user , the symbol received during the -th channel transmission is (2) where represents the channel parameter from the transmitter to user , represents the number of antennas at the transmitter, represents independent and identically distributed circularly symmetric complex Gaussian (CSCG) noise with a mean of 0 and a variance of . So the signal-to-interference-plus-noise ratio (SINR) of user can be expressed as (3) The receiver continuously receives the channel symbols according to formula (2) until the complete received symbol sequence is obtained, and then uses the channel decoder to decode into the recovered bit sequence . According to the random coding rate formula for finite block lengths, the average packet error probability is (4) where represents the Gaussian Q function.

[0016] After obtaining as shown Figure 1 for the data user , use the data user decoder to decode to obtain the restored source data . For the semantic user , use the semantic user decoder to decode to obtain the restored semantic information . The end-to-end distortion of the data user during transmission is expressed as where represents all the channel noises experienced by user , and represents the data distortion metric. The end-to-end distortion of the semantic user is expressed as , represents the semantic distortion metric.

[0017] S2. End-to-end distortion modeling for data users and semantic users based on data regression; Now introduce the end-to-end distortion and modeling methods for data users and semantic users. The introduction is divided into end-to-end distortion modeling based on data regression and bit error rate analysis.

[0018] End-to-end distortion modeling based on data regression First, introduce the end-to-end distortion modeling method for data users. For an image source, train data compression neural network models arranged in ascending order of source coding rate , each model consists of an encoder and a decoder, the encoder corresponds to the deep source encoder in Figure 1 , and the decoder corresponds to the data user decoder in Figure 1 . Then for the th pre-trained model , , pass the bitstream compressed by the encoder through a binary symmetric channel with an error probability of and then input the obtained bitstream into the decoder for image restoration. Thus, statistically obtain the end-to-end distortion of the data user at the source coding rate and the bit error probability Finally, we use the logistic function to fit and The relationship is expressed as (5) where , , and represent the logistic function fitting parameters of the data user.

[0019] For the end - to - end distortion modeling of semantic users, the same method is adopted. First, for the image source, considering the image classification task, using the image label as semantic information, based on the pre - trained data compression neural network models, freezing the encoder network parameters of the model, constructing a semantic information decoder to perform the image restoration task, and the semantic decoder corresponds to the Figure 1 semantic user decoder in . Similarly, for the th pre - trained model, , passing the bitstream compressed by the encoder through a binary symmetric channel with an error probability of , and then inputting the obtained bitstream into the semantic information decoder for image classification. Thus, the end - to - end distortion of the semantic user under the source coding rate and the bit error probability is statistically obtained. Finally, we use the logistic function to fit the relationship between and , which is expressed as (6) where , , and represent the logistic function fitting parameters of the semantic user.

[0020] Bit Error Rate Analysis Now, the bit error rate analysis of finite - length random coding is introduced. As can be seen from formula (4), for a finite - length random coding, the block error probability is jointly determined by the channel coding rate, signal - to - interference - plus - noise ratio, and code length. When a block error occurs, assuming that the bit error probabilities in each data packet are independent and identically distributed, then the bit error rate can be estimated as (7) where represents the channel coding rate, represents the signal - to - noise ratio / signal - to - interference - plus - noise ratio.

[0021] S3. Perform joint optimization of source - channel coding rate, power allocation, and beamforming.

[0022] Problem formulation Based on the multi-user semantic communication system and the end-to-end distortion modeling constructed above, the optimization problem we constructed is as follows: (8) where denotes the end-to-end distortion weight of user ; denotes the source compression rate of user ; denotes the channel coding rate of user ; denotes the power of user ; denotes the beamforming vector of user ; denotes the total power of the system; denotes the maximum transmission delay of user ; denotes the end-to-end data distortion of data user , which is calculated by substituting and in (7) with the corresponding and of user to obtain , and then substituting and in formula (5) with and ; denotes the end-to-end semantic distortion of semantic user , which is calculated by substituting and in (7) with the corresponding and of user to obtain , and then substituting and in formula (6) with and .

[0023] Solution to the optimization problem For the solution to the optimization problem (8). First, relax the discrete constraint on the source rate in problem (8) and transform the problem into (9) For problem (9), the solution idea is as follows: Divide it into two sub-problems: rate allocation and joint optimization of power and beamforming. Solve these two sub-problems alternately until convergence, and then complete the solution to problem (9). After obtaining the continuous source coding rate, quantize the continuous result downward to The closest source coding rate, and then solve problem (8). The solution steps for these two sub-problems are introduced separately below.

[0024] Among them, rate allocation: For a fixed set of power and beamforming , represents the number of iterations of alternating solutions. The signal-to-interference-plus-noise ratio (SINR) of each user is also fixed accordingly. Since the source-channel coding rate of each user has no impact on the end-to-end distortion of other users at this time, the problem of minimizing the weighted sum of end-to-end distortion is equivalent to the problem of independently allocating source-channel rates for each user. By further setting the delay constraint of each user to an equality, the rate allocation problem is simplified to an optimization problem with one-dimensional continuous variables, which can be solved by the subgradient descent method to obtain the source-channel rate result .

[0025] Joint optimization of power and beamforming: When the source-channel coding rate of each user is fixed at , represents the number of iterations of alternating solutions. The joint optimization problem of power and beamforming is decomposed into a power allocation sub-problem and a beamforming optimization sub-problem. For the beamforming optimization problem, when the user power allocation is fixed, it can be proved by the uplink-downlink duality theory that the optimal beamforming is the MMSE beamforming method. By converting the original downlink system into an equivalent uplink system, the optimal beamforming can be expressed as (10) where represents the MMSE beamforming vector, represents the identity matrix of represents the power allocation of the uplink system to user , is the channel parameter of the uplink system.

[0026] When the beamforming vector of each user is fixed, the original optimization problem (9) is simplified to: (11) where represents the bit error rate of user in the uplink system. By introducing intermediate variables, problem (11) is transformed into: (12) where when , otherwise, ; ; ; ; is the introduced intermediate variable. By performing a first-order Taylor approximation on the non-convex term in equation (12), equation (12) can be transformed into a convex problem, and then the expansion point is iteratively updated to solve equation (12), obtaining the power allocation result of the uplink system . Substituting into formula (10) gives the beamforming result of the uplink system . Through the uplink-downlink duality theory, the power and beamforming results of the uplink system are transformed into the downlink system results, obtaining the power and beamforming joint optimization result when the source-channel coding rate is , as well as the weighted sum end-to-end distortion of the system at this time . .

[0027] The joint optimization of the source-channel coding rate, power allocation, and beamforming is performed in an alternating solution manner, including: (1) First, initialize the power allocation as uniform power allocation, that is, the power of all users is the same, initialize the beamforming as zero-forcing beamforming, initialize the source coding rate of all users as the minimum source coding rate among all pre-trained models, and initialize the channel coding rate as the channel capacity of each user at this time; (2) In the nth iteration process, fix the source-channel coding rate of all users, and solve the power and beamforming joint optimization result , as well as the weighted sum end-to-end distortion of the system at this time ; (3) Then, based on , optimize the source-channel coding rate for the next iteration according to the channel coding rate ; (4) Let n = n + 1, and repeat steps (2) to (3); when the change value of the weighted sum end-to-end distortion is less than the pre-set convergence threshold , that is, , terminate the alternating optimization algorithm, and the convergence result at this time is the finally obtained source-channel coding rate, power allocation, and beamforming results.

[0028] In the embodiments of this application, we use the classic hyper-prior model as the encoder and decoder of the data user, and the semantic information recovery network of the semantic user adopts the classic ResNet architecture. We tested the proposed method on the CUB-200-2011 image dataset. The training dataset of the source coding model is the training dataset of CUB-200-2011.

[0029] When creating the lookup table, we use the Adam optimizer to train the neural network model. The batch size is 16, and there are 200 epochs in total. The initial learning rate is set to , and it decays by a factor of 0.1 when the loss function remains unchanged. The established lookup table contains 16 models, and the average compressed bit value per pixel ranges from 0.012 to 1.36.

[0030] In Figure 5 's experiment, the abscissa is the average bandwidth rate, which is obtained by dividing the number of symbols transmitted after channel coding by the image dimension, and the ordinate is the weighted sum end-to-end distortion. It can be seen that compared with the zero-forcing beamforming + waterfilling + BPG + 2.0, 1.5 channel coding rate scheme and the zero-forcing beamforming + waterfilling + traditional deep source-channel method, our scheme shows significant performance gains at different average bandwidth rates, indicating that the proposed scheme in this paper can save more bandwidth while maintaining the same weighted sum end-to-end distortion.

[0031] In Figure 6 's experiment, where the abscissa is the total transmission power and the ordinate is the weighted sum end-to-end distortion. From the figure, we can see that compared with the traditional deep source-channel coding method and the source-channel separation coding method, our method has significant performance gains at different total transmission powers.

Claims

1. A multi-user source-channel adaptive semantic coding method based on deep learning, characterized in that: It includes the following steps: S1. Construct a multi-user semantic communication system based on a deep neural network: At the sending end, after processing the signal to be sent through a deep source encoder and a digital channel encoder, superposition coding and signal transmission are performed; At the receiving end, after processing the received signal through a digital channel decoder, the source data and semantic information are obtained through processing by a data user decoder and a semantic user decoder; S2. Perform end-to-end distortion modeling for data users and semantic users based on data regression; S3. Jointly optimize the source-channel coding rate, power allocation, and beamforming.

2. The multi-user source-channel adaptive semantic coding method based on deep learning according to claim 1, wherein: The step S1 includes: S101. At the sending end, for the user , the input data is encoded by the deep source encoder into a bitstream with a length of ; ; Then, is divided into packets of length , where denotes the ceiling operation; For the th data packet, use a channel encoder with a channel coding rate of to encode it into a symbol sequence of length where represents the set of complex numbers, and the transmitted symbols satisfy the unit power constraint ; ​​ Usage Indicates the symbol transmitted during the -th channel transmission of the channel transmission symbol sequence of the user, and uses the unit beamforming vector and power to perform superposition coding on all user transmission symbols, expressed as: ​ （1） Among them , is broadcast from the sending end to all receiving ends for transmission; S302. At the receiving end, for the user , the symbol received during the th channel transmission is （2） Among them represents the channel parameter from the sending end to the user of represents the number of antennas at the sending end represents independent and identically distributed circularly symmetric complex Gaussian noise with a mean of 0 and a variance of The signal-to-interference-plus-noise ratio of the user is expressed as （3） The receiving end continuously receives channel symbols according to formula (2) until a complete received symbol sequence is obtained , and then uses a channel decoder to decode it into a recovered bit sequence ; According to the random coding rate formula for finite block lengths, the average packet error probability is （4） wherein represents the Gaussian Q function; After obtaining for the data user use the data user decoder to decode to obtain the restored source data ; For a semantic user , use a semantic user decoder to decode and obtain the restored semantic information ; The end-to-end distortion of data users during transmission is expressed as ,in Indicates user All channel noise experienced, Representing the data distortion metric, the end-to-end distortion of semantic users is expressed as , Represents the semantic distortion measure.

3. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 2, characterized in that: In the step S1, for any user , consider an independent and identically distributed image source semantic pair , where represents the original image data, represents the semantic information corresponding to the image data, and represent the dimensions of the image and semantic information respectively; the image data is extracted through the function to obtain dimensional continuous vector ; is a function composed of a neural network, and the neural network parameters are ; Continuous feature vector Further pass through a quantization module to obtain a discrete semantic feature vector , and then use arithmetic coding to compress it into a bitstream with a length of The average source coding rate of the source encoder is expressed as . .

4. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 2, characterized in that: In the said step S1, two types of users are considered: data users aiming to restore source data, with their index being , being the number of data users, and semantic users performing downstream machine tasks through semantic information, with their index being , being the number of semantic users, representing the total number of users in the system, representing the set of all users; For two types of users, construct a design scheme for the source decoder: First, for the data user, namely , the received bitstream is first decoded by an arithmetic decoder to obtain the restored semantic feature vector , and then the source data recovery function is used to obtain the restored source data according to , where is a function composed of a neural network, and are the neural network parameters; For semantic users, namely , similarly, the received bitstream is arithmetically decoded to obtain a semantic feature vector . Different from data users, semantic users use a semantic information recovery function to execute a semantic task user to recover semantic information according to , where is a function composed of a neural network being the neural network parameters 5. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 1, characterized in that: The step S2 includes: S201. End-to-end distortion modeling for data users: For an image source, when the source coding rates are arranged from small to large as train data compression neural network models. Each model consists of an encoder and a decoder. The encoder is the deep source encoder, and the decoder is the data user decoder; For the th pre-trained model, , the bitstream compressed by the encoder is passed through a binary symmetric channel with an error probability of , and then the obtained bitstream is input into the decoder for image restoration; The end-to-end distortion of the data user under the source coding rate and the bit error probability is obtained through this statistics ; Finally, we use the logistic function to fit and and The relationship is expressed as （5） Among them , , and represent the logistic function fitting parameters of the data user; S202. End-to-end distortion modeling for semantic users: For an image source, considering the image classification task, using the image label as semantic information, based on the pre-trained data compression neural network models, freezing the encoder network parameters of the models, constructing a semantic information decoder to perform the image restoration task, and the semantic decoder is the semantic user decoder; For the th pre-trained model, the bitstream compressed by the encoder is passed through a binary symmetric channel with an error probability of , and then the obtained bitstream is input into the semantic information decoder for image classification; The semantic user end-to-end distortion under the source coding rate , bit error probability is obtained through this statistics ; Use the logistic function to fit and and The relationship is expressed as （6） Among them , , and represent the logistic function fitting parameters of the semantic user.

6. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 1, characterized in that: In the said step S2, for a random coding with finite code length, the block error probability is jointly determined by the channel coding rate, the signal-to-interference-plus-noise ratio, and the code length. When a block error occurs, assuming that the bit error probabilities in each data packet are independent and identically distributed, the bit error rate is estimated as follows: （7） wherein represents the channel coding rate, represents the signal-to-noise ratio / signal-to-interference-plus-noise ratio.

7. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 1, characterized in that: The step S3 includes: S301. Based on the multi-user semantic communication system and end-to-end distortion modeling, construct the following optimization problem: （8） Among them represents the end - to - end distortion weight of the user ; represents the source compression rate of the user ; represents the channel coding rate of the user ; represents the power of the user ; represents the beamforming vector of the user ; represents the total power of the system ; represents the maximum transmission delay of the user ; represents the end - to - end data distortion of the data user. Its calculation method is to replace and in (7) with the corresponding and of the corresponding user , and then replace and and in formula (5) with and ; represents the end - to - end semantic distortion of the semantic user. Its calculation method is to replace and in (7) with the corresponding and of the corresponding user and , and then replace and and in formula (6) with and ; When solving the optimization problem (8), first relax the discrete constraint on the source rate in problem (8), and transform the problem into （9） For problem (9), the solution idea is as follows: it is divided into two sub-problems: rate allocation and joint optimization of power and beamforming; by alternately solving these two sub-problems until convergence, and then completing the solution of problem (9); after obtaining the continuous source coding rate, the continuous result is quantized downward to the closest source coding rate in 8. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 7, characterized in that: The channel coding rate optimization includes: For a fixed set of power and beamforming , representing the number of iterations of alternating solutions, the signal-to-interference-plus-noise ratio (SINR) of each user is also fixed. Since the source-channel coding rate of each user has no impact on the end-to-end distortion of other users at this time, the problem of minimizing the weighted sum of end-to-end distortion is equivalent to the problem of source-channel rate allocation for each user independently. By further setting the delay constraint of each user to an equality, the rate allocation problem is simplified to an optimization problem of one-dimensional continuous variables, which is solved by the subgradient descent method to obtain the source-channel rate result .

9. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 8, characterized in that: The power allocation and beamforming optimization include: When the source-channel coding rate of each user is fixed at while denotes the number of iterations for alternating solutions, the joint optimization problem of power and beamforming is decomposed into a power allocation sub-problem and a beamforming optimization sub-problem; For the beamforming optimization problem, when the user power allocation is fixed, through the uplink-downlink duality theory, the optimal beamforming is the MMSE beamforming method. By converting the original downlink system into an equivalent uplink system, the optimal beamforming is expressed as: （10） Among them represents the MMSE beamforming vector, represents the identity matrix of, represents the power allocation of the uplink system to user ; is the channel parameter of the uplink system; When the beamforming vector of each user is fixed, the original optimization problem (9) is simplified to: (11) Among them represents the bit error rate of the user in the uplink system By introducing intermediate variables, problem (11) is transformed into: (12) Where when ; Otherwise, ; ; ; ; is the introduced intermediate variable. By performing a first-order Taylor approximation on the non-convex term in problem (12), (12) is transformed into a convex problem, and then the expansion point is iteratively updated to solve problem (12), obtaining the power allocation result of the uplink system . Substituting into formula (10) gives the beamforming result of the uplink system . Through the uplink-downlink duality theory, the power and beamforming results of the uplink system are transformed into the downlink system results, obtaining the source-channel coding rate of and the joint optimization result of power and beamforming at this time , as well as the weighted sum end-to-end distortion of the system at this time .

10. A multi-user source-channel adaptive semantic coding method based on deep learning according to claim 9, characterized in that: The joint optimization of the source-channel coding rate, power allocation, and beamforming includes: (1) First, initialize the power allocation as uniform power allocation, that is, the power of all users is the same, initialize the beamforming as zero-forcing beamforming, initialize the source coding rate of all users as the minimum source coding rate among all pre-trained models, and initialize the channel coding rate as the channel capacity of each user at this time; (2) During the n-th iteration, fix the source-channel coding rates of all users, and solve the joint optimization results of power and beamforming according to the power allocation and beamforming optimization methods , and the weighted sum end-to-end distortion of the system at this time ; (3) Then, based on , the source-channel coding rate for the next iteration is optimized according to the channel coding rate ; (4) Let \(n = n + 1\), and repeat steps (2) to (3); when the change value of the weighted end-to-end distortion is less than the pre-set convergence threshold , that is , terminate the alternating optimization algorithm, and the convergence result at this time is the source-channel coding rate, power allocation, and beamforming result finally obtained.

Images