Data and knowledge driven end-to-end wireless communication system model design method
By using Bayesian neural networks and knowledge-driven methods in an end-to-end wireless communication system, the problem of few-sample training is solved, the transmission efficiency and bit error rate performance of the system are improved, and better global optimization results are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2023-06-26
- Publication Date
- 2026-06-23
AI Technical Summary
Existing end-to-end wireless communication systems do not perform well under small sample conditions and face problems such as insufficient sample quantity and partial missing data sets, which affect the performance and generalization ability of neural networks and make it difficult to construct the global optimal solution.
We replace convolutional neural networks with Bayesian neural networks, combining data-driven and knowledge-driven approaches. We utilize Bayesian layers and power normalization layers, optimize model parameters through Bayesian inference and gradient descent, and train the model using both theoretical and real channel models.
It significantly improves the transmission efficiency and performance of end-to-end wireless communication systems under small sample conditions, and achieves better generalization ability and bit error rate optimization.
Smart Images
Figure CN116667958B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of communications, and is particularly for optimization scenarios of end-to-end wireless communication systems. Background Technology
[0002] 5G, with its wide bandwidth and low latency, has already brought about tremendous changes to many industries. However, the development of emerging technologies such as telemedicine, connected cities, intelligent connected vehicles, and virtual reality has placed higher demands on the capacity, speed, and latency of communication systems, and existing 5G solutions are clearly insufficient to meet these growing needs. Therefore, the development of a new sixth-generation mobile communication technology (6G) has been put on the agenda. Since 6G technology requires greater bandwidth to achieve higher transmission rates, and real-world frequency band resources are not unlimited, efforts are being made to compensate for resource shortages by improving the efficiency of communication systems. One promising solution is an end-to-end wireless communication system that combines data-driven and knowledge-driven approaches.
[0003] Traditional end-to-end wireless communication system models often only allow for optimization of local modules, failing to achieve the global optimal solution. Conversely, end-to-end wireless communication system models based on convolutional neural networks do not perform well under limited sample conditions.
[0004] In the field of communications, training neural networks often faces two problems: insufficient sample size and partially missing datasets. These problems affect the performance and generalization ability of neural networks, making it difficult to build end-to-end wireless communication systems. Summary of the Invention
[0005] The present invention aims to provide an end-to-end wireless communication system model suitable for small sample training, so as to improve the transmission efficiency of the entire communication system.
[0006] The end-to-end wireless communication system model suitable for few-shot training provided by this invention includes the following steps:
[0007] 1) Obtain communication system parameters for algorithm training.
[0008] 2) Establish an end-to-end wireless communication system model framework: Use Bayesian layers and power normalization layers in neural networks to replace the transmitter and receiver in traditional communication systems, and use theoretical channel models and real channel models in wireless communication as transmission media.
[0009] 3) Update the variational parameters of the Bayesian inference.
[0010] 4) Update the network weights of the Bayesian layer: Update the deep learning network parameters using gradient descent.
[0011] 5) Determine if the optimization termination condition is met: After the system outputs the predicted value, calculate the binary cross-entropy loss function with the tag and observe whether the bit error rate requirement of the end-to-end wireless communication system is met. If not, return to step 3) to proceed to the next iteration. Attached Figure Description
[0012] Figure 1 The flowchart of the method of the present invention
[0013] Figure 2 This is the end-to-end wireless communication system model of the present invention.
[0014] Figure 3 This is a structural diagram of the Bayesian neural network used in this invention.
[0015] Figure 4 This is a performance comparison chart of the present invention and the traditional model in a Gaussian channel.
[0016] Figure 5 This is a performance comparison chart between the present invention and the traditional model in the Rayleigh channel.
[0017] Figure 6 This is a performance comparison chart of the present invention and existing models in Gaussian channels under small sample conditions.
[0018] Figure 7 This is a performance comparison chart of the present invention and existing models in Rayleigh channels under small sample conditions.
[0019] Figure 8 This is a performance comparison chart of the present invention and existing models in MIMO channels under small sample conditions.
[0020] Figure 9 A comparison of the losses of Bayesian neural networks and convolutional neural networks under small sample conditions. Detailed Implementation
[0021] In end-to-end wireless communication scenarios, the communication dataset samples available for neural network training are relatively small. Therefore, it is necessary to introduce expert knowledge from the communication field as prior knowledge through Bayesian neural networks to assist model training and achieve better end-to-end transmission performance. This invention utilizes Bayesian inference to address the difficulty of calculating the posterior distribution in Bayesian networks. It proposes a data-driven and knowledge-driven end-to-end wireless communication system model, replacing the convolutional neural network in existing models with a Bayesian neural network, thereby enabling the system to achieve good end-to-end performance even under training conditions with small samples.
[0022] The implementation process of the model proposed in this invention is as follows:
[0023] 1. Obtain communication system parameters for algorithm training, including the number of effective information bits. Number of channel time slots Modulated base number Channel signal-to-noise ratio and channel state information for each time slot. Parameters such as (i=1,2,…,n) are used for training deep learning models.
[0024] 2. Establish Figure 2 The end-to-end wireless communication system model framework is shown; two Bayesian layers and one power normalization layer are used as the transmitter model f(S), and two Bayesian layers are used as the receiver model g(S). A theoretical channel model or a channel model built from real channel state information is used. The transmitting end model, channel model, and receiving end model need to maintain consistency across the network dimension. Both the input and output of the models use one-hot vectors to pre-encode the binary data transmitted by the communication system, a common technique in deep learning.
[0025] 3. Update the variational parameters of the Bayesian inference;
[0026] (1) Choosing the Gaussian distribution as the prior distribution simplifies the problem in actual calculation, that is:
[0027] .
[0028] In the above formula For the prior distribution of the variational distribution, It follows a Gaussian distribution. and These are the mean and variance of the distribution, respectively. This refers to network weights.
[0029] (2) The difference between the variational distribution q and the posterior distribution p, i.e., the KL divergence, is expressed as:
[0030] .
[0031] (3) Define the objective function for parameter optimization as the likelihood function of the variational distribution q:
[0032] .
[0033] The objective function in the above formula It consists of two parts: and .in, This is the log-likelihood function of the observed data, representing the probability of the observed data occurring given the variational parameter θ. Generally, it is desirable to maximize this probability to make the model fit the observed data better. It is a variational distribution With posterior distribution The KL divergence between them.
[0034] (4) By maximizing To find the joint probability distribution of the observed data X and Y The maximized variational parameter θ, i.e.:
[0035]
[0036] The above formula uses the idea of maximum likelihood estimation, a commonly used parameter estimation method. Maximum likelihood estimation is used to find the parameter values that maximize the probability of the observed data, thereby obtaining the variational parameter estimates that make the model perform optimally on the given observed data. .
[0037] 4. Update Figure 3 The weights of the Bayesian neural network in the diagram are: X is the network input layer, H is the network hidden layer, and Y is the network output layer. and These represent the mean and variance of the probability distribution followed by the network weights, respectively. Regularization objects are created for the transmitter and receiver models, regularized using the L2 norm. Adam is used as the gradient descent optimizer, and the gradient descent process can be represented as:
[0038]
[0039]
[0040] In the above formula, The output value of the neural network. For data label values, The number of samples in a batch For loss function, and These represent the weights of the Bayesian neural network in the transmitter and receiver, respectively.
[0041] 5. Determine if the optimization termination condition is met; after the system outputs the predicted value, calculate the binary classification cross-entropy loss function with the tag, and observe whether the bit error rate requirement of the end-to-end wireless communication system is met. If not, return to step 3) for the next iteration.
[0042] 6. Simulation was performed on the model proposed in this invention, and the results were compared with traditional end-to-end wireless communication system models and existing system models based on convolutional neural networks. The results are shown in the appendix. Figures 4 to 9 As shown, it is proven that the proposed algorithm can indeed significantly improve the end-to-end performance of wireless communication systems under small sample training conditions.
[0043] The model parameters of the communication system trained in the simulation are shown in the table below.
[0044]
[0045] The foregoing description illustrates and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.
Claims
1. A data- and knowledge-driven end-to-end wireless communication system model design method, characterized in that, To jointly optimize modulation / demodulation, encoding / decoding, and channel transmission, the following steps are included: 1) Obtain communication system parameters for algorithm training; 2) Establish an end-to-end wireless communication system model framework: The transmitter and receiver of the traditional communication system are replaced by Bayesian layers and power normalization layers in the neural network, and the transmission medium is selected from the theoretical channel model and the real channel model in wireless communication. 3) Update the variational parameters of the Bayesian inference: Establish the prior distribution of variational parameters in Bayesian inference, use KL divergence to represent the difference between the variational distribution and the posterior distribution, use the likelihood function as the objective function for parameter optimization, and determine the variational parameters that maximize the objective function. 4) Update the network weights of the Bayesian layer: Update the network weights of the Bayesian layer using gradient descent. 5) Determine if the optimization termination condition is met: After the system outputs the predicted value, calculate the binary classification cross-entropy loss function with the tag, and observe whether the bit error rate requirement of the end-to-end wireless communication system is met. If it is not met, return to step 3) for the next iteration. Step 2) establishes the system model framework; Two Bayesian layers and one power normalization layer are used as the transmitter model of the system, and two Bayesian layers are used as the receiver model of the system. In step 4), the weights of the deep learning network are updated; Create regularization objects for the transmitter and receiver models, perform regularization using the L2 norm, and use Adam as the gradient descent optimizer. The gradient descent process is represented as follows: In the above formula, The output value of the neural network. For data label values, The number of samples in a batch For loss function, and The network weights are for the transmitter and receiver, respectively.
2. The method as described in claim 1, characterized in that: Step 1) involves obtaining communication system parameters for algorithm training, including the number of effective information bits. Number of channel time slots Modulated base number Channel signal-to-noise ratio and channel state information for each time slot. (i=1,2,…,n) parameters.
3. The method as described in claim 1, characterized in that: In step 3), a Gaussian distribution is chosen as the prior distribution to simplify the problem in actual calculation, that is: In the above formula For the prior distribution of the variational distribution, It follows a Gaussian distribution. and These are the mean and variance of the distribution, respectively. This refers to network weights.
4. The method as described in claim 1, characterized in that: Step 3) uses KL divergence to measure the difference between the variational distribution q and the posterior distribution p in Bayesian inference: In the above formula For network weights, For variational parameters, and These are the network's input and output, respectively.
5. The method as described in claim 1, characterized in that: In step 3), the objective function for parameter optimization is defined as the likelihood function of the variational distribution q. : 。 6. The method as described in claim 1, characterized in that: In step 3), by maximizing To find the joint probability distribution of the observed data X and Y The maximized variational parameter θ, i.e.: 。 7. The method as described in claim 1, characterized in that: In step 5), determine whether the optimization termination condition is met; After the system outputs the predicted value, the loss function of the tag is calculated, and it is observed whether the bit error rate requirement of the end-to-end wireless communication system is met. If it is not met, return to step 3) for the next iteration.