Light-weight SC-IFDMA system channel estimation method based on knowledge distillation

Abstract

The invention relates to a lightweight SC-IFDMA system channel estimation method based on knowledge distillation, and the method comprises the steps: inputting a channel response matrix obtained through LS channel estimation into a trained deployed student network, and obtaining a global channel response matrix with higher estimation precision; wherein in the training stage of the knowledge distillation network, the teacher network assists in training the student network, and after a large number of training rounds, the channel estimation performance of the student network approaches the teacher network; and loading and deploying the training weight of the student network to obtain a deployed knowledge distillation network. Compared with a traditional LS estimation method, the method for carrying out explicit channel estimation on the SC-IFDMA system based on deep learning has the advantage that the channel estimation precision can be remarkably improved. According to the method, the thought of knowledge distillation is used for carrying out lightweight processing on the model, the parameter quantity of the model can be greatly reduced, the model deployment difficulty is reduced, and the estimation performance better than that of a common super-resolution model is obtained.

Description

Technical Field

[0001] The present invention relates to a channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation, belonging to the technical field of channel estimation. Background Art

[0002] The single-carrier interleaved frequency division multiple access (SC-IFDMA) system is an alternative to the orthogonal frequency division multiple access (OFDMA) system. While retaining the advantages of the OFDMA system in resisting time-frequency doubly selective fading, it can significantly reduce the peak-to-average power ratio of the system, thereby reducing the cost of the power amplifier at the transmitter end. Therefore, it has important application value in high-power long-distance communication.

[0003] The channel estimation module is an important part of the SC-IFDMA system. Channel estimation technology is a technology for accurately obtaining channel state information. An excellent channel estimation scheme helps to better understand the channel transmission characteristics, realize channel equalization, and then more accurately recover the transmitted signal.

[0004] Channel estimation in the SC-IFDMA system requires inserting frequency-domain pilot signals at the transmitter end. The pilot signals generally use known autocorrelation constant envelope sequences. At the receiver end, according to the change of the signal at the received pilot position relative to the transmitted pilot sequence, the channel characteristics can be predicted by means of channel estimation, and then corresponding equalization algorithms can be adopted to perform equalization processing on all received data.

[0005] Due to the relatively serious noise and inter-carrier interference in the SC-IFDMA system, the traditional LS channel estimation method has poor estimation performance in the SC-IFDMA system, while the MMSE method is difficult to implement in the SC-IFDMA system because it requires prior information and is difficult to implement. In recent years, with the rise of deep learning, more and more researchers use its powerful learning ability to use neural networks to learn channel characteristics and realize channel estimation. Channel estimation based on deep learning improves the prediction accuracy and reduces the pilot overhead at the same time. However, the existing deep learning-based channel estimation models have a relatively deep network depth and a large number of parameters, which are very difficult to deploy in actual engineering and will occupy a large amount of precious hardware resources. Summary of the Invention

[0006] Aiming at the deficiencies of the prior art, the present invention provides a channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation;

[0007] The present invention is implemented based on knowledge distillation in the field of deep learning. It uses a three-dimensional spatial attention mechanism to enhance the model's ability to extract channel characteristics, and performs upsampling and high-precision reconstruction on the LS channel estimation matrix, achieving higher channel estimation accuracy than existing deep learning models on models with very low parameter quantities.

[0008] The technical solution of the present invention is as follows:

[0009] A lightweight SC-IFDMA system channel estimation method based on knowledge distillation, including:

[0010] Input the channel response matrix obtained by LS channel estimation into the trained and deployed student network to obtain a global channel response matrix with higher estimation accuracy;

[0011] Among them, in the training stage of the knowledge distillation network, the teacher network is used to assist in training the student network. After a large number of training rounds, the channel estimation performance of the student network approaches that of the teacher network; load and deploy the training weights of the student network to obtain the deployed knowledge distillation network.

[0012] According to the preference of the present invention, before channel estimation, perform a fast Fourier transform (FFT) on the received signal after removing the cyclic prefix to the frequency domain. Among them, the frequency-domain received signal at the pilot is expressed as:

[0013] Y = XH + Z;

[0014] Among them, Y represents the frequency-domain received signal at the pilot, X represents the pilot signal, H represents the channel frequency-domain response, and Z represents additive white Gaussian noise;

[0015] In order to obtain channel estimation That is, the channel response matrix obtained by LS channel estimation. LS channel estimation needs to minimize the following cost function, that is, the objective function

[0016]

[0017] In order to minimize the sum of squared errors, find the first-order partial derivative of the above objective function:

[0018]

[0019] Let the first-order partial derivative be 0, and obtain:

[0020]

[0021] According to the preference of the present invention, the knowledge distillation network includes a teacher network and a student network;

[0022] The teacher network includes a 3×3 convolutional layer, a residual group, a three-dimensional spatial attention module, and an upsampling module; the residual group includes 20 serially-connected residual blocks; the student network includes a 3×3 convolutional layer, a residual group, a three-dimensional spatial attention module, and an upsampling module; the residual group includes 4 serially-connected residual blocks.

[0023] Preferably according to the present invention, a separate training method for the teacher network and the student network is adopted; it includes:

[0024] Training the teacher network; it includes:

[0025] For the channel response matrix with an input size of H×W×2, where H represents the height of the channel response matrix and W represents the width of the channel response matrix, first, use a 3×3 convolutional layer to perform a dimensionality increase operation on the channel response matrix in the channel direction to obtain shallow features with a size of H×W×C; subsequently, extract deep features through the residual group; the residual block introduces a three-dimensional spatial attention module to enhance the feature extraction ability; finally, perform an upsampling reconstruction operation on the deep features through an upsampling module to output the reconstructed estimated matrix;

[0026] Training the student network: it includes:

[0027] For the channel response matrix with an input size of H×W×2, first, use a 3×3 convolutional layer to perform a dimensionality increase operation on the channel response matrix in the channel direction to obtain shallow features with a size of H×W×C; subsequently, extract deep features through the residual group; the three-dimensional spatial attention module is used in the residual block to enhance the feature extraction ability; finally, perform an upsampling reconstruction operation on the deep features through an upsampling module to output the reconstructed estimated matrix.

[0028] Preferably according to the present invention, in the training stage of the knowledge distillation network,

[0029] Set the knowledge expression method for the teacher network to supervise the training of the student network; regard the intermediate features as knowledge, assuming that the intermediate feature of the teacher network for the i-th group of input data is The intermediate feature of the student network is The intermediate feature is the output feature after the input feature passes through its respective residual blocks, then define the loss of the knowledge distillation network as:

[0030]

[0031] where N is the total number of samples;

[0032] Define the loss of the student network as:

[0033]

[0034] Among them, represents the output after the i-th group of input data is input into the student network, represents the true value of the channel response corresponding to the i-th group of input data;

[0035] The training objective of the student network is to minimize the loss function:

[0036]

[0037] where α and β are adjustable hyperparameters representing their respective weights.

[0038] According to the preference of the present invention, the teacher network and the student network use a three-dimensional spatial attention module to enhance the feature extraction ability; including:

[0039] In the student network, assuming the input feature H, W, and C are the length, width, and channels of the feature respectively; first, perform pooling from three dimensions to obtain three features T1, T2, and T3:

[0040]

[0041] where Concat is concatenation and Avg is taking the mean

[0042] Subsequently, perform a dimensionality reduction operation on the dimension 2 direction of the three features T1, T2, and T3 using a 7×7 convolution to change the dimension channels to 1, obtaining T1′, T2′, and T3′; then perform element-wise multiplication with the input feature respectively to obtain the feature maps T H 、T C 、T W :

[0043]

[0044] Take the mean of the obtained feature maps T H 、T C 、T W to obtain the three-dimensional spatial attention module and output the intermediate feature map Y:

[0045]

[0046] The attention module dynamically assigns different weights to different parts of the input feature, highlighting key information and suppressing irrelevant information.

[0047] Input the channel response matrix after LS estimation into the knowledge distillation model, and the output of the model is the reconstructed global channel response matrix with higher precision. Using this channel matrix for subsequent channel equalization operations can recover the signal with higher precision.

[0048] After performing an upsampling operation on the output intermediate feature map Y, a global channel response matrix with higher estimation accuracy is output.

[0049] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation are implemented.

[0050] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation are implemented.

[0051] The beneficial effects of the present invention are as follows:

[0052] 1. The explicit channel estimation of the SC-IFDMA system based on deep learning in the present invention can significantly improve the channel estimation accuracy compared with the traditional LS estimation method.

[0053] 2. The present invention uses the idea of knowledge distillation to lightweight the model, which can greatly reduce the number of model parameters, reduce the difficulty of model deployment, and obtain better estimation performance than ordinary super-resolution models. Description of the Drawings

[0054] Figure 1 is a block diagram of an SC-IFDMA system;

[0055] Figure 2 is a flowchart of the channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation;

[0056] Figure 3 is a schematic diagram of a frame structure;

[0057] Figure 4 is a schematic diagram of the comparison of the estimation errors between the channel estimation method proposed in the present invention and other existing channel estimation methods;

[0058] Figure 5 is a schematic diagram of the signal-to-noise ratio-bit error rate curve of the SC-IFDMA system using different channel estimation methods under different signal-to-noise ratios;

[0059] Figure 6 is a schematic diagram of the specific framework of the knowledge distillation network;

[0060] Figure 7 is a schematic diagram of the structure of a residual block;

[0061] Figure 8 is a schematic diagram of a three-dimensional spatial attention module. Detailed Embodiments

[0062] The present invention will be further defined below in conjunction with the accompanying drawings of the specification and embodiments, but is not limited thereto.

[0063] Embodiment 1

[0064] A classical LS estimation method for an SC-IFDMA system.

[0065] The SC-IFDMA system is as Figure 1 shown. Before channel estimation, the received signal after removing the cyclic prefix is subjected to a fast Fourier transform (FFT) to the frequency domain. Among them, the frequency-domain received signal at the pilot is expressed as:

[0066] Y = XH + Z;

[0067] Among them, Y represents the frequency-domain received signal at the pilot, X represents the pilot signal, H represents the channel frequency-domain response, and Z represents additive white Gaussian noise;

[0068] To obtain the channel estimation That is, the channel response matrix obtained by LS channel estimation. The LS channel estimation needs to minimize the following cost function, that is, the objective function

[0069]

[0070] To minimize the sum of squared errors, find the first-order partial derivative of the above objective function:

[0071]

[0072] Let the first-order partial derivative be 0, and we get:

[0073]

[0074] Embodiment 2

[0075] According to the classical LS channel estimation method for an SC-IFDMA system described in Embodiment 1, the difference is that:

[0076] A lightweight SC-IFDMA system channel estimation method based on knowledge distillation, as Figure 2 shown, includes:

[0077] Input the channel response matrix obtained by LS channel estimation into the trained and deployed student network to obtain a global channel response matrix with higher estimation accuracy;

[0078] Among them, in the training stage of the knowledge distillation network, a teacher network with a deeper number of network layers is used to assist in training a student network with a very shallow number of network layers. After a large number of training rounds, the channel estimation performance of the student network approaches that of the teacher network; the training weights of the student network are loaded and deployed to obtain a deployed knowledge distillation network. Since the number of parameters of the student network is very small, the model deployment pressure can be greatly reduced.

[0079] As Figure 6 shown, the knowledge distillation network includes a teacher network and a student network;

[0080] The teacher network includes a 3×3 convolutional layer, a residual group, a three-dimensional spatial attention module, and an upsampling module; the residual group includes 20 residually connected residual blocks; the student network includes a 3×3 convolutional layer, a residual group, a three-dimensional spatial attention module, and an upsampling module; the residual group includes 4 residually connected residual blocks.

[0081] The residual block structures of the teacher network and the student network are the same, only the quantity is different.

[0082] Adopt a separate training method for the teacher network and the student network; including:

[0083] Train the teacher network; including:

[0084] For the channel response matrix with an input size of H×W×2, where H represents the height of the channel response matrix, which is equivalent to the height of the pilot matrix, and W represents the width of the channel response matrix, which is equivalent to the width of the pilot matrix. First, use a 3×3 convolutional layer to perform a dimension increase operation on the channel response matrix in the channel direction to obtain shallow features with a size of H×W×C; subsequently, extract deep features through the residual group; the residual block introduces a three-dimensional spatial attention module to enhance the feature extraction ability; finally, perform an upsampling reconstruction operation on the deep features through an upsampling module to output the reconstructed estimation matrix; this training process uses the real channel response matrix for supervised training, and the training process can be regarded as minimizing the mean square error loss between the estimated matrix output by the teacher network and the real channel response matrix.

[0085] Train the student network: including:

[0086] For a channel response matrix with an input size of H×W×2, first, a 3×3 convolutional layer is used to perform a dimensionality increase operation on the channel response matrix in the channel direction to obtain shallow features with a size of H×W×C. Subsequently, deep features are extracted through a residual group. The residual group of the student network is composed of 4 residual blocks connected in series, which is shallower than the teacher network in terms of the number of layers. A three-dimensional spatial attention module is used in the residual block to enhance the feature extraction ability. Finally, an upsampling module is used to perform an upsampling reconstruction operation on the deep features to output the reconstructed estimated matrix. The supervision in the training process of the student network is different from that of the teacher network. In addition to using the real channel response matrix for supervision, the knowledge distillation loss between the teacher network and the student network is also used for supervision. The training process can be regarded as simultaneously minimizing the distillation loss and the mean square error loss between the output estimated matrix and the real channel response matrix.

[0087] In the actual deployment stage, only the student model needs to be deployed, and there is no need to deploy the teacher model.

[0088] Since the channel estimation matrix is a complex-valued matrix, it is split into two matrices, the real part and the imaginary part, and the number of input channels is 2. After passing through a 3×3 convolutional layer, the number of channels is increased to C (C = 64). The structure of the residual block is as Figure 7 shown.

[0089] The residual block includes two 3×3 convolutional layers, two activation functions, and a three-dimensional spatial attention module.

[0090] In the training stage of the knowledge distillation network,

[0091] Set the knowledge expression method for the teacher network to supervise the training of the student network; so that the teacher network can more efficiently supervise the training of the student. Regarding the intermediate features as knowledge, assume that the intermediate feature of the teacher network for the i-th group of input data is and the intermediate feature of the student network is The intermediate feature is the output feature after the input feature passes through its respective residual block. Then, define the loss of the knowledge distillation network as:

[0092]

[0093] where N is the total number of samples;

[0094] Define the loss of the student network as:

[0095]

[0096] where, represents the output after the i-th group of input data is input into the student network, represents the true value of the channel response corresponding to the i-th group of input data;

[0097] Training objective of the student network is to minimize the loss function:

[0098]

[0099] where α and β are adjustable hyperparameters representing their respective weights.

[0100] Input the channel response matrix obtained by LS channel estimation into the trained and deployed student network to obtain a global channel response matrix with higher estimation accuracy.

[0101] The teacher network and the student network introduce a three-dimensional spatial attention module to highlight key information and suppress irrelevant information.

[0102] As Figure 8 shown, including:

[0103] In the teacher network and the student network, in order to enhance the feature extraction ability, a three-dimensional spatial attention mechanism with low parameter quantity is used to establish low-cost spatial attention without involving dimensionality reduction operations. Assume the input feature H, W, and C are the length, width, and channels of the feature respectively; first, perform pooling from three dimensions to obtain three features T1, T2, and T3:

[0104]

[0105] where Concat is concatenation and Avg is taking the mean

[0106] Subsequently, perform dimensionality reduction operation on the direction with dimension 2 in the three features T1, T2, and T3 using a 7×7 convolution to change the dimension channel to 1, obtaining T1′, T2′, and T3′; then perform element-wise multiplication with the input feature respectively to obtain the feature maps T H 、T C 、T W :

[0107]

[0108] Take the mean of the obtained feature maps T H 、T C 、T W to obtain the three-dimensional spatial attention module and output the intermediate feature map Y:

[0109]

[0110] The attention module dynamically assigns different weights to different parts of the input feature to highlight key information and suppress irrelevant information.

[0111] Channel response matrix after LS estimation After passing through the above knowledge distillation model, it is reconstructed into a global channel response matrix with higher precision. Using this channel matrix for subsequent channel equalization operations can recover signals with higher precision.

[0112] In this embodiment, simulation experiments are used to evaluate the performance of the SC-IFDMA channel estimation method proposed by the present invention. In the simulation experiment, the parameters of the SC-IFDMA system are set, where the number of subcarriers is set to 128, the number of symbols per subframe is 64, the cyclic prefix length is 16, the carrier frequency is 41.5 MHz, the bandwidth is 0.5 MHz, and the frame structure is as Figure 3 shown. The frame structure is mainly used to describe the number of subcarriers of the SC-IFDMA system and the number of symbols included in each subframe, which are N and S respectively. The specific values of the number of subcarriers, the number of symbols, and the cyclic prefix length are given in the simulation experiment. The blue squares represent pilots and are inserted at intervals in a trellis insertion manner.

[0113] After that, the training parameters of the knowledge distillation model are set. The optimizer is SGD, the learning rate is 0.0001, the number of iterations is 100, and the batch size is 32. Figure 4 The comparison of the estimation errors between the channel estimation method proposed by the present invention and other existing channel estimation methods under different signal-to-noise ratios is given, and the mean square error (MSE) is used as the evaluation index.

[0114] In addition, the number of deployed parameters of the knowledge distillation model in the present invention is about 50K, while the number of parameters of the super-resolution model can reach 800K. It can be seen that as the channel estimation performance using the knowledge distillation method of the present invention is higher than that of the LS channel estimation method and the channel estimation method based on the super-resolution model. In the low signal-to-noise ratio range, the performance of this method is improved by about 8 dB compared with the LS estimation and by about 2.9 dB compared with the super-resolution model. In the high signal-to-noise ratio range, the advantage of this method is more obvious, with an improvement of about 10.5 dB compared with the LS estimation and about 4.1 dB compared with the super-resolution model.

[0115] Figure 5 It is the signal-to-noise ratio-bit error rate curve of the SC-IFDMA system using different channel estimation methods under different signal-to-noise ratios. It can be seen that when using the knowledge distillation model for channel estimation, the bit error rate of the system is less than that of the other two channel estimation methods, and the higher the signal-to-noise ratio, the more obvious the performance advantage of this method.

[0116] Embodiment 3

[0117] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the lightweight SC-IFDMA system channel estimation method based on knowledge distillation described in Embodiment 1 or 2.

[0118] Embodiment 4

[0119] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the steps of a channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation described in Embodiment 1 or 2 are implemented.

Claims

1. A lightweight SC-IFDMA system channel estimation method based on knowledge distillation, characterized in that, Including: Input the channel response matrix obtained by LS channel estimation into the trained and deployed student network to obtain a global channel response matrix with higher estimation accuracy. Among them, in the training stage of the knowledge distillation network, the teacher network is used to assist in training the student network. After a large number of training rounds, the channel estimation performance of the student network approaches that of the teacher network; the training weights of the student network are loaded and deployed to obtain the deployed knowledge distillation network.

2. The channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation according to claim 1, wherein Before channel estimation, perform a fast Fourier transform on the received signal after removing the cyclic prefix to the frequency domain. Among them, the frequency-domain received signal at the pilot is expressed as: Y = XH + Z; Among them, Y represents the frequency-domain received signal at the pilot, X represents the pilot signal, H represents the channel frequency response, and Z represents additive white Gaussian noise. In order to obtain channel estimation That is, the channel response matrix obtained by LS channel estimation. LS channel estimation needs to minimize the cost function, that is, the objective function To minimize the sum of squared errors, find the first-order partial derivative of the above objective function: Let the first-order partial derivative be 0, and obtain:

3. A channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation according to claim 1, characterized in that, The knowledge distillation network includes a teacher network and a student network; The teacher network includes a 3×3 convolutional layer, a residual group, a three-dimensional spatial attention module, and an upsampling module; the residual group includes 20 serially connected residual blocks; the student network includes a 3×3 convolutional layer, a residual group, a three-dimensional spatial attention module, and an upsampling module; the residual group includes 4 serially connected residual blocks.

4. A channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation according to claim 3, characterized in that, Adopt a separate training method for the teacher network and the student network; including: Train the teacher network; including: For the channel response matrix with an input size of H×W×2, where H represents the height of the channel response matrix and W represents the width of the channel response matrix. First, use a 3×3 convolutional layer to perform a dimension-increasing operation on the channel response matrix in the channel direction to obtain shallow features with a size of H×W×C; then, extract deep features through the residual group; the residual block introduces a three-dimensional spatial attention module to enhance the feature extraction ability; finally, perform an upsampling reconstruction operation on the deep features through an upsampling module to output the reconstructed estimation matrix. Train the student network: including: For the channel response matrix with an input size of H×W×2, first, use a 3×3 convolutional layer to perform a dimension-increasing operation on the channel response matrix in the channel direction to obtain shallow features with a size of H×W×C; then, extract deep features through the residual group; the three-dimensional spatial attention module is used in the residual block to enhance the feature extraction ability; finally, perform an upsampling reconstruction operation on the deep features through an upsampling module to output the reconstructed estimation matrix.

5. A channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation according to claim 3, characterized in that, In the training stage of the knowledge distillation network, Set the knowledge expression method for the teacher network to supervise the student network training; regard the intermediate features as knowledge, and assume that the intermediate features of the teacher network for the i-th group of input data are The intermediate features of the student network are If the intermediate features are the output features after the input features pass through their respective residual blocks, then define the loss of the knowledge distillation network as: Among them, N is the total number of samples; Define the loss of the student network as follows: Among them, represents the output after the i-th group of input data is input into the student network, represents the true value of the channel response corresponding to the i-th group of input data; Training objective of the student network is to minimize the loss function: Among them, α and β are adjustable hyperparameters representing their respective weights.

6. A channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation according to claim 3, characterized in that, The teacher network and the student network use a three-dimensional spatial attention module to enhance the feature extraction ability; including: In the student network, assume the input feature where H, W, and C are the length, width, and channels of the feature respectively; first, perform pooling in three dimensions to obtain three features T1, T2, and T3: Among them, Concat is concatenation, and Avg is taking the mean. Then, the dimensionality of the three features T1, T2, and T3 in the direction of dimension 2 is reduced using a 7×7 convolution, and the dimension channel is changed to 1 to obtain T1′, T2′, and T3′; then they are element-wise multiplied with the input features to obtain the feature map T H , T C , T W : The obtained feature map T H , T C , T W Take the mean to obtain a three-dimensional spatial attention module and output the intermediate feature map Y: The attention module dynamically assigns different weights to different parts of the input features, highlighting key information and suppressing irrelevant information.

7. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for channel estimation of a lightweight SC-IFDMA system based on knowledge distillation according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of a channel estimation method for a lightweight SC-IFDMA system based on knowledge distillation according to any one of claims 1-6.

Images