Large-scale MIMO CSI feedback method fusing self-attention mechanism

Abstract

The invention discloses a large-scale MIMO (Multiple Input Multiple Output) CSI (Channel State Information) feedback method fused with a self-attention mechanism, which is fused with the self-attention mechanism and is combined with residual connection and a deep separable convolutional layer to realize efficient and accurate CSI feedback. In the encoder part, a multi-layer self-attention mechanism is adopted to carry out feature extraction and compression; in a decoder part, through cooperative work of multiple convolutional layers and a self-attention mechanism, an embedded vector generated by an encoder is reconstructed into an original image. The model performance is remarkably enhanced by combining residual connection and a depth separable convolutional layer sequence in an attention mechanism. In the encoder part, a multi-layer self-attention mechanism is adopted to realize efficient feature extraction and compression. In a decoder part, multiple convolutional layers and a self-attention mechanism work cooperatively, and an embedded vector generated by an encoder is reconstructed into an original image. The method can be used for efficiently and accurately realizing the CSI feedback task of the large-scale MIMO system.

Description

Technical Field

[0001] The present invention relates to the field of computer intelligence and wireless technology, and in particular to a large-scale MIMO CSI feedback method integrating a self-attention mechanism. Background Art

[0002] Massive MIMO achieves high spectral efficiency and energy efficiency by using a large number of antennas at the transmit and receive ends. However, in order to take advantage of these advantages, the base station (BS) needs to obtain the CSI of the downlink for beamforming. In frequency division duplex (FDD) mode, it is challenging for the base station to obtain CSI information due to the weak reciprocity between the downlink and uplink channels. Therefore, the user equipment (UE) needs to estimate the downlink CSI and feed it back to the base station. However, the feedback cost increases linearly with the number of transmit antennas. In addition, the user's transmission power and uplink bandwidth are limited, making it challenging to effectively compress the feedback CSI of massive MIMO systems. In the prior art, some methods use compressed sensing (CS)-based techniques to alleviate this problem. However, the CS method requires the CSI matrix to remain highly sparse, which is not always feasible in practical systems.

[0003] In recent years, the great success of deep learning (DL) in computer vision and signal processing, especially its dominance in image compression tasks, has inspired researchers to explore DL-based codec networks to solve the CSI feedback problem. Some studies have applied deep learning techniques to the CSI feedback problem, and proposed methods such as CsiNet, which show overwhelming advantages over traditional CS methods. Subsequently, many researchers have extended these frameworks to massive MIMO systems. Inspired by these advances, various DL-based massive MIMO CSI feedback methods have been widely studied. Some studies have introduced a lightweight neural network method that exploits the correlation of CSI in the time and angle domains through deep learning, which improves feedback accuracy and reduces complexity. Other studies have proposed models that significantly reduce computational complexity through spatially separable attention mechanisms and perform well in CSI feedback. Other studies have achieved higher accuracy through forged complex input layers and spatial attention mechanisms, and other studies have improved CSI feedback performance while maintaining computational complexity through multi-resolution feature extraction and advanced training schemes. In addition, there are attention-based deep learning networks that directly predict downstream CSI, effectively eliminating feedback overhead in FDD systems while achieving high prediction accuracy and low complexity. Summary of the invention

[0004] The purpose of the present invention is to address the defects and deficiencies of the above-mentioned prior art and propose a large-scale MIMO CSI feedback method integrating a self-attention mechanism. The method integrates the self-attention mechanism, combines residual connections and deep separable convolutional layers to achieve efficient and accurate CSI feedback. In the encoder part, a multi-layer self-attention mechanism is used for feature extraction and compression; in the decoder part, the embedded vector generated by the encoder is reconstructed into the original image through the multi-layer convolutional layer and the self-attention mechanism. Experimental results show that the feedback performance of FSAMNet in various compression scenarios is better than that of the benchmark scheme, and it has significant performance advantages. The present invention is used to efficiently and accurately implement the CSI feedback task of a large-scale MIMO system. The model performance is significantly enhanced by combining residual connections and deep separable convolutional layer sequences in the attention mechanism. In the encoder part, a multi-layer self-attention mechanism is used to achieve efficient feature extraction and compression. In the decoder part, multi-layer convolutional layers work together with the self-attention mechanism to reconstruct the embedded vector generated by the encoder into the original image. The effectiveness and feasibility of the method are verified through comprehensive analysis and experiments, and compared with existing CSI compression technologies. Simulation results show that the feedback performance of the proposed FSAMNet in various compression scenarios is better than that of the traditional benchmark scheme.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a large-scale MIMO CSI feedback method integrating a self-attention mechanism, which is used to efficiently and accurately implement the CSI feedback task of a large-scale MIMO system. Specifically, it includes the following steps:

[0006] In step 1, the input data is passed through a series of convolutional layers for preliminary feature extraction to generate a set of feature maps.

[0007] Step 2: Send the feature map to Transformer Blocks, apply the self-attention mechanism in each block to capture the feature relationship between each position in the image, and perform layer normalization after each block.

[0008] Normalization) to ensure the stability of training. The specific calculation formula of the self-attention mechanism is:

[0009]

[0010] Among them, Q represents the query matrix, K represents the key matrix, V represents the value matrix, and d k is the dimension of the query and key.

[0011] In step 3, a deconvolution operation is performed on the feature map through a transposed convolution layer (ConvT) to restore it to the original image size.

[0012] In step 4, the obtained feature map is added and fused with the output of the third convolutional layer, and then normalized (Layer Normalization2) and reshaped into a one-dimensional vector.

[0013] Step 5: The vector passes through two fully connected layers (FC1 and FC2), and each layer uses the ReLU activation function to generate a low-dimensional encoded representation (encoded_dim), which is the final encoded output. The specific calculation formula for layer normalization is:

[0014]

[0015] Where x is the input vector, μ is the mean of the input, and σ 2 is the variance of the input, ∈ is a small constant used to avoid division by zero, and γ and β are learnable scaling and bias parameters.

[0016] Step 6, starting from the low-dimensional encoded representation (encoded_dim), expands it to a feature map through a fully connected layer (FC).

[0017] Step 7: The feature map undergoes a series of convolution transformations, combining convolution operations and maximum pooling operations to extract key image features. The specific calculation formula for the convolution operation is:

[0018]

[0019] Among them, X is the input two-dimensional matrix, K is the convolution kernel, and Y is the output feature map.

[0020] In step 8, these feature maps are processed through a series of decoding feature layers, which consist of convolutional layers and transposed convolutional layers, responsible for reconstructing the details and structure of the image.

[0021] Step 9: The output of the decoded feature layer is fused with the output of the encoder to integrate the information of the encoder and decoder.

[0022] Step 10, to ensure the training stability of the architecture, a normalization layer is applied after each transformer block.

[0023] Step 11, the last step of the decoding process uses the sigmoid activation function to generate the final reconstructed image. The sigmoid function is selected as the activation function of the final layer of the decoder to limit the output value to the range [0,1], thereby generating accurate pixel value estimates, which is beneficial for image reconstruction.

[0024] Furthermore, in step 11, the specific calculation formula of the sigmoid activation function is:

[0025]

[0026] The large-scale MIMO CSI feedback method integrating the self-attention mechanism of the present invention mainly includes the following contents:

[0027] The encoder module processes the input data and extracts its features, while the decoder module remaps the encoded features to the original representation of the input data.

[0028] The encoder utilizes a multi-layer self-attention mechanism, which consists of multiple self-attention blocks arranged in sequence. This configuration enables the model to flexibly capture the correlation between pixels in an image, regardless of the size and shape of the image. Through hyperparameter adjustment, the behavior of the self-attention mechanism is fine-tuned so that it can effectively adapt to various image characteristics;

[0029] In order to fully capture local and global features, the present invention combines traditional convolution operations with transposed convolution operations to ensure the preservation of contextual information and achieve accurate image reconstruction. To further improve performance, feature fusion and reshaping techniques are used to enhance the information content of the generated feature representation;

[0030] In the decoder part, the present invention adopts a collaborative approach combining multi-layer convolutional layers with a self-attention mechanism. The design aims to efficiently restore the low-dimensional embedding vector generated by the encoder to the original image. The design concept of this architecture is to solve the complexity and information loss problems commonly encountered in image reconstruction tasks;

[0031] The decoder first transforms the embedding vector into a representation that matches the size of the original image through a fully connected layer. Subsequently, a convolutional layer is used to depict the features and structure of the image. At the same time, the integration of the self-attention mechanism enables the model to process both global and local information of the image, thereby enhancing its ability to accurately reconstruct the details and structure of the original image.

[0032] Beneficial effects:

[0033] 1. The present invention realizes efficient and accurate CSI feedback and significantly improves the performance of large-scale MIMO systems.

[0034] 2. The large-scale MIMO CSI feedback architecture of the present invention integrates the self-attention mechanism, improves the model performance by introducing residual connections in the attention mechanism, and introduces a sequence of deeply separable convolutional layers to enhance the expressiveness of the model; a multi-layer self-attention mechanism is used in the encoder part to realize feature extraction and compression; and multiple convolutional layers and the self-attention mechanism are used in the decoder part to convert the embedded vector generated by the encoder back to the original image.

[0035] 3. The performance of the model in this paper and other models is evaluated on the COST2100 dataset. Compared with the classic CsiNet, the CSI feedback scheme proposed in this paper has been significantly improved in terms of the overall average performance of NMSE.

[0036] 4. The present invention not only performs well in image reconstruction tasks and effectively preserves image details and structures, but also performs well in reducing information loss compared to traditional decoder designs. While ensuring contextual information, it achieves accurate image reconstruction and effectively addresses the complexity and information loss problems in image reconstruction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an overall framework diagram of a large-scale MIMO CSI feedback method integrating a self-attention mechanism of the present invention.

[0038] Figure 2 It is a schematic diagram of the principle of a large-scale MIMO CSI feedback method integrating a self-attention mechanism of the present invention.

[0039] Figure 3 This is a graph showing the loss decrease trend of FSAMNet during 1000 training cycles under the condition of η=1 / 64 for a large-scale MIMO CSI feedback method integrating a self-attention mechanism according to the present invention, both indoors and outdoors.

[0040] Figure 4 This is a test NMSE (dB) trend chart of a large-scale MIMO CSI feedback method integrating a self-attention mechanism under the condition of η=1 / 64, and 1000 epochs of FSAMNet training in indoor and outdoor scenarios. DETAILED DESCRIPTION

[0041] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical method of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0042] like Figure 1 and Figure 2 As shown, the present invention proposes a large-scale MIMO CSI feedback method integrating a self-attention mechanism, which is used to efficiently and accurately implement the CSI feedback task of a large-scale MIMO system. Specifically, the following steps are included:

[0043] In step 1, the input data is passed through a series of convolutional layers for preliminary feature extraction to generate a set of feature maps.

[0044] In step 2, the feature map is fed into Transformer Blocks, and a self-attention mechanism is applied in each block to capture the feature relationship between each position in the image. Layer Normalization is performed after each block to ensure the stability of training.

[0045] In step 3, a deconvolution operation is performed on the feature map through a transposed convolution layer (ConvT) to restore it to the original image size.

[0046] In step 4, the obtained feature map is added and fused with the output of the third convolutional layer, and then normalized (Layer Normalization2) and reshaped into a one-dimensional vector.

[0047] In step 5, the vector passes through two fully connected layers (FC1 and FC2), each using the ReLU activation function to generate a low-dimensional encoded representation (encoded_dim), which is the final encoded output.

[0048] Step 6, starting from the low-dimensional encoded representation (encoded_dim), expands it to a feature map through a fully connected layer (FC).

[0049] In step 7, the feature map undergoes a series of convolution transformations, combining convolution operations and maximum pooling operations to extract key image features.

[0050] In step 8, these feature maps are processed through a series of decoding feature layers, which consist of convolutional layers and transposed convolutional layers, responsible for reconstructing the details and structure of the image.

[0051] Step 9: The output of the decoded feature layer is fused with the output of the encoder to integrate the information of the encoder and decoder.

[0052] Step 10, to ensure the training stability of the architecture, a normalization layer is applied after each transformer block.

[0053] Step 11, the last step of the decoding process uses the sigmoid activation function to generate the final reconstructed image. The sigmoid function is selected as the activation function of the final layer of the decoder to limit the output value to the range [0,1], thereby generating accurate pixel value estimates, which is beneficial for image reconstruction.

[0054] In step 2, the specific calculation formula of the self-attention mechanism is:

[0055]

[0056] Among them, Q represents the query matrix, K represents the key matrix, V represents the value matrix, and d k is the dimension of the query and key.

[0057] In step 5, the specific calculation formula for layer normalization is:

[0058]

[0059] Where x is the input vector, μ is the mean of the input, and σ 2 is the variance of the input, ∈ is a small constant used to avoid division by zero, and γ and β are learnable scaling and bias parameters.

[0060] In step 7, the specific calculation formula of the convolution operation is:

[0061]

[0062] Among them, X is the input two-dimensional matrix, K is the convolution kernel, and Y is the output feature map.

[0063] In step 11, the specific calculation formula of the sigmoid activation function is:

[0064]

[0065] Through the above steps, the present invention realizes efficient and accurate CSI feedback and significantly improves the performance of the large-scale MIMO system.

[0066] The large-scale MIMO CSI feedback method integrating the self-attention mechanism of the present invention mainly includes the following contents:

[0067] The encoder module processes the input data and extracts its features, while the decoder module remaps the encoded features to the original representation of the input data.

[0068] The encoder utilizes a multi-layer self-attention mechanism, which consists of multiple self-attention blocks arranged in sequence. This configuration enables the model to flexibly capture the correlation between pixels in an image, regardless of the size and shape of the image. Through hyperparameter adjustment, the behavior of the self-attention mechanism is fine-tuned so that it can effectively adapt to various image characteristics;

[0069] In order to fully capture local and global features, the traditional convolution operation and the transposed convolution operation are combined to ensure the preservation of contextual information and achieve accurate image reconstruction. In order to further improve the performance, feature fusion and reshaping techniques are used to enhance the information content of the generated feature representation;

[0070] In the decoder part, a collaborative approach combining multiple convolutional layers and self-attention mechanism is adopted. This design aims to efficiently restore the low-dimensional embedding vector generated by the encoder to the original image. The design concept of this architecture is to solve the complexity and information loss problems commonly encountered in image reconstruction tasks;

[0071] The decoder first converts the embedding vector into a representation that matches the size of the original image through a fully connected layer. Subsequently, a convolutional layer is used to depict the features and structure of the image. At the same time, the integration of the self-attention mechanism enables the model to process both global and local information of the image, thereby enhancing its ability to accurately reconstruct the details and structure of the original image;

[0072] The present invention effectively preserves image details and structures, and also performs better than traditional decoder designs in reducing information loss;

[0073] Through the above content, the method proposed in the present invention achieves accurate image reconstruction while ensuring context information, and effectively addresses the complexity and information loss problems in the image reconstruction task.

[0074] In the present invention, the effectiveness of the CSI feedback method proposed in the present invention is verified by simulation results. Specifically, the present invention uses a data set generated by the COST2100 channel model, focusing on two different scenarios: an indoor microcell environment in the 5.3GHz frequency band and an outdoor scenario in the 300MHz frequency band. All training and test computing experiments are performed on an NVIDIA 2080 GPU. The network is trained in the PyTorch framework using the Adam optimizer. The training protocol includes 1000 epochs, a batch size of 200, and an initial learning rate set to 0.001.

[0075] In order to evaluate the effect of CSI reconstruction, the present invention mainly uses the normalized mean square error (NMSE) as an evaluation index. This index quantitatively evaluates the original channel state information H a Corresponding to its reconstruction The difference between the two is as follows:

[0076]

[0077] In our comparative analysis, we evaluate the performance of the proposed model against established models such as CLNet and CRNet on the COST2100 dataset. This evaluation specifically focuses on the NMSE metric to assess the accuracy of each model in indoor and outdoor scenarios. The results of this comparison (excluding quantization effects) are systematically presented in Table 1 (NMSE comparison of a massive MIMO CSI feedback method with self-attention mechanism and other methods). The best results are highlighted in bold in the table, allowing the reader to immediately identify the leading performance metric in the considered environment.

[0078] Table 1

[0079]

[0080]

[0081] The simulation results highlight the improvement of the proposed CSI feedback scheme in NMSE performance compared to the established CsiNet model. In indoor scenarios, the proposed scheme shows better reconstruction performance than CRNet and CLNet at compression ratios of 1 / 8, 1 / 16, 1 / 32 and 1 / 64, demonstrating its effectiveness at different data compression levels. Specifically, the proposed model exceeds the best results of CRNet and CLNet in indoor CSI feedback accuracy, marking a significant advancement in this field. In outdoor scenarios, the proposed model continues to show better reconstruction performance than CRNet and CLNet at compression ratios of 1 / 32 and 1 / 64. However, at a compression ratio of 1 / 8, the NMSE performance of the proposed scheme is 1.43dB lower than that of CLNet, and 0.60dB lower at a compression ratio of 1 / 16, indicating that CLNet still has a slight advantage in some aspects.

[0082] like Figure 3 As shown in the figure, the present invention demonstrates the training performance of the model under indoor and outdoor conditions under the setting of η = 1 / 64. In 1000 training epochs, the method of the present invention shows a continuous decline in loss, indicating a gradual and stable optimization process. The loss curve shows a clear downward trend, and finally reaches and maintains a stable state within a specific range. This stability shows the effective convergence of the model and its strong generalization ability.

[0083] like Figure 4 Figure 1 shows the performance of the test phase, again focusing on indoor and outdoor scenes under the η = 1 / 64 setting. The NMSE trend over the same 1000 training epochs is shown here. The results confirm the ability of the proposed method to accurately capture the complex relationship between local and global CSI during compression and restoration. This ability highlights the effectiveness of the model in maintaining high reconstruction accuracy at significant compression rates.

[0084] Based on the CSI feedback method in FDD massive MIMO system, this paper proposes a massive MIMO CSI feedback architecture integrating self-attention mechanism. The residual connection is introduced in the attention mechanism to improve the model performance. At the same time, the depth-separable convolution layer sequence is introduced to enhance the model's expressiveness. The encoder part uses a multi-layer self-attention mechanism to achieve feature extraction and compression. In the decoder part, multiple convolutional layers and self-attention mechanism are used to convert the embedding vector generated by the encoder back to the original image. We evaluated the performance of the model of this paper and other models on the COST2100 dataset. Compared with the classic CsiNet, the CSI feedback scheme proposed in this paper has been improved in terms of NMSE overall average performance.

Claims

1. A large-scale MIMO CSI feedback method integrating self-attention mechanism, It is characterized in that The application method comprises the following steps: Step 1: The input data is passed through a series of convolutional layers for preliminary feature extraction to generate a set of feature maps; Step 2: Send the feature map to Transformer Blocks, apply the self-attention mechanism in each block to capture the feature relationship between each position in the image, and perform layer normalization after each block to ensure the stability of training. The specific calculation formula of the self-attention mechanism is: Among them, Q represents the query matrix, K represents the key matrix, V represents the value matrix, and d k is the dimension of query and key; Step 3: Deconvolve the feature map through a transposed convolution layer (ConvT) to restore it to the original image size; Step 4: Add the obtained feature map to the output of the third convolutional layer, perform another round of normalization (Layer Normalization2) and reshape it into a one-dimensional vector; Step 5: The vector passes through two fully connected layers (FC1 and FC2), and each layer uses the ReLU activation function to generate a low-dimensional encoded representation (encoded_dim), which is the final encoded output. The specific calculation formula for layer normalization is: Where x is the input vector, μ is the mean of the input, and σ 2 is the variance of the input, ∈ is a small constant used to avoid division by zero, and γ and β are learnable scaling and offset parameters; Step 6: Starting from the low-dimensional encoded representation (encoded_dim), expand it to a feature map through a fully connected layer (FC); Step 7: The feature map undergoes a series of convolution transformations, combining convolution operations and maximum pooling operations to extract key image features. The specific calculation formula for the convolution operation is: Among them, X is the input two-dimensional matrix, K is the convolution kernel, and Y is the output feature map; Step 8: These feature maps are processed through a series of decoding feature layers, which consist of convolutional layers and transposed convolutional layers, responsible for reconstructing the details and structure of the image; Step 9: The output of the decoded feature layer is fused with the output of the encoder to integrate the information of the encoder and decoder; Step 10: To ensure the training stability of the architecture, a normalization layer is applied after each transformer block; Step 11: The last step of the decoding process uses the sigmoid activation function to generate the final reconstructed image. The sigmoid function is selected as the activation function of the final layer of the decoder to limit the output value to the range of [0,1], thereby generating accurate pixel value estimates, which is beneficial for image reconstruction.

2. According to the large-scale MIMO CSI feedback method integrating the self-attention mechanism according to claim 1, It is characterized in that In step 11, the specific calculation formula of the sigmoid activation function is:

3. According to claim 1, a large-scale MIMO CSI feedback method integrating a self-attention mechanism, It is characterized in that The method comprises: The encoder module processes the input data and extracts its features, while the decoder module remaps the encoded features to the original representation of the input data; The encoder utilizes a multi-layer self-attention mechanism, which consists of multiple self-attention blocks arranged in sequence. This configuration enables the model to flexibly capture the correlations between pixels in an image, regardless of the size and shape of the image. Through hyperparameter adjustment, the behavior of the self-attention mechanism is fine-tuned so that it can effectively adapt to various image characteristics.

Images