Channel information compression method and device based on cross attention

Abstract

The present invention provides a channel information compression method and device based on cross attention. The method includes: performing dimension transformation on any channel state information CSI matrix to obtain row vectors and column vectors corresponding to the CSI matrix; based on the row vectors and column vectors, obtaining a first matrix, the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column; based on the product of the amplitude of each response point in the CSI matrix with the remaining response points in the same row and column, determining the second matrix; dividing the first matrix and the second matrix to obtain the cosine similarity matrix of each response point in the CSI matrix with the remaining response points in the same row and column. The channel information compression method and device based on cross attention provided by the present invention can ensure that effective information in the channel state information matrix can be extracted, and can also cleverly reduce the computational complexity through matrix transformation.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a channel information compression method and device based on cross attention. Background Art

[0002] The impact of communications benefits mankind all the time. From 4G to 5G, the advantages of high speed, low latency and large capacity have helped the implementation of artificial intelligence products, while also promoting the development of the Internet of Things, bringing beneficial impacts to human life now and in the future.

[0003] With the development of science and technology, people have higher requirements for the speed and quality of communication. Multi-input Multi-output (MIMO) technology occupies a core position in the 5G system. Although multiple antennas can greatly improve wireless performance, sometimes too much information from the received paths will cause phase superposition, which will affect the quality of communication.

[0004] How to complete beamforming well at the base station has become an important issue. Accurately obtaining the channel state information (CSI) of the downlink can well assist the completion of the beamforming task. Due to the mutual differences between the uplink and downlink in the frequency division duplex (FDD) system, the base station must be able to obtain the channel estimated by the user equipment (UE). Accurately analyzing the data characteristics of the CSI matrix will be of great help to tasks such as channel estimation and channel feedback.

[0005] In reality, communication systems are often changeable and data collection is difficult. Generally, there are relatively few real channel data sets, so simulated channels are often used to process data. Commonly used channel models include the Spatial Channel Model (SCM), COST2100 model, etc. The modeled channels usually maintain the statistical characteristics of space, time, and frequency domains, and set different clusters, antenna arrays, or line-of-sight and non-line-of-sight methods so that researchers can generate channel data under different channel models for different experiments.

[0006] Accurately analyzing the data characteristics before modeling is of great help to the construction of the model. For the differences between different points on the CSI matrix, it is an essential step to select a reasonable metric distance for visual analysis according to different measurement methods in data analysis. Commonly used methods include: Euclidean distance, Manhattan distance, cosine distance, Mahalanobis distance, etc. In signal processing, information is usually expressed in the form of complex numbers. For a data set expressed in complex form such as the CSI matrix, the cosine similarity method is the best way to measure the distribution differences between different data points. However, it is obviously impractical to use cosine similarity to measure the difference between each response point and the other different points on the CSI matrix. First, the correlation between different points is different. In the MIMO system, even after compression by some methods, the CSI matrix still has a large dimension. Measuring the cosine similarity of all points will result in too much redundant information. Secondly, if the cosine similarity of each response point on the matrix and the other response points on the matrix is ​​simply calculated by traversal, the computational complexity is too high. Summary of the invention

[0007] The present invention provides a channel information compression method and device based on cross attention, which are used to solve the technical problems in the prior art that the traditional similarity estimation technology cannot be well applied to the channel state information matrix and the calculation complexity is high when estimating data correlation.

[0008] The present invention provides a channel information compression method based on cross attention, comprising:

[0009] Performing dimension transformation on any channel state information CSI matrix to obtain row vectors and column vectors corresponding to the CSI matrix;

[0010] Based on the row vector and the column vector, a first matrix is ​​obtained, where the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column;

[0011] Determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column;

[0012] The first matrix is ​​divided by the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the other response points in the same row and column, where the cosine similarity matrix is ​​used to indicate data characteristics of the CSI matrix.

[0013] In some embodiments, the CSI matrix is ​​represented as c represents the number of input channels of the target neural network, h represents the number of antennas corresponding to the CSI matrix, and w represents the number of carriers corresponding to the CSI matrix;

[0014] The performing dimension transformation on any channel state information CSI matrix to obtain a row vector and a column vector corresponding to the CSI matrix comprises:

[0015] Regarding the After dimension transformation, we get and Said is the row vector, the is the column vector;

[0016] The step of obtaining a first matrix based on the row vector and the column vector includes:

[0017] Regarding the and stated Transform and get the transformed row vector and the transformed column vector

[0018] Determine the dot product of each response point in the CSI matrix with the rest of the response points in the same row And the dot product of each response point in the CSI matrix and the other response points in the same column

[0019] Keep the F r The dimension of F remains unchanged. l The dimension is transformed into

[0020] The F r and the transformed F l Merge to get

[0021] Among them, the is the first matrix.

[0022] In some embodiments, before determining the second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column, the method further includes:

[0023] Based on a preset algorithm, determine the product of the amplitude of each response point in the CSI matrix and the other response points in the same row and the product of the amplitude of each response point in the CSI matrix and the other response points in the same column;

[0024] Based on the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same row and the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same column, the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same row and column is determined.

[0025] In some embodiments, the target neural network comprises:

[0026] A coding layer, used to extract a eigenvector of the CSI matrix to obtain a CSI matrix after feature extraction;

[0027] A downsampling layer, used to compress the CSI matrix after the feature extraction;

[0028] A fully connected layer, used to compress the high-dimensional vector into a low-dimensional vector at the mobile end, and restore the low-dimensional vector to the high-dimensional vector at the base station end;

[0029] The decoding layer is used to restore the low-dimensional vector returned by the mobile terminal into the CSI matrix.

[0030] In some embodiments, the CSI matrix is ​​a data set generated by a COST2100 model.

[0031] The present invention also provides a channel information compression device based on cross attention, comprising:

[0032] A first determination module is used to perform dimension transformation on any channel state information CSI matrix to obtain a row vector and a column vector corresponding to the CSI matrix;

[0033] A second determination module, configured to obtain a first matrix based on the row vector and the column vector, wherein the first matrix is ​​used to indicate a dot product of each response point in the CSI matrix with other response points in the same row and column;

[0034] A third determination module, configured to determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column;

[0035] The fourth determination module is used to divide the first matrix and the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the remaining response points in the same row and column, wherein the cosine similarity matrix is ​​used to indicate the data characteristics of the CSI matrix.

[0036] In some embodiments, the CSI matrix is ​​represented as c represents the number of input channels of the target neural network, h represents the number of antennas corresponding to the CSI matrix, and w represents the number of carriers corresponding to the CSI matrix;

[0037] The first determining module is further used for:

[0038] Regarding the After dimension transformation, we get and Said is the row vector, the is the column vector;

[0039] The second determining module is further used for:

[0040] Regarding the and stated Transform and get the transformed row vector and the transformed column vector

[0041] Determine the dot product of each response point in the CSI matrix with the rest of the response points in the same row And the dot product of each response point in the CSI matrix and the other response points in the same column

[0042] Keep the F r The dimension of F remains unchanged. l The dimension is transformed into

[0043] The F r and the transformed F l Merge to get

[0044] Among them, the is the first matrix.

[0045] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the channel information compression method based on cross attention as described above is implemented.

[0046] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described cross-attention-based channel information compression methods.

[0047] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the channel information compression method based on cross attention as described above is implemented.

[0048] The cross-attention-based channel information compression method and device provided by the present invention calculate the cosine similarity between each response point and the other response points in the same row and column, and expand the data characteristics of all points in the same matrix by row and column to obtain the data characteristics of the channel state information matrix, thereby helping to enhance or compress data in communication tasks. The data evaluation method is a new method designed based on communication characteristics, which is more conducive to analyzing the characteristics of the channel data itself and thus helping the analysis of the channel data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a flow chart of a channel information compression method based on cross attention provided by the present invention;

[0051] Figure 2 It is a general structural framework diagram of the channel state information feedback based on deep learning technology of the channel information compression method based on cross attention provided by the present invention;

[0052] Figure 3 It is a visualization diagram of the real part of the channel state information matrix using the channel information compression method based on cross attention provided by the present invention;

[0053] Figure 4 It is a preset algorithm flow chart of the channel information compression method based on cross attention provided by the present invention;

[0054] Figure 5 It is one of the schematic diagrams of the cosine similarity matrix of the channel information compression method based on cross attention provided by the present invention;

[0055] Figure 6 This is the second schematic diagram of the cosine similarity matrix of the channel information compression method based on cross attention provided by the present invention;

[0056] Figure 7 It is a schematic diagram of indoor and outdoor data distribution using the channel information compression method based on cross attention provided by the present invention;

[0057] Figure 8 It is a structural schematic diagram of a channel information compression device based on cross attention provided by the present invention;

[0058] Fig. 9 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Figure 1 A schematic diagram of a flow chart of a channel information compression method based on cross attention provided by the present invention. Figure 1 The channel information compression method based on cross attention provided by the present invention includes: step 110, step 120, step 130 and step 140.

[0061] Step 110: Perform dimension transformation on any channel state information CSI matrix to obtain row vectors and column vectors corresponding to the CSI matrix;

[0062] Step 120: Based on the row vector and the column vector, a first matrix is ​​obtained, where the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column;

[0063] Step 130: determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column;

[0064] Step 140: Divide the first matrix and the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the other response points in the same row and column. The cosine similarity matrix is ​​used to indicate the data characteristics of the CSI matrix.

[0065] The execution subject of the cross-attention-based channel information compression method provided by the present invention can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited by the present invention.

[0066] The following takes a computer executing the cross-attention-based channel information compression method provided by the present invention as an example to explain the technical solution of the present invention in detail.

[0067] In actual implementation, the basic scenario to which the present invention is applied is the feedback process of the channel state information (CSI) matrix.

[0068] The embodiment of the present invention provides a channel information compression method based on cross cosine similarity, the main purpose of which is to explore the data characteristics of the channel matrix in the carrier direction and the antenna direction, so as to facilitate Figure 2 The Down Sample module adopts a reasonable down sampling mode to achieve the purpose of reducing the parameters of the fully connected layer. Figure 2 Briefly describe one of the practical scenarios of the application of the present invention:

[0069] First, the overall module is briefly introduced. The target neural network corresponding to the overall module provided by the present invention includes:

[0070] The coding layer is used to extract the eigenvector of the CSI matrix and obtain the CSI matrix after feature extraction;

[0071] The downsampling layer is used to compress the CSI matrix after feature extraction;

[0072] The fully connected layer is used to compress the high-dimensional vector into a low-dimensional vector on the mobile end and restore the low-dimensional vector to a high-dimensional vector on the base station end;

[0073] The decoding layer is used to restore the low-dimensional vector returned by the mobile terminal to the CSI matrix.

[0074] For the feedback process, the overall module adopts a data-driven approach. The entire module uses an automatic encoder and can be divided into four layers: encoding layer, downsampling layer, fully connected layer and decoding layer.

[0075] The encoding layer corresponds to the encoder part, which is used to extract the eigenvector of the original CSI matrix for compression;

[0076] The decoding layer corresponds to the decoder part, which is used to restore the low-dimensional vector returned by the mobile terminal to the original CSI matrix.

[0077] Two small modules are added in the middle: one is the down sampling layer, and the other is the fully connected layer.

[0078] The downsampling layer is used to further compress the CSI matrix after feature extraction and reduce the number of parameters.

[0079] The fully connected layer is used to compress the high-dimensional vector into a low-dimensional vector on the mobile end and restore the low-dimensional vector to a high-dimensional vector on the base station end.

[0080] like Figure 2 As shown in the figure, assuming that the input matrix of the image is H, the encoder is composed of a cross-attention module (Criss-Cross attention) and a depth-separable convolution (Dpwise Conv).

[0081] After the input matrix H is subjected to a 2D Discrete Fourier Transform (2D DFT), near-zero elements are removed.

[0082] It is then converted into a feature vector through the encoder (Encoder end), and the feature vector is fed back to the base station end, and enters the decoder (Decoder end) through the wireless channel (Wireless channel). After passing through two modules CRBlock for CSI feedback, the activation function (Sigmoid) is input to obtain the final feedback matrix (Onput matrix), and the output feedback matrix is ​​padded with zeros and inverse discrete Fourier transform (Inverse Discrete Fourier Transform, IDFT) to obtain the final restored original CSI matrix.

[0083] For 5G and 6G scenarios, communication systems are expected to achieve low loss and low latency. In order to achieve low loss, in data-driven deep learning scenarios, the designed models often reduce losses at the expense of increasing computational complexity and parameters, or increase the amount of data for data enhancement to obtain data distribution that is closer to the ideal situation. However, a large number of parameters often brings difficulties to model deployment and increases the speed of model inference.

[0084] Moreover, for tasks such as channel state information feedback and channel estimation, the model deployed on the mobile terminal cannot be too large so as not to affect the operation of the device. The benefit of a small improvement in accuracy brought by an excessive number of parameters is often not greater than the impact of the difficulty in model deployment caused by an excessive number of parameters. For example, when the CRnet network feeds back the channel matrix, the recovery rate of the channel matrix at the base station can reach more than 90% under a delay of 1 / 4 and 1 / 8. On this basis, increasing the number of parameters to achieve the purpose of reducing losses is obviously not worth the cost.

[0085] For the feedback process of the entire channel state information CSI matrix, in the neural network based on the convolutional neural network (CNN), since the convolutional neural network has the characteristic of parameter sharing, the overall parameter amount is related to the number of network channels and the size of the convolution kernel. The size of the convolution kernel and the number of channels are often not too large in actual deployment. Therefore, after rough calculation, about 80% of the parameters of the neural network come from the fully connected layer that completes the compression and decompression module. Therefore, how to reduce the number of neurons in the fully connected layer and thus reduce the number of network parameters for easy deployment is one of the difficulties that the overall model needs to overcome.

[0086] In traditional image processing, the main downsampling methods used are pooling, increasing the convolution step size, and interval sampling.

[0087] The pooling process and the interval sampling process are irreversible. For the channel matrix, especially indoor situations, when very few response points have valuable information, the lack of any response point may have a great adverse effect on the subsequent decoder (Decoder). Therefore, the method of expanding the convolution step size is undoubtedly in line with the method of channel matrix related tasks. There are two main forms of expanding the convolution step size. One is similar to the way of constructing patches in VIT (Vision Transformer) and Swin Transformer. By moving a fixed-size convolution kernel with a step size greater than or equal to 2, the downsampling process is achieved. The other way is to ensure that one dimension does not change in the two-dimensional matrix, and the step size of another dimension is expanded to achieve the purpose of downsampling.

[0088] Unlike traditional RGB images, the channel matrix does not have strong semantic characteristics. For blocks of 3*3 or 5*5 size, it cannot provide much meaningful information. That is, the patching method is not suitable for communication tasks related to the channel matrix. In other words, it is very reasonable to find long-distance dependencies of fixed dimensions and use one dimension in a fixed two-dimensional matrix while downsampling another dimension. The more dimensions that can bring useful information to the encoder, the greater the fluctuation between different responses on it should be, and vice versa. Therefore, how to determine which dimension needs to be downsampled requires analyzing the correlation between different dimensions, that is, the invention needs to provide a distance calculation method to calculate the difference between each response of the channel matrix and its response score, and select the dimension for compression according to the size of the fluctuation in the carrier direction and the antenna direction.

[0089] Finally, let’s briefly describe the data set used in this invention. The collection of real channel information is often costly. At the same time, the real channel often changes with the influence of environmental factors and has uncertainty. Therefore, simulated channels are usually used for experiments. Next, two commonly used channel models are introduced: SCM channel model and COST2100 channel model.

[0090] The SCM channel model is a basic model based on the geometric distribution statistical channel model.

[0091] The SCM model is a public standard reference developed by 3GPP / 3GPP2 (3rd Generation Partnership Project) to evaluate Multiple Input Multiple Output (MIMO) in different outdoor environments. It is applicable to systems with 5MHZ bandwidth and 2GHZ carrier frequency.

[0092] The coefficients of the SCM channel model in the polarization channel are generated by the following formulas:

[0093]

[0094] In addition to local clusters, the COST2100 model also adds single reflection clusters and double clusters. It is a geometric random channel model that is extended on the basis of the earlier COST259 and COST273 models. Compared with the previous angle modeling, COST2100 proposes the concept of the visible area of ​​scattering clusters to describe the changes in the number of scattering clusters experienced by the mobile station at different locations, which further accurately describes the channel characteristics.

[0095] In some embodiments, the CSI matrix provided by the present invention is a data set generated by the COST2100 model.

[0096] The present invention adopts the COST2100 model as the source of partial data generation, and most of the existing channel state information feedback also adopts the COST2100 model as the benchmark model to generate data and then compare it with the SOTA method.

[0097] The CSI matrix visualized by the data set generated by the present invention under the COST2100 model is divided into two parts: real part and imaginary part. The overall generated data set can include indoor and outdoor situations. The indoor situation is used as a demonstration in the present invention.

[0098] like Figure 3 As shown, Figure 3This is the visualization of the real part of the CSI matrix generated under the COST2100 model. Since the real and imaginary parts of the matrix generated by the COST2100 model are highly similar, the imaginary part visualization is not shown here.

[0099] Reference Figure 3 Only a few responses of the generated CSI matrix are distinctive, that is, they are responses that can provide information at the encoder end for feature extraction for compression, while the fluctuation range of most other responses is not large, which can intuitively provide reliable evidence for the subsequent further compression of the channel matrix of the model to reduce the number of parameters.

[0100] The present invention will be introduced below from the traditional similarity evaluation method in combination with the relevant characteristics of the communication matrix, the core process part of the present invention will be introduced in detail, and the advantages of the present invention in time complexity will be analyzed.

[0101] First, we introduce the traditional similarity estimation method: The similarity estimation method is an important tool in data analysis. Analyzing the difference and correlation between data before modeling can help designers understand the numerical characteristics of the data and assist in the design of the overall model. Next, we introduce several mainstream similarity estimation methods, including: Euclidean distance, Manhattan distance, Mahalanobis distance and cosine similarity.

[0102] Euclidean distance is also called Euclidean distance. It measures the absolute distance between two points in multidimensional space. It is widely used in machine learning methods, including K-Nearest Neighbor (KNN) and K-means. Suppose two vectors X1 = (x1, x2, x3... x n ),Y1=(y1,y2,y3……y n ), then the Euclidean distance formula between two vectors is:

[0103]

[0104] Manhattan distance is used to describe geometric metric space, which reflects the sum of the axis distances of two points in the standard coordinate system.

[0105] Assume that the points in n-dimensional space are:

[0106] a(x 11 ,x 12 ,x 13 ……x 1n ) and b(x 21 ,x 22 ,x 23 ……x 2n ).

[0107] Then the Manhattan distance between points ab is:

[0108]

[0109] The Mahalanobis distance is a distance based on sample distribution, which represents the distance of the overall data covariance. Therefore, the Mahalanobis distance can effectively measure the similarity between two position sample sets.

[0110] Assume that the population G is an n-dimensional population, the covariance matrix is ​​∑, and the mean vector is μ = (μ1, μ2, μ3…μ n ), that is, sample X(x1, x2, x3…x n ) and the overall Mahalanobis distance G is:

[0111] d 2 (X, G) = (X-μ)'∑ -1 (X-μ)

[0112] For cosine similarity, the cosine value of the angle between two vectors in space is used as a measure of the size of the difference between two individuals.

[0113] In the communication system, the signal is represented by a complex value, that is, the real part and the imaginary part on the complex coordinate axis can represent the amplitude and phase of the vector. Using cosine similarity to measure the difference between the two is obviously more in line with the signal characteristics. Assuming that vector X1 is (x1, y1) and vector X2 is (x2, y2), the cosine similarity between the two vectors can be expressed as:

[0114]

[0115] According to the description of the similarity method above, cosine similarity is obviously the most suitable method for analyzing the correlation of channel matrix data, because it can measure the similarity of complex signals to the greatest extent. Figure 2 As in the analysis, the response at any point is not highly correlated with responses in non-rows and non-columns. When analyzing the relationship between different responses, the feature information between the same antenna and the same carrier (i.e., corresponding to the same row and column on the image) should be extracted as much as possible, rather than calculating the similarity between each response of the entire channel matrix and the remaining responses. Therefore, inspired by the cross attention module, the present invention provides a new data similarity judgment method suitable for communication tasks: a cross-cosine similarity evaluation standard. It is designed according to the characteristics of the channel matrix itself when it is generated, and can accurately extract the features between different responses of the entire channel matrix, thereby reflecting the correlation between different responses.

[0116] It can be seen from the above cosine similarity formula that its numerator is composed of the inner product between two vectors, while the denominator is composed of the sum of the absolute values ​​of the vectors. Inspired by the cross attention, and in order to facilitate the implementation of cross cosine similarity in the pytorch framework, the algorithm calculates rows and columns separately, and finally merges them to obtain a score, which is convenient for similarity analysis between the calculated response points and the remaining response points.

[0117] Different from directly calculating the score of each response point on the CSI matrix in the data correlation analysis by traversing the points in the same row and column, the embodiment of the present invention divides the overall cosine similarity calculation process into two parts, a numerator and a denominator, and splices them after calculation, specifically including:

[0118] 1. Calculating Molecules

[0119] For cosine similarity, the numerator is the inner product between two vectors, so simply copy the original matrix and transpose it, and then directly calculate the inner product between the two response values ​​of the transposed channel matrix and the real and imaginary parts of the original matrix. It is worth noting that both the numerator and the denominator need to be calculated separately from the same column. This is to facilitate the implementation of the similarity estimation function in the program.

[0120] In actual implementation, the dimension of any channel state information CSI matrix is ​​transformed to obtain the row vector and column vector corresponding to the CSI matrix, and the first matrix is ​​obtained based on the row vector and the column vector.

[0121] The first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column, which is the numerator.

[0122] 2. Calculate the denominator

[0123] For the denominator of the cosine similarity, the square root of the real and imaginary parts of the matrix is ​​first taken to obtain the amplitude, and then the amplitude matrix is ​​transposed and multiplied with the original matrix in rows and columns respectively to obtain the denominator of the cosine similarity.

[0124] In actual execution summary, the second matrix is ​​used to indicate the product of the amplitude of each response point in the CSI matrix and the remaining response points in the same row and column, that is, the denominator.

[0125] Based on the strong correlation between the channel state information matrix in the same row and column, a criterion for calculating the cosine similarity of images in the same row and column is proposed, which is different from the previous traditional image similarity judgment standard. To address the problem of high computational complexity, the computational complexity is cleverly reduced to O(n) through dimensional transformation of the channel state information matrix and matrix inner product.

[0126] The cross-attention-based channel information compression method provided by the present invention adopts a new data evaluation method for judging the correlation between different points on an image, calculates the cosine similarity between each response point and the other response points on the same row and column, and expands the data characteristics of all points in the same matrix by row and column to obtain the data characteristics of the channel state information matrix, thereby helping to enhance or compress data in communication tasks. The data evaluation method is a new method designed based on communication characteristics, which is more conducive to analyzing the characteristics of the channel data itself and thus helping the analysis of the channel data.

[0127] In some embodiments, the CSI matrix is ​​represented as c represents the number of input channels of the target neural network, h represents the number of antennas corresponding to the CSI matrix, and w represents the number of carriers corresponding to the CSI matrix;

[0128] The dimension of any channel state information CSI matrix is ​​transformed to obtain the row vector and column vector corresponding to the CSI matrix, including:

[0129] right After dimension transformation, we get and is a row vector, is a column vector;

[0130] Based on the row vector and column vector, the first matrix is ​​obtained, including:

[0131] right and Transform and get the transformed row vector and the transformed column vector

[0132] Determine the dot product of each response point in the CSI matrix with the rest of the response points in the same row And the dot product of each response point in the CSI matrix and the other response points in the same column

[0133] Keep F r The dimension of F remains unchanged. l The dimension is transformed into

[0134] F r and the transformed F l Merge to get

[0135] in, is the first matrix.

[0136] In actual implementation, assume that the CSI matrix obtained by random sampling is

[0137] Among them, c represents the number of input channels of the target neural network, h represents the number of antennas corresponding to the CSI matrix (i.e., the length on the image), and w represents the number of carriers corresponding to the CSI matrix (i.e., the width on the image).

[0138] After the two-dimensional signal is processed by discrete Fourier transform DFT, it corresponds to the time delay domain and the angle domain. In the simulation experiment of the present invention, the parameters are set as: c=2, h=w=32.

[0139] Next, the calculation process of the numerator and denominator will be explained separately:

[0140] Compared to calculating the dot product of a matrix or a single vector, there is no commonly used distance measurement method to directly calculate the row and column features, so the present invention separates the calculation steps into calculating the cosine similarity on the rows and the cosine similarity on the columns.

[0141] First, the CSI matrix is ​​transformed into and They represent column vectors and row vectors respectively.

[0142] Secondly, in order to facilitate the subsequent calculation of the dot product, the last two dimensions of the two matrices obtained will be exchanged to obtain two new matrices: and Multiply the corresponding row and column matrices respectively:

[0143]

[0144]

[0145] The two matrices obtained and They represent the dot product of each response point in the CSI matrix with the rest of the response points in the same row, and the dot product of each response point in the CSI matrix with the rest of the response points in the same column.

[0146] In order to facilitate the subsequent division with the denominator, it is necessary to connect the dot products of the rows and columns together to keep F r The dimension of F remains unchanged. l The dimension becomes

[0147] F r With F l Merge in dimension 3 to get the final dot product matrix on rows and columns:

[0148]

[0149] The F matrix represents the dot product of each of the (h+w) points on the CSI matrix with a total of (h+w) response points in the same row and column, that is, the dot product of each response point in the CSI matrix with the remaining response points in the same row and column.

[0150] The final F matrix will be used as the numerator of the cross cosine similarity.

[0151] Next, let's explain the calculation of the denominator:

[0152] In some embodiments, before determining the second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column, the method further includes:

[0153] Based on a preset algorithm, determine the product of the amplitude of each response point in the CSI matrix and the other response points in the same row and the product of the amplitude of each response point in the CSI matrix and the other response points in the same column;

[0154] Based on the product of the amplitudes of each response point in the CSI matrix and the other response points in the same row and the product of the amplitudes of each response point in the CSI matrix and the other response points in the same column, the product of the amplitudes of each response point in the CSI matrix and the other response points in the same row and column is determined.

[0155] For the denominator, it represents the product of the amplitudes of different vectors, so the algorithm first needs to calculate the amplitude of each response point on the channel matrix. Assume that H is the channel state information matrix to be analyzed, and the input matrix is ​​a dual channel representing the real part and the imaginary part:

[0156] H′=sqrt(H[0,:,:] 2 +H[1,:,:] 2 )

[0157] H′ represents the input of the preset algorithm for calculating the denominator. Similar to the numerator calculation part, the preset algorithm is divided into two parts, row and column, for calculation. The specific preset algorithm is implemented through two for loops.

[0158] Figure 4 A preset algorithm flow chart for applying the cross-attention based channel information compression method provided by the present invention.

[0159] like Figure 4 As shown in the figure, matrixc and matrixr represent the product of each response point with the amplitude of the same column and the amplitude of the same row respectively. Both matrices are obtained by multiplying the original channel state information matrix after different dimensional transformations. The final matrixf represents the amplitude product of each response point on the channel matrix with the other response points on the same row and column. Its dimension is The dimension of the inner product of the same row and column calculated in the above embodiment is the same.

[0160] The reason why the preset algorithm is divided into row and column calculations is not only for the convenience of implementation in pytorch, but also to ensure that the matrix calculated in the denominator part has the same dimension order as the matrix obtained by the inner product of the numerator part, that is, (h+w) means that the inner product of the rows and the product of the amplitude are calculated first, and then the inner product of the columns and the product of the amplitude are calculated, otherwise they cannot be divided subsequently.

[0161] Finally, the F matrix is ​​divided by the matrixf matrix to obtain the cross cosine similarity matrix.

[0162] Each response point is equivalent to the response score of the original channel matrix, and the score represents the average difference between the response of the same row and column and the response of the current calculation point. It should be noted that the purpose of the present invention is not only to look at the amplitude of any point in the final score matrix, but the volatility between different response value scores, because the data-driven deep learning method has shown great advantages over traditional algorithms in communication tasks, and how to reduce the size of the deep learning model requires analysis and design based on the correlation between data points.

[0163] The evaluation method of cross cosine similarity can exploit the inherent characteristics of the channel data itself, which is its unique advantage compared with the distance measurement method. Another advantage of the present invention is that its computational complexity is only at the O(n) level, which greatly reduces the inference delay during data analysis.

[0164] When calculating the cosine similarity of the same row and column, the present invention does not loop through each response point, but adopts the form of matrix dimension transformation and matrix inner product. The numerator part only needs transposition and inner product operations to complete, and the denominator part is calculated through two for loops. The calculation complexity is only O(n). Therefore, compared with the O(n^2) algorithm of traversing all response points and then traversing the same row and column to calculate the cosine similarity, this algorithm can better reduce the reasoning speed during data analysis, and has very obvious advantages for larger matrices or more data volumes.

[0165] Figure 5 and Figure 6 It is the result of expanding the score matrix along the rows and columns after performing cross cosine similarity calculation on the indoor channel matrix. As described in the above embodiment, the present invention uses the COST2100 model to generate the CSI matrix.

[0166] In the actual implementation process, it is divided into two scenarios: indoor scenario at 5GHz and outdoor rural scenario at 300MHZ.

[0167] A 2×1024×32 dimensional vector is generated through the MATLAB program. 2 represents the real part and the imaginary part, 1024 and 32 represent the number of subcarriers and the number of antennas, respectively. After 2D DFT transformation and compression, the CSI matrix becomes a 2×32×32 dimensional matrix in the angular delay domain.

[0168] The effectiveness of the channel state information matrix estimation algorithm is verified by comparing the cosine similarity matrix calculated by the CSI matrix in the indoor scene by expanding it by rows and columns respectively, and by comparing the cosine similarity matrix expanded by rows in the indoor and outdoor scenes. The embodiment of the present invention can be implemented by a python program.

[0169] Figure 5 and Figure 6 It shows that the estimated cross cosine similarity matrix is ​​expanded in rows and columns in indoor scenes.

[0170] from Figure 5 and Figure 6 It can be seen that the cosine similarity energy on the columns is more concentrated than that on the rows, which is consistent with the result of the discrete Fourier transform analysis of the channel state information matrix at the beginning (i.e. Figure 3 That is, the energy in the angular domain is concentrated because the angular power spectrum is assumed to be Laplace distributed.

[0171] As to Figure 3 According to the analysis, the purpose of the channel state information feedback task is to compress the feature map size and thus reduce the parameters of the fully connected layer. More attention should be paid to the volatility between different response values ​​to determine the dimensions that need to be compressed. Obviously, the energy in the angular domain is concentrated, and a large amount of information cannot provide enough features for restoring the channel matrix. Therefore, the final network model selects (1, 2) as the step size for convolution, thereby achieving the purpose of compression in the angular domain.

[0172] On the other hand, in indoor situations, there are many points where the average value of the cross cosine similarity is close to 1. By comparing the definition of cosine similarity, we can see that this is because most of the response values ​​are very similar. Figure 3 It can also be intuitively found that this also confirms the necessity of compressing the model from another perspective.

[0173] Figure 7 The comparison of indoor and outdoor data distribution is shown. The solid line and dotted line represent the cosine similarity of outdoor and indoor data, respectively. It can be seen that the cosine similarity fluctuation of outdoor data is much greater than that of indoor data. This is because there are more clusters outdoors that affect the channel matrix distribution, making its response more complex and changeable than that of indoor data. The indoor and outdoor differences estimated by this experimental result are consistent with the actual situation. Figure 5 and Figure 6From the analysis results, we can see that the average cosine similarity of indoor situations is close to 1, while most of the response values ​​of outdoor situations are far away from 1. This is because the more complex environment leads to differences in the response values ​​on the channel matrix. Therefore, compression is necessary for indoor situations, while for outdoor situations, it is necessary to balance accuracy and model size. Compression can be performed for low compression ratios, and for high compression ratios, if there are clear requirements for feedback accuracy or estimation accuracy, compression of the channel matrix should be avoided as much as possible.

[0174] In summary, the original purpose of the cross cosine similarity is to be able to more accurately analyze the data characteristics of the channel matrix. Because the channel matrix will be applied to most communication tasks, such as channel estimation, channel feedback, channel prediction, etc., the help of the present invention for data analysis before communication tasks is obvious.

[0175] The channel information compression device based on cross-attention provided by the present invention is described below. The channel information compression device based on cross-attention described below and the channel information compression method based on cross-attention described above can be referred to each other.

[0176] Figure 8 This is a schematic diagram of the structure of the channel information compression device based on cross attention provided by the present invention. Figure 8 The channel information compression device based on cross attention provided by the present invention comprises:

[0177] A first determination module 810 is used to perform dimension transformation on any channel state information CSI matrix to obtain a row vector and a column vector corresponding to the CSI matrix;

[0178] A second determination module 820 is used to obtain a first matrix based on the row vector and the column vector, where the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column;

[0179] A third determination module 830 is used to determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column;

[0180] The fourth determination module 840 is used to divide the first matrix and the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the remaining response points in the same row and column, wherein the cosine similarity matrix is ​​used to indicate the data characteristics of the CSI matrix.

[0181] The cross-attention based channel information compression device provided by the present invention calculates the cosine similarity between each response point and the other response points in the same row and column, and expands the data characteristics of all points in the same matrix by row and column to obtain the data characteristics of the channel state information matrix, thereby helping to enhance or compress data in communication tasks. The data evaluation method is a new method designed based on communication characteristics, which is more conducive to analyzing the characteristics of the channel data itself and thus helping the analysis of the channel data.

[0182] In some embodiments, the CSI matrix is ​​represented as c represents the number of input channels of the target neural network, h represents the number of antennas corresponding to the CSI matrix, and w represents the number of carriers corresponding to the CSI matrix;

[0183] The first determining module is further used for:

[0184] Regarding the After dimension transformation, we get and Said is the row vector, the is the column vector;

[0185] The second determining module is further used for:

[0186] Regarding the and stated Transform and get the transformed row vector and the transformed column vector

[0187] Determine the dot product of each response point in the CSI matrix with the rest of the response points in the same row And the dot product of each response point in the CSI matrix and the other response points in the same column

[0188] Keep the F r The dimension of F remains unchanged. l The dimension is transformed into

[0189] The F r and the transformed F l Merge to get

[0190] Among them, the is the first matrix.

[0191] In some embodiments, the apparatus further comprises:

[0192] A fifth determination module, configured to determine, based on a preset algorithm, the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same row and the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same column;

[0193] The sixth determination module is used to determine the product of the amplitude of each response point in the CSI matrix and the remaining response points in the same row and column based on the product of the amplitude of each response point in the CSI matrix and the remaining response points in the same row and the product of the amplitude of each response point in the CSI matrix and the remaining response points in the same column.

[0194] In some embodiments, the target neural network comprises:

[0195] A coding layer, used to extract a eigenvector of the CSI matrix to obtain a CSI matrix after feature extraction;

[0196] A downsampling layer, used to compress the CSI matrix after the feature extraction;

[0197] A fully connected layer, used to compress the high-dimensional vector into a low-dimensional vector at the mobile end, and restore the low-dimensional vector to the high-dimensional vector at the base station end;

[0198] The decoding layer is used to restore the low-dimensional vector returned by the mobile terminal into the CSI matrix.

[0199] In some embodiments, the CSI matrix is ​​a data set generated by a COST2100 model.

[0200] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the channel information compression method based on cross attention, and the method includes:

[0201] Performing dimension transformation on any channel state information CSI matrix to obtain row vectors and column vectors corresponding to the CSI matrix;

[0202] Based on the row vector and the column vector, a first matrix is ​​obtained, where the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column;

[0203] Determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column;

[0204] The first matrix is ​​divided by the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the other response points in the same row and column, where the cosine similarity matrix is ​​used to indicate data characteristics of the CSI matrix.

[0205] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0206] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the channel information compression method based on cross attention provided by the above methods, the method comprising:

[0207] Performing dimension transformation on any channel state information CSI matrix to obtain row vectors and column vectors corresponding to the CSI matrix;

[0208] Based on the row vector and the column vector, a first matrix is ​​obtained, where the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column;

[0209] Determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column;

[0210] The first matrix is ​​divided by the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the other response points in the same row and column, where the cosine similarity matrix is ​​used to indicate data characteristics of the CSI matrix.

[0211] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the channel information compression method based on cross attention provided by the above methods, the method comprising:

[0212] Performing dimension transformation on any channel state information CSI matrix to obtain row vectors and column vectors corresponding to the CSI matrix;

[0213] Based on the row vector and the column vector, a first matrix is ​​obtained, where the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column;

[0214] Determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column;

[0215] The first matrix is ​​divided by the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the other response points in the same row and column, where the cosine similarity matrix is ​​used to indicate data characteristics of the CSI matrix.

[0216] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0217] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0218] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A channel information compression method based on cross attention, characterized in that: include: Performing dimension transformation on any channel state information CSI matrix to obtain row vectors and column vectors corresponding to the CSI matrix; Based on the row vector and the column vector, a first matrix is ​​obtained, where the first matrix is ​​used to indicate the dot product of each response point in the CSI matrix with the remaining response points in the same row and column; Determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column; Dividing the first matrix by the second matrix, to obtain a cosine similarity matrix between each response point in the CSI matrix and the remaining response points in the same row and column, the cosine similarity matrix being used to indicate data characteristics of the CSI matrix; The CSI matrix is ​​expressed as c represents the number of input channels of the target neural network, h represents the number of antennas corresponding to the CSI matrix, and w represents the number of carriers corresponding to the CSI matrix; The performing dimension transformation on any channel state information CSI matrix to obtain a row vector and a column vector corresponding to the CSI matrix comprises: Regarding the After dimension transformation, we get and Said is the row vector, the is the column vector; The step of obtaining a first matrix based on the row vector and the column vector includes: Regarding the and stated Transform and get the transformed row vector and the transformed column vector Determine the dot product of each response point in the CSI matrix with the rest of the response points in the same row And the dot product of each response point in the CSI matrix and the other response points in the same column Keep the F r The dimension of F remains unchanged. l The dimension is transformed into The F r and the transformed F l Merge to get Among them, the is the first matrix.

2. The channel information compression method based on cross attention according to claim 1 is characterized in that: Before determining the second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column, the method further includes: Based on a preset algorithm, determine the product of the amplitude of each response point in the CSI matrix and the other response points in the same row and the product of the amplitude of each response point in the CSI matrix and the other response points in the same column; Based on the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same row and the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same column, the product of the amplitudes of each response point in the CSI matrix and the remaining response points in the same row and column is determined.

3. The channel information compression method based on cross attention according to claim 1 is characterized in that: The target neural network comprises: A coding layer, used to extract a eigenvector of the CSI matrix to obtain a CSI matrix after feature extraction; A downsampling layer, used to compress the CSI matrix after the feature extraction; A fully connected layer, used to compress the high-dimensional vector into a low-dimensional vector at the mobile end, and restore the low-dimensional vector to the high-dimensional vector at the base station end; The decoding layer is used to restore the low-dimensional vector returned by the mobile terminal into the CSI matrix.

4. The channel information compression method based on cross attention according to any one of claims 1 to 3, characterized in that: The CSI matrix is ​​a data set generated by the COST2100 model.

5. A channel information compression device based on cross attention, characterized in that: include: A first determination module is used to perform dimension transformation on any channel state information CSI matrix to obtain a row vector and a column vector corresponding to the CSI matrix; A second determination module, configured to obtain a first matrix based on the row vector and the column vector, wherein the first matrix is ​​used to indicate a dot product of each response point in the CSI matrix with other response points in the same row and column; A third determination module, configured to determine a second matrix based on the product of the amplitude of each response point in the CSI matrix and the amplitude of the remaining response points in the same row and column; a fourth determination module, configured to divide the first matrix by the second matrix to obtain a cosine similarity matrix between each response point in the CSI matrix and the remaining response points in the same row and column, wherein the cosine similarity matrix is ​​used to indicate data characteristics of the CSI matrix; The CSI matrix is ​​expressed as c represents the number of input channels of the target neural network, h represents the number of antennas corresponding to the CSI matrix, and w represents the number of carriers corresponding to the CSI matrix; The first determining module is further used for: Regarding the After dimension transformation, we get and Said is the row vector, the is the column vector; The second determining module is further used for: Regarding the and stated Transform and get the transformed row vector and the transformed column vector Determine the dot product of each response point in the CSI matrix with the rest of the response points in the same row And the dot product of each response point in the CSI matrix and the other response points in the same column Keep the F r The dimension of F remains unchanged. l The dimension is transformed into The F r and the transformed F l Merge to get Among them, the is the first matrix.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the channel information compression method based on cross attention as described in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the channel information compression method based on cross attention as described in any one of claims 1 to 4 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the channel information compression method based on cross attention as described in any one of claims 1 to 4 is implemented.

Images