Cross attention mechanism-based sensing integrated beam forming method and medium

Abstract

The invention relates to a cross attention mechanism-based communication and sensing integrated beam forming method and a medium, and the method comprises the following steps: obtaining a channel state information matrix and a target response matrix, inputting a communication and sensing integrated beam forming model based on a Transform architecture, and obtaining a beam forming matrix; wherein the transformer architecture-based general sensing integrated beam forming model comprises a coding module, a general sensing feature fusion module and a decoding module which are connected in sequence; the coding module is used for preliminarily extracting features of the channel state information matrix and the target response matrix; the sensing feature fusion module is used for performing interaction enhancement on the communication features and the sensing features; and the decoding module is used for further fusing the communication features and the sensing features after interaction enhancement, and obtaining a reconstructed beam forming matrix based on a fusion result and the randomly initialized beam forming matrix. Compared with the prior art, the fusion degree of communication and sensing information can be improved, and more accurate beam forming is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communications, and in particular relates to a synaesthesia integrated beamforming method and medium based on a cross-attention mechanism. Background Art

[0002] In the development of mobile communication technology from 2G to 5G, the improvement of communication performance has always received special attention, while the perception ability of radio frequency electromagnetic waves itself has not been deeply explored and applied. With the increase of communication frequency, radar perception and communication have gradually converged in spectrum resources, equipment form and transmission technology, laying the foundation for the integrated design of communication and perception.

[0003] Millimeter waves have short wavelengths, extremely wide spectrum resources and high bandwidth capabilities, and can meet the needs of applications such as enhanced mobile broadband and ultra-reliable low-latency communications in future 6G and subsequent networks. In the integrated telepathy scenario, the large bandwidth of the millimeter wave band supports high-speed transmission of large amounts of data, which helps to improve the capacity and anti-interference capability of the communication link; the high directivity and narrow beam characteristics can achieve precise beamforming, significantly improving the accuracy of environmental perception, and are suitable for application scenarios that require precise detection and positioning.

[0004] However, millimeter waves also face challenges such as weak penetration and limited transmission distance, and more complex physical layer technologies are needed to compensate for transmission losses. The combination of massive MIMO and beamforming technology can provide high-directivity, high-gain beams, which helps to overcome many non-ideal factors in millimeter wave signal transmission. Specifically, massive MIMO technology can suppress interference between users on the same time-frequency resources by deploying large-scale antenna arrays at base stations, and achieve a significant increase in network capacity without increasing bandwidth and base station density; beamforming technology, as a native capability of massive MIMO, is the basis for achieving joint performance optimization of communication and perception. The design of the beamforming matrix can be modeled as an optimization problem. Since multiple constraints need to be met at the same time, such as system total power constraints, maximum communication rate, physical layer security constraints, etc., non-convexity will be introduced into the beamforming optimization problem. The high-frequency characteristics and multipath effects of millimeter wave signals will further increase the complexity of the objective function and bring new challenges to the solution of the optimization problem. Therefore, it is necessary to redesign a synaesthesia integrated beamforming method to improve the degree of fusion of communication and perception information and achieve more accurate beamforming. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a synaesthesia integrated beamforming method and medium based on the cross-attention mechanism, so as to improve the degree of fusion of communication and perception information and achieve more accurate beamforming.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a synaesthesia integrated beamforming method based on a cross-attention mechanism, which is applied to a millimeter-wave large-scale MIMO synaesthesia integrated system, comprising the following steps:

[0008] The channel state information matrix of the downlink communication user and the target response matrix of the potential sensing target are obtained, and input into the synaesthesia integrated beamforming model based on the Transformer architecture to obtain the beamforming matrix;

[0009] The synaesthesia integrated beamforming model based on the Transformer architecture includes an encoding module, a synaesthesia feature fusion module and a decoding module connected in sequence;

[0010] The encoding module includes two stacked Transformer encoders based on a multi-head attention mechanism, which are used to perform preliminary feature extraction on the channel state information matrix and the target response matrix to obtain communication features and perception features;

[0011] The synaesthesia feature fusion module includes two parallel feature enhancement structures based on a cross-attention mechanism, which are used to interactively enhance the communication feature and the perception feature;

[0012] The decoding module includes a superimposed Transformer decoder based on a masked multi-head attention mechanism and a Transformer decoder based on a multi-head cross attention mechanism, which are used to further fuse the communication features and perception features after interaction enhancement, and obtain a reconstructed beamforming matrix based on the fusion result and a randomly initialized beamforming matrix as the final output.

[0013] Furthermore, the channel state information matrix and the target response matrix are both vectorized and position-coded and then input into the synaesthesia integrated beamforming model based on the Transformer architecture.

[0014] Furthermore, when the channel state information matrix and the target response matrix are vectorized, the real part and the imaginary part of each matrix are separated respectively, and the real number matrix is ​​reconstructed.

[0015] Furthermore, the channel state information matrix and the target response matrix are position-coded by the following formula:

[0016]

[0017]

[0018] Among them, PE represents position encoding, pos represents the position index of input data, i represents dimension index, and d model is an adjustable hyperparameter representing the model dimension.

[0019] Furthermore, in the synaesthesia feature fusion module, one feature enhancement structure uses the communication feature as the value matrix and the perception feature as the query matrix and the key matrix; another feature enhancement structure uses the perception feature as the value matrix and the communication feature as the query matrix and the key matrix.

[0020] Furthermore, in the decoding module, the interactively enhanced communication features and perception features are concatenated through the Concat function as the query matrix and key matrix of the Transformer decoder based on the multi-head cross attention mechanism; the randomly initialized beamforming matrix calculated by the Transformer decoder based on the masked multi-head attention mechanism is used as the value matrix of the Transformer decoder based on the multi-head cross attention mechanism; the Transformer decoder based on the multi-head cross attention mechanism performs feature fusion based on its value matrix, query matrix and key matrix to obtain a reconstructed beamforming matrix.

[0021] Furthermore, the Transformer architecture-based synaesthesia integrated beamforming model is trained in an offline training manner. During the training process, a loss function is constructed based on a weighted integration index and a penalty term. The weighted integration index is the weighted sum of a communication evaluation index and a perception evaluation index. The communication evaluation index is the sum of the achievable rates of communication users, and the perception evaluation index is the mutual information between the echo signal received by the base station and the target response matrix.

[0022] Furthermore, the penalty term is used to constrain the total power of the system, and the penalty term introduces a penalty parameter determined based on penalty convergence theory.

[0023] Furthermore, the specific expression of the loss function is as follows:

[0024]

[0025] Among them, w is the beamforming matrix to be optimized, K is the number of communication targets, R k is the sum of the achievable rates of the communication users, I(y s ; g S |X=WS) is the mutual information between the target response matrices, α k and α s R k and I(y s ; g S |X=WS) weight, λ is the penalty parameter, P0 The total power limit of the system.

[0026] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. The present invention proposes a synaesthesia integrated beamforming method based on a cross-attention mechanism for use in a millimeter-wave large-scale MIMO synaesthesia integrated system, and designs a synaesthesia integrated beamforming model based on a Transformer architecture. The model takes the channel state information matrix of the downlink communication user and the target response matrix of the potential perception target as input, and can output a reconstructed beamforming matrix. Specifically, the model includes a sequentially connected encoding module, a synaesthesia feature fusion module, and a decoding module, wherein the encoding module includes two stacked Transformer encoders based on a multi-head attention mechanism, which are used to perform preliminary feature extraction on the channel state information matrix and the target response matrix to obtain communication features and perception features; the synaesthesia feature fusion module includes two parallel feature enhancement structures based on a cross-attention mechanism, which are used to interactively enhance communication features and perception features. This module can promote deep interaction and joint processing of communication and perception, and improve the degree of fusion of communication and perception information; the decoding module includes a superimposed Transformer decoder based on a masked multi-head attention mechanism and a multi-head cross-attention mechanism. The Transformer decoder is used to further fuse the communication features and perception features after interaction enhancement. The reconstructed beamforming matrix is ​​obtained as the final output based on the fusion results and the randomly initialized beamforming matrix. The Transformer architecture realizes global information processing through the self-attention mechanism, and has superior multimodal fusion and feature representation capabilities, powerful parallel computing capabilities and excellent generalization performance. In the synaesthesia integration scenario, there is a complex mapping relationship between input features such as the channel state information matrix and the target response matrix and the beamforming matrix. The self-attention mechanism can accurately capture the long-range dependency between input features while maintaining computational efficiency, thereby achieving more accurate and efficient beamforming matrix design.

[0029] 2. The present invention vectorizes and position-encodes the channel state information matrix and the target response matrix and then inputs them into the synaesthesia integrated beamforming model based on the Transformer architecture, which can retain the spatial position information and enhance the model's perception of spatial features.

[0030] 3. The present invention uses the sum of the achievable rates of communication users as the communication evaluation index, and uses the mutual information between the echo signal received by the base station and the target response matrix as the perception evaluation index. Using mutual information as a perception evaluation index can effectively suppress interference and improve the robustness and stability of the system. At the same time, the mutual information has the same dimension as the communication achievable rate, and is more suitable for the design and research of weighted integrated indicators. The present invention constructs a weighted integrated indicator based on the weighted sum of the communication evaluation index and the perception evaluation index, which can reflect the total performance of communication and perception and flexibly adapt to more scenario requirements; based on the weighted integrated index and the penalty term, the loss function is constructed, and the base station end transmit beamforming design can be realized under the condition of meeting the system total power constraint. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the structure of the synaesthesia integrated beamforming model based on the Transformer architecture.

[0032] Wherein, CSI represents channel state information, TRG represents target response matrix;

[0033] Figure 2 This is a schematic diagram of the structure of the synaesthesia feature fusion module.

[0034] Among them, d model represents the model dimension, N represents the vector dimension after position encoding, Q, K, and V represent the query matrix, key matrix, and value matrix respectively;

[0035] Figure 3 Schematic diagram of the embodiment method. DETAILED DESCRIPTION

[0036] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0037] Example 1

[0038] In the synaesthesia integration scenario, there is a complex mapping relationship between input features such as Channel State Information (CSI) and Target Response Matrix (TRM) and the beamforming matrix. This embodiment provides a synaesthesia integration beamforming method that can be applied to the millimeter wave large-scale MIMO synaesthesia integration system. The deep fusion and efficient processing of communication and perception information are achieved through the Transformer deep learning framework based on the cross-attention mechanism; at the same time, mutual information is introduced into the perception evaluation index to further improve the robustness and accuracy of the system, and the needs of different synaesthesia integration scenarios are flexibly adapted through custom loss functions and adjustable weights. Figure 3 As shown, it mainly includes the following steps:

[0039] S1. Establish a millimeter-wave massive MIMO interaceptive integrated system, set the position of communication users and sensing targets with the base station as the origin, calculate the steering vector of the antenna array, and use the sparse scattering characteristic structure of the millimeter-wave channel to construct the CSI matrix and TRG of the downlink.

[0040] S2. Construct data set and weighted integrated indicators.

[0041] S3. Preprocess the CSI matrix and TRG.

[0042] S4. Construct an encoding module, including two serially connected Transformer encoders and a fully connected layer. Each encoder includes a multi-head attention layer and a feedforward layer.

[0043] S5. Construct a synaesthesia feature fusion module, including two parallel superposition encoders, a cross-attention based feature enhancement structure, and a multi-head attention fusion structure.

[0044] S6. Construct a decoding module, including two serially connected Transformer decoders and a fully connected layer. Each decoder includes a masked multi-head attention layer, a multi-head attention layer, and a feedforward layer.

[0045] S7. Build a synaesthesia integrated beamforming model based on the Transformer architecture, and use a custom loss function to train the above model, with the maximization of the integrated system evaluation index as the optimization goal, and adjust the model's hyperparameters to adapt to different synaesthesia integration scenario requirements.

[0046] S8. Combine the real and imaginary parts of the vector obtained in step S7 and normalize them to construct a final beamforming matrix and calculate the corresponding system index.

[0047] S9. Use the test set to perform the final performance evaluation and analyze the performance of the model in different scenarios, such as the performance changes under different numbers of users, numbers of targets, signal-to-noise ratios, etc., to evaluate the generalization ability and robustness of the model.

[0048] The following is the specific implementation process:

[0049] S1. First, establish a millimeter wave large-scale MIMO interawareness integrated system. The system is equipped with N T Transmitting antennas and N R Root receiving antennas, using the uniform linear array (ULA) arrangement. While detecting M sensing targets, K equipped with N UE To ensure the feasibility of beamforming design, K≤N should be satisfied. T <N R In order to ensure the performance of the radar, an additional structural extension spatial degree of freedom is introduced into the transmitted signal X. The extended signal is dedicated to the radar detection signal and does not carry communication data. Transmitted signal is a linear combination of the communication signal and the radar detection signal, that is:

[0050]

[0051] In formula (1), and denote the communication beamforming matrix and the auxiliary beamforming matrix of the sounding stream, respectively, is the unit power data stream transmitted to K users in L time slots, Represents the radar detection signal. Both the communication signal and the radar detection signal are wide-sense stationary random processes with zero mean and unit power. The communication data signals of different users are unrelated, so The radar detection signal is a pseudo-random sequence with zero mean and unit variance, which is uncorrelated with each other.

[0052] Then, the communication signal model is set. The signal X is transmitted to K downlink communication users, and the signal matrix received by the user receiving end is:

[0053] Y C =HX+Z C =HW C S C +HW s S S +Z C (2)

[0054] For the kth user, the received signal is:

[0055]

[0056] In formula (2), Represents the CSI matrix between the base station and the communication user. Assuming that the base station channel estimation is ideal, an accurate CSI matrix can be obtained. In formula (3), the first term represents the ideal signal, the second term represents the self-interference of the communication user, the third term represents the interference of perception to communication, and the fourth term represents the additive white Gaussian noise (AWGN).

[0057] Considering the sparse scattering characteristics of the millimeter wave channel, each scatterer contributes a propagation path between the base station and the target, that is, there is a line-of-sight channel (line-of-sight, LoS) and K+M-1 non-line-of-sight channels (nLoS). The CSI matrix H is the superposition of K+M multipath channels, that is:

[0058]

[0059] In formula (4), is satisfied The normalization factor, α l is the complex gain of the lth path, which obeys the complex Gaussian distribution a t (θ t,l ) and a r (θ r,l ) represent the antenna array responses at the transmitting end and the receiving end, θ t,l and θ r,l are the arrival angle and departure angle of the lth path respectively. Taking the midpoint of the transmitting and receiving antennas as reference positions, the array steering vectors of the transmitting and receiving antennas are obtained as follows:

[0060]

[0061]

[0062] In equations (5) and (6), λ is the signal wavelength, and d represents the distance between adjacent antennas, which is half the wavelength.

[0063] Next, a multi-user perception model is set. In the perception model, the base station is regarded as a co-located massive MIMO radar, and the departure angle and arrival angle of each signal are the same. The interference echo signal generated by the communication user will affect the accuracy of radar target estimation and detection. The base station simultaneously receives the reflected signals of the communication user's target scatterer and other perception target scatterers. The received echo signal expression is:

[0064]

[0065] In formula (7), and Represent the TRM matrices of communication and perception respectively. The first term of the signal expression represents the ideal echo signal, the second term represents the echo of the communication beam at the perception target, the third term represents the echo of the perception beam at the communication target, the fourth term represents the echo of the communication beam at the communication target, and the last term represents the AWGN matrix

[0066] The extended scatterer expression of K communication targets and M sensing targets is:

[0067]

[0068]

[0069] In formula (8) and (9), G C represents the communication extended scatterer, G S represents the perceived extended scatterer, α i and θ i are the reflection coefficient and arrival angle / departure angle of the i-th point scatterer, and represents the steering vectors of the transmit array and the receive array.

[0070] S2. Change the relative position and system parameters, repeat the operation of step S1, obtain multiple sets of corresponding steering vectors, CSI matrices and TRGs, and form a data set for training. Separate the real and imaginary parts of the original CSI matrix and TRG, stack and divide them as training sets and verification sets for training and verification in subsequent steps.

[0071] Then, an optimization problem is designed. The angle of maximizing the communication transmission rate and the weighted sum of the perceptual mutual information is optimized jointly under the consideration of the system power constraint. The optimization problem thus constructed is:

[0072]

[0073] In order to intuitively display the impact of weight coefficient changes on the system, set

[0074] The unconstrained optimization problem after introducing the penalty term is:

[0075]

[0076] In formula (11), λ is the penalty parameter selected based on the penalty convergence theory. Gradually adjusting the penalty parameter can gradually approach the optimal solution while maintaining computational stability.

[0077] The synaesthesia weight coefficient is designed according to the system's requirements for communication and perception functions, and a weighted integrated index is constructed. Specifically, the sum of the achievable rates of communication users is used as the communication evaluation index, and the mutual information between the echo signal received by the base station and the target response matrix is ​​used as the perception evaluation index. The SINR of the kth communication user k And the achievable rate expression R k They are:

[0078]

[0079] R k =log 2 (1+SINR k ) (13)

[0080] The performance indicator of radar perception is the mutual information between the base station echo signal and TRM. Before calculating the mutual information, the echo signal and the perceived target TRM need to be converted into a vector y S and g S The base station knows the transmitted signal, the mutual information expression between the echo signal and TRM is:

[0081]

[0082] In formula (14), represents differential entropy, and p(·) represents probability density function.

[0083] S3, separate the real and imaginary parts of the CSI matrix and TRG, and re-stack them to construct a real matrix. Vectorize the real matrix, adjust the dimensions of the vectors to be consistent, and then use the encoding module to perform preliminary feature extraction on the CSI matrix and TRM respectively. The specific process includes:

[0084] Add positional encoding. The formula for calculating positional encoding is:

[0085]

[0086]

[0087] In formulas (15) and (16), PE represents positional encoding, pos represents the position index of the input data, i represents the dimension index, and d model is an adjustable hyperparameter representing the model dimension.

[0088] S4. Construct an encoding module, including two serially connected Transformer encoders and a fully connected layer. Each encoder includes a multi-head attention layer and a feedforward layer.

[0089] The multi-head self-attention mechanism focuses on different features of the input sequence in different representation subspaces. There are a total of h attention heads, each with an independent query matrix Key-value matrix Sum weight matrix i=1,2,...,h. For each head i, the self-attention weight is calculated as:

[0090]

[0091] In formula (17), X is the representation matrix of the input sequence, Denotes the query, key, and value weight matrices of the i-th head.

[0092] The output of the multi-head attention is combined with the output of all attention heads through a linear transformation, namely:

[0093] MultiHead(Q,K,V)=Concat(head 1 ,head 2 ,...,head h )W O (18)

[0094] In formula (18), Concat(·) represents concatenation, W O Is a linear transformation matrix used to map the concatenated output back to the original dimension.

[0095] Then, the nonlinear transformation capability is increased through the feedforward neural network, and the expression is:

[0096] FFN(x)=max(0,x*W 1 +b 1 )W 2 +b 2 (19)

[0097] In formula (19), W 1 and W 2 is the linear transformation weight matrix, b 1 and b 2 is the bias vector.

[0098] Again, add residual connections and layer normalization.

[0099] Repeat the above steps to stack two Transformer encoders based on the multi-head attention mechanism, obtain the features after the CSI matrix and TRM preliminary processing, and obtain the communication features and perception features.

[0100] S5. Construct Figure 2 The synaesthesia feature fusion module shown in the figure includes two parallel superposition encoders, a feature enhancement structure based on cross attention, and a multi-head attention fusion structure, which interactively enhances the communication features and the perception features. The specific process includes:

[0101] S501, realize mutual enhancement of features through cross attention. After the initial processing of CSI and TRM, the features are enhanced in parallel, one of which uses the features of CSI as the query matrix and key matrix, and the features of TRM as the value matrix; the other uses the features of TRM as the query matrix and key matrix, and the features of CSI as the value matrix. The expressions are:

[0102]

[0103]

[0104] S502: Stack the mutually enhanced CSI and TRM features, and the expression is:

[0105] head ISAC =Concat(head com ,head sen ) (twenty two)

[0106] S6. Construct a decoding module, including two serially connected Transformer decoders and a fully connected layer. Each decoder includes a masked multi-head attention layer, a multi-head cross attention layer, and a feedforward layer.

[0107] The decoder used in the decoding module adds a masked multi-head attention layer and a cross multi-head attention layer on the basis of the encoder, and the input is a randomly initialized beamforming matrix. The specific process includes:

[0108] S601, the masked multi-head attention layer is used to ensure that the current position can only access the previous position information. The expression of the masked attention layer output is:

[0109]

[0110] In formula (23), M is the mask matrix. If time step i cannot see time step j, then M[i,j] = -∞, otherwise it is 0.

[0111] S602, head ISACAs the query matrix and key matrix of the multi-head cross attention layer, the beamforming matrix after the masked multi-head attention layer is used as the value matrix of the multi-head cross attention layer. The expression for feature fusion of the multi-head cross attention layer is:

[0112] CrossMultiHead(Q ISAC ,K ISAC ,V BF )=Concat(head 1 ,head 2 ,...,head h )W O (twenty four)

[0113] S7, according to the three modules constructed in steps S5-S7, establish Figure 1 The Transformer-based synaesthesia integrated beamforming model shown is trained with the optimization objective constructed in step S2, and the hyperparameters of the model are adjusted to adapt to different synaesthesia integrated scenario requirements.

[0114] S8. Combine the real and imaginary parts of the vector obtained in step S7 and normalize them to construct a final beamforming matrix and calculate the corresponding system index.

[0115] S9. Use the test set to conduct the final performance evaluation, analyze the performance of the model in different scenarios, such as the performance changes under different numbers of users, numbers of targets, signal-to-noise ratios, etc., and evaluate the generalization and robustness of the model. The model uses offline training and online calculation. During offline training, the weights of the integrated system evaluation indicators are modulated according to the emphasis on communication and perception. The historical system information is used as the data set to train the model. After the model training is completed, the synaesthesia integrated beamforming matrix is ​​calculated online.

[0116] Compared with the prior art, the above method has the following advantages:

[0117] (1) The mutual information between the echo and target response matrix and the achievable rate of the communication user are selected to jointly construct a weighted integrated index. Using mutual information as a perception evaluation index can effectively suppress interference and improve the robustness and stability of the system. At the same time, the mutual information and the achievable rate of communication have the same dimension, which is more suitable for the design and research of integrated indicators. The weighted integrated index can adjust the weight according to different system requirements, reflect the total performance of communication and perception, and flexibly adapt to more scenario requirements.

[0118] (2) Use the Transformer-based deep learning architecture for beamforming design. The Transformer architecture implements global information processing through the self-attention mechanism and focuses on multiple features of the input through the multi-head attention mechanism. It has excellent multimodal fusion and feature representation capabilities, powerful parallel computing capabilities and excellent generalization performance, which can effectively solve the computational delay problem caused by traditional beamforming design problems.

[0119] (3) A feature fusion module based on the cross-attention mechanism is designed. After the encoder extracts the features of the communication channel state information and the perception target response matrix, two parallel multi-head cross-attentions are used to achieve feature enhancement, and then the fusion and reconstruction are performed in the decoder to obtain the beamforming matrix. This fusion method increases the interaction between communication and perception, fully utilizes system information for beamforming design, and improves the efficiency and accuracy of the system.

[0120] Example 2

[0121] This embodiment provides an electronic device, including a memory and a processor, wherein the processor is used to execute a program stored in the memory, wherein the program includes a number of instructions and can execute all or part of the steps of the method described in Example 1. The memory includes a computer-readable storage medium, which can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0122] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A synaesthesia integrated beamforming method based on a cross-attention mechanism, applied to a millimeter-wave massive MIMO synaesthesia integrated system, characterized in that: The following steps are involved: The channel state information matrix of the downlink communication user and the target response matrix of the potential sensing target are obtained, and input into the synaesthesia integrated beamforming model based on the Transformer architecture to obtain the beamforming matrix; The synaesthesia integrated beamforming model based on the Transformer architecture includes an encoding module, a synaesthesia feature fusion module and a decoding module connected in sequence; The encoding module includes two stacked Transformer encoders based on a multi-head attention mechanism, which are used to perform preliminary feature extraction on the channel state information matrix and the target response matrix to obtain communication features and perception features; The synaesthesia feature fusion module includes two parallel feature enhancement structures based on a cross-attention mechanism, which are used to interactively enhance the communication feature and the perception feature; The decoding module includes a superimposed Transformer decoder based on a masked multi-head attention mechanism and a Transformer decoder based on a multi-head cross attention mechanism, which are used to further fuse the communication features and perception features after interaction enhancement, and obtain a reconstructed beamforming matrix based on the fusion result and a randomly initialized beamforming matrix as the final output.

2. According to the cross-attention mechanism-based synaesthesia integrated beamforming method of claim 1, it is characterized in that: After the channel state information matrix and the target response matrix are both vectorized and position-coded, they are input into the synaesthesia integrated beamforming model based on the Transformer architecture.

3. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 2 is characterized in that: When the channel state information matrix and the target response matrix are vectorized, the real part and the imaginary part of each matrix are separated respectively, and the real number matrix is ​​reconstructed.

4. According to claim 2, the synaesthesia integrated beamforming method based on the cross attention mechanism is characterized in that: The channel state information matrix and the target response matrix are position-encoded by the following formula: Among them, PE represents position encoding, pos represents the position index of input data, i represents dimension index, and d model is an adjustable hyperparameter representing the model dimension.

5. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 1 is characterized in that: In the synaesthesia feature fusion module, one feature enhancement structure uses the communication feature as a value matrix and the perception feature as a query matrix and a key matrix; another feature enhancement structure uses the perception feature as a value matrix and the communication feature as a query matrix and a key matrix.

6. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 1 is characterized in that: In the decoding module, the interactively enhanced communication features and perception features are concatenated through the Concat function as the query matrix and key matrix of the Transformer decoder based on the multi-head cross attention mechanism; the randomly initialized beamforming matrix calculated by the Transformer decoder based on the masked multi-head attention mechanism is used as the value matrix of the Transformer decoder based on the multi-head cross attention mechanism; the Transformer decoder based on the multi-head cross attention mechanism performs feature fusion based on its value matrix, query matrix and key matrix to obtain a reconstructed beamforming matrix.

7. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 1 is characterized in that: The Transformer architecture-based synaesthesia integrated beamforming model is trained in an offline training manner. During the training process, a loss function is constructed based on a weighted integration index and a penalty term. The weighted integration index is the weighted sum of a communication evaluation index and a perception evaluation index. The communication evaluation index is the sum of the achievable rates of communication users, and the perception evaluation index is the mutual information between the echo signal received by the base station and the target response matrix.

8. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 7 is characterized in that: The penalty term is used to constrain the total power of the system, and the penalty term introduces a penalty parameter determined based on penalty convergence theory.

9. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 8, characterized in that: The specific expression of the loss function is as follows: Among them, w is the beamforming matrix to be optimized, K is the number of communication targets, R k is the sum of the achievable rates of the communication users, I(y S ; g S |X=WS) is the mutual information between the target response matrices, α k and α s R k and I(y S ; g S |X=WS), λ is the penalty parameter, and P0 is the upper limit of the total system power.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images