Synaesthesia integrated beamforming method and medium based on cross-attention mechanism

Abstract

The present invention relates to a synaesthesia integrated beamforming method and medium based on a cross-attention mechanism. The method comprises the following steps: obtaining a channel state information matrix and a target response matrix, inputting them into a synaesthesia integrated beamforming model based on a Transformer architecture, and obtaining a beamforming matrix; wherein the synaesthesia integrated beamforming model based on the Transformer architecture comprises an encoding module, a synaesthesia feature fusion module, and a decoding module connected in sequence; the encoding module is used to perform preliminary feature extraction on the channel state information matrix and the target response matrix; the synaesthesia feature fusion module is used to interactively enhance the communication features and the perception features; the decoding module is used to further fuse the interactively enhanced communication features and perception features, and obtain a reconstructed beamforming matrix based on the fusion results and a randomly initialized beamforming matrix. Compared with the prior art, the present invention can improve the degree of fusion of communication and perception information and achieve more accurate beamforming.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communications, and in particular relates to a synaesthesia integrated beamforming method and medium based on a cross-attention mechanism. Background Art

[0002] In the evolution of mobile communications technology from 2G to 5G, improvements in communication performance have received significant attention, while the inherent sensing capabilities of radio frequency electromagnetic waves have been less deeply explored and applied. As communication frequencies increase, radar sensing and communication are converging in spectrum resources, device form factors, and transmission technologies, laying the foundation for integrated communication and sensing design.

[0003] Millimeter waves, with their short wavelengths, offer extremely broad spectrum resources and high bandwidth capabilities, meeting the demands of future applications such as enhanced mobile broadband and ultra-reliable, low-latency communications in 6G and subsequent networks. In integrated telepresence scenarios, the wide bandwidth of millimeter wave bands supports high-speed transmission of large amounts of data, helping to increase the capacity and anti-interference capabilities of communication links. Their high directivity and narrow beam characteristics enable precise beamforming, significantly improving environmental perception accuracy and making them suitable for applications requiring precise detection and positioning.

[0004] However, millimeter waves also face challenges such as weak penetration and limited transmission distance, requiring more complex physical layer technologies to compensate for transmission losses. Combining massive MIMO with beamforming technology can provide highly directional, high-gain beams, helping to overcome the many non-idealities in millimeter wave signal transmission. Specifically, by deploying large-scale antenna arrays at base stations, massive MIMO can suppress inter-user interference on the same time-frequency resources, significantly increasing network capacity without increasing bandwidth or base station density. Beamforming, as a native capability of massive MIMO, is fundamental for achieving joint performance optimization of communication and perception. The design of the beamforming matrix can be modeled as an optimization problem. The need to simultaneously satisfy multiple constraints, such as total system power, maximum communication rate, and physical layer security constraints, introduces non-convexity into the beamforming optimization problem. The high-frequency characteristics and multipath effects of millimeter wave signals further increase the complexity of the objective function, posing new challenges to solving the optimization problem. Therefore, a redesigned integrated beamforming method is needed to enhance the fusion of communication and perception information and achieve more precise beamforming. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a synaesthesia integrated beamforming method and medium based on the cross-attention mechanism, so as to improve the degree of fusion of communication and perception information and achieve more accurate beamforming.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a synaesthesia integrated beamforming method based on a cross-attention mechanism, which is applied to a millimeter-wave massive MIMO synaesthesia integrated system, comprising the following steps:

[0008] Obtain the channel state information matrix of the downlink communication user and the target response matrix of the potential sensing target, input them into the synaesthesia integrated beamforming model based on the Transformer architecture, and obtain the beamforming matrix;

[0009] The synaesthesia integrated beamforming model based on the Transformer architecture includes an encoding module, a synaesthesia feature fusion module and a decoding module connected in sequence;

[0010] The encoding module includes two stacked Transformer encoders based on a multi-head attention mechanism, which are used to perform preliminary feature extraction on the channel state information matrix and the target response matrix to obtain communication features and perception features;

[0011] The synaesthesia feature fusion module includes two parallel feature enhancement structures based on a cross-attention mechanism, which are used to interactively enhance the communication feature and the perception feature;

[0012] The decoding module includes a superimposed Transformer decoder based on a masked multi-head attention mechanism and a Transformer decoder based on a multi-head cross attention mechanism, which are used to further fuse the communication features and perception features after interaction enhancement, and obtain a reconstructed beamforming matrix based on the fusion results and a randomly initialized beamforming matrix as the final output.

[0013] Furthermore, the channel state information matrix and the target response matrix are both vectorized and position-coded, and then input into the synaesthesia integrated beamforming model based on the Transformer architecture.

[0014] Furthermore, when vectorizing the channel state information matrix and the target response matrix, the real part and the imaginary part of each matrix are separated respectively, and the real number matrix is ​​reconstructed.

[0015] Furthermore, the channel state information matrix and the target response matrix are position-coded using the following formula:

[0016]

[0017]

[0018] Among them, PE represents position encoding, pos represents the position index of input data, i represents dimension index, d model is an adjustable hyperparameter representing the model dimension.

[0019] Furthermore, in the synaesthesia feature fusion module, one feature enhancement structure uses the communication feature as the value matrix and the perception feature as the query matrix and the key matrix; another feature enhancement structure uses the perception feature as the value matrix and the communication feature as the query matrix and the key matrix.

[0020] Furthermore, in the decoding module, the interactively enhanced communication features and perception features are concatenated through the Concat function as the query matrix and key matrix of the Transformer decoder based on the multi-head cross-attention mechanism; the randomly initialized beamforming matrix calculated by the Transformer decoder based on the masked multi-head attention mechanism is used as the value matrix of the Transformer decoder based on the multi-head cross-attention mechanism; the Transformer decoder based on the multi-head cross-attention mechanism performs feature fusion based on its value matrix, query matrix and key matrix to obtain a reconstructed beamforming matrix.

[0021] Furthermore, the Transformer architecture-based synaesthesia integrated beamforming model is trained using an offline training method. During the training process, a loss function is constructed based on a weighted integration index and a penalty term. The weighted integration index is the weighted sum of a communication evaluation index and a perception evaluation index. The communication evaluation index is the sum of the reachable rates of communication users, and the perception evaluation index is the mutual information between the echo signal received by the base station and the target response matrix.

[0022] Furthermore, the penalty term is used to constrain the total power of the system, and the penalty term introduces a penalty parameter determined based on penalty convergence theory.

[0023] Furthermore, the specific expression of the loss function is as follows:

[0024]

[0025] Among them, w is the beamforming matrix to be optimized, K is the number of communication targets, R k is the sum of the achievable rates of the communication users, I(y s ;g S |X=WS) is the mutual information between the target response matrices, α k and α s R k and I(y s ;g S|X=WS) weight, λ is the penalty parameter, and P0 is the upper limit of the total system power.

[0026] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program implements the above method when executed by a processor.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. The present invention proposes a synaesthesia integrated beamforming method based on a cross-attention mechanism for use in a millimeter-wave large-scale MIMO synaesthesia integrated system, and designs a synaesthesia integrated beamforming model based on a Transformer architecture. The model takes the channel state information matrix of the downlink communication user and the target response matrix of the potential perception target as input, and can output a reconstructed beamforming matrix. Specifically, the model includes an encoding module, a synaesthesia feature fusion module, and a decoding module connected in sequence, wherein the encoding module includes two stacked Transformer encoders based on a multi-head attention mechanism, which are used to perform preliminary feature extraction on the channel state information matrix and the target response matrix to obtain communication features and perception features; the synaesthesia feature fusion module includes two parallel feature enhancement structures based on a cross-attention mechanism, which are used to interactively enhance communication features and perception features. This module can promote deep interaction and joint processing of communication and perception, and improve the degree of fusion of communication and perception information; the decoding module includes a stacked Transformer decoder based on a masked multi-head attention mechanism and a decoder based on a multi-head cross-attention mechanism. The Transformer decoder is designed to further fuse the communication features and perception features after interaction enhancement, and obtain the reconstructed beamforming matrix as the final output based on the fusion results and the randomly initialized beamforming matrix. The Transformer architecture realizes global information processing through the self-attention mechanism, and has superior multimodal fusion and feature representation capabilities, powerful parallel computing capabilities and excellent generalization performance. In the synaesthesia integration scenario, there is a complex mapping relationship between input features such as the channel state information matrix and the target response matrix and the beamforming matrix. The self-attention mechanism can accurately capture the long-range dependency between input features while maintaining computational efficiency, thereby achieving more accurate and efficient beamforming matrix design.

[0029] 2. The present invention vectorizes and position-encodes the channel state information matrix and the target response matrix before inputting them into the synaesthesia integrated beamforming model based on the Transformer architecture, which can retain the spatial position information and enhance the model's perception of spatial features.

[0030] 3. The present invention uses the sum of the achievable rates of communication users as the communication evaluation index, and uses the mutual information between the echo signal received by the base station and the target response matrix as the perception evaluation index. Using mutual information as a perception evaluation index can effectively suppress interference and improve the robustness and stability of the system. At the same time, the mutual information has the same dimension as the communication achievable rate, and is more suitable for the design and research of weighted integrated indicators. The present invention constructs a weighted integrated indicator based on the weighted sum of the communication evaluation index and the perception evaluation index, which can reflect the total performance of communication and perception and flexibly adapt to more scenario requirements; based on the weighted integrated index and the penalty term, a loss function is constructed, which can realize the base station end transmit beamforming design under the condition of meeting the system total power constraint. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a structural diagram of the synaesthesia integrated beamforming model based on the Transformer architecture.

[0032] Wherein, CSI represents channel state information, TRG represents target response matrix;

[0033] Figure 2 This is a structural diagram of the synaesthesia feature fusion module.

[0034] Among them, d model represents the model dimension, N represents the vector dimension after position encoding, Q, K, and V represent the query matrix, key matrix, and value matrix respectively;

[0035] Figure 3 Schematic diagram of the embodiment method. DETAILED DESCRIPTION

[0036] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0037] Example 1

[0038] In the synaesthesia integration scenario, there is a complex mapping relationship between input features such as Channel State Information (CSI) and Target Response Matrix (TRM) and the beamforming matrix. This embodiment provides a synaesthesia integration beamforming method that can be applied to the millimeter wave large-scale MIMO synaesthesia integration system. The deep fusion and efficient processing of communication and perception information are achieved through the Transformer deep learning framework based on the cross-attention mechanism. At the same time, mutual information is introduced into the perception evaluation index to further improve the robustness and accuracy of the system. The customized loss function and adjustable weights are used to flexibly adapt to the needs of different synaesthesia integration scenarios. Figure 3 As shown, it mainly includes the following steps:

[0039] S1. Establish a millimeter-wave massive MIMO interaceptive integrated system. Set the position of communication users and sensing targets with the base station as the origin, calculate the steering vector of the antenna array, and use the sparse scattering characteristic structure of the millimeter-wave channel to construct the downlink CSI matrix and TRG.

[0040] S2. Construct data sets and weighted integrated indicators.

[0041] S3. Preprocess the CSI matrix and TRG.

[0042] S4. Construct an encoding module, which includes two serially connected Transformer encoders and a fully connected layer. Each encoder includes a multi-head attention layer and a feedforward layer.

[0043] S5. Construct a synaesthesia feature fusion module, including two parallel superposition encoders, a feature enhancement structure based on cross attention, and a multi-head attention fusion structure.

[0044] S6. Construct a decoding module, including two serially connected Transformer decoders and a fully connected layer. Each decoder includes a masked multi-head attention layer, a multi-head attention layer, and a feedforward layer.

[0045] S7. Build a synaesthesia-integrated beamforming model based on the Transformer architecture and train it using a custom loss function. Maximize the integrated system evaluation index as the optimization goal, and adjust the model's hyperparameters to adapt to different synaesthesia-integrated scenarios.

[0046] S8. Combine the real and imaginary parts of the vector obtained in step S7 and normalize them to construct a final beamforming matrix and calculate the corresponding system indicators.

[0047] S9. Use the test set to perform the final performance evaluation and analyze the performance of the model in different scenarios, such as the performance changes under different conditions such as the number of users, number of targets, and signal-to-noise ratio, to evaluate the generalization ability and robustness of the model.

[0048] The following is the specific implementation process:

[0049] S1. First, establish a millimeter wave massive MIMO intersensory integrated system. The system is equipped with N T Transmitting antennas and N R Root receiving antennas, using the uniform linear array (ULA) arrangement. While detecting M sensing targets, K equipped with N UE To ensure the feasibility of beamforming design, K≤N should be satisfied. T <N R In order to ensure the performance of the radar, an additional structural extension space degree of freedom is introduced into the transmission signal X. The extended signal is dedicated to the radar detection signal and does not carry communication data. Transmitted signal is a linear combination of the communication signal and the radar detection signal, that is:

[0050]

[0051] In formula (1), and denote the communication beamforming matrix and the auxiliary beamforming matrix of the sounding stream, respectively, is the unit power data stream transmitted to K users in L time slots, Represents the radar detection signal. Both the communication signal and the radar detection signal are wide-sense stationary random processes with zero mean and unit power. The communication data signals of different users are uncorrelated, so The radar detection signal is a pseudo-random sequence with zero mean and unit variance, which is uncorrelated with each other.

[0052] Then, set up the communication signal model. Transmit signal X to K downlink communication users. The signal matrix received by the user receiving end is:

[0053] Y C =HX+Z C =HW C S C +HW s S S +Z C (2)

[0054] For the kth user, the received signal is:

[0055]

[0056] In formula (2), Represents the CSI matrix between the base station and the communication user. Assume that the base station channel estimation is ideal and an accurate CSI matrix can be obtained. The matrix In formula (3), the first term represents the ideal signal, the second term represents the self-interference of the communication user, the third term represents the interference of perception on communication, and the fourth term represents the additive white Gaussian noise (AWGN).

[0057] Considering the sparse scattering characteristics of the millimeter wave channel, each scatterer contributes a propagation path between the base station and the target, that is, there is a line-of-sight channel (LoS) and K+M-1 non-line-of-sight channels (nLoS). The CSI matrix H is the superposition of K+M multipath channels, that is:

[0058]

[0059] In formula (4), is satisfied The normalization factor, α l is the complex gain of the lth path, which obeys the complex Gaussian distribution a t (θ t,l ) and a r (θ r,l ) represent the antenna array responses at the transmitting and receiving ends, θ t,l and θ r,l are the arrival angle and departure angle of the lth path respectively. Taking the midpoint of the transmitting and receiving antennas as the reference positions, the array steering vectors of the transmitting antenna and the receiving antenna are obtained as follows:

[0060]

[0061]

[0062] In equations (5) and (6), λ is the signal wavelength, and d represents the distance between adjacent antennas, which is half the wavelength.

[0063] Next, a multi-user perception model is established. In this perception model, the base station is considered a co-located massive MIMO radar, where each signal has the same departure and arrival angles. The interference echo signals generated by the communication users affect the accuracy of radar target estimation and detection. The base station simultaneously receives reflection signals from the communication user's target scatterers and other perception target scatterers. The received echo signal is expressed as:

[0064]

[0065] In formula (7), and Represent the TRM matrices of communication and perception respectively. The first term of the signal expression represents the ideal echo signal, the second term represents the echo of the communication beam at the perception target, the third term represents the echo of the perception beam at the communication target, the fourth term represents the echo of the communication beam at the communication target, and the last term represents the AWGN matrix

[0066] The extended scatterer expression of K communication targets and M sensing targets is:

[0067]

[0068]

[0069] In formula (8) and (9), G C stands for communication extended scatterer, G S represents the perceived extended scatterer, α i and θ i are the reflection coefficient and arrival / departure angle of the i-th point scatterer, and represents the steering vectors of the transmit and receive arrays.

[0070] S2. Repeat step S1, changing the relative position and system parameters, to obtain multiple sets of corresponding steering vectors, CSI matrices, and TRGs, forming the training dataset. Separate the real and imaginary parts of the original CSI matrix and TRG, stack them, and partition them to form the training and validation sets for subsequent training and validation steps.

[0071] Then, an optimization problem is designed. The angle of maximizing the communication transmission rate and the weighted sum of the perceptual mutual information is optimized while considering the system power constraint. The optimization problem thus constructed is:

[0072]

[0073] In order to intuitively display the impact of weight coefficient changes on the system, set

[0074] The unconstrained optimization problem after introducing the penalty term is:

[0075]

[0076] In formula (11), λ is the penalty parameter selected based on the penalty convergence theory. Gradually adjusting the penalty parameter can gradually approach the optimal solution while maintaining computational stability.

[0077] The synaesthesia weight coefficient is designed according to the system's requirements for communication and perception functions, and a weighted integrated index is constructed. Specifically, the sum of the communication user's achievable rates is used as the communication evaluation index, and the mutual information between the echo signal received by the base station and the target response matrix is ​​used as the perception evaluation index. The SINR of the kth communication user k And the achievable rate expression R k They are:

[0078]

[0079] R k =log2(1+SINR k ) (13)

[0080] The performance indicator of radar perception is the mutual information between the base station echo signal and TRM. Before calculating the mutual information, the echo signal and the perceived target TRM need to be converted into a vector y S and g S The base station knows the transmitted signal, the mutual information expression between the echo signal and TRM is:

[0081]

[0082] In formula (14), represents differential entropy, and p(·) represents the probability density function.

[0083] S3: Separate the real and imaginary parts of the CSI matrix and TRG, and re-stack them to construct a real matrix. Vectorize the real matrix and adjust the vector dimensions to be consistent. Then, use the encoding module to perform preliminary feature extraction on the CSI matrix and TRM. The specific process includes:

[0084] Add position coding. The formula for calculating position coding is:

[0085]

[0086]

[0087] In formulas (15) and (16), PE represents positional encoding, pos represents the position index of the input data, i represents the dimension index, and d model is an adjustable hyperparameter representing the model dimension.

[0088] S4. Construct an encoding module, which includes two serially connected Transformer encoders and a fully connected layer. Each encoder includes a multi-head attention layer and a feedforward layer.

[0089] The multi-head self-attention mechanism focuses on different features of the input sequence in different representation subspaces. There are a total of h attention heads, each with an independent query matrix Key-value matrix Sum weight matrix i=1,2,...,h. For each head i, the self-attention weight is calculated as:

[0090]

[0091] In formula (17), X is the representation matrix of the input sequence, Represents the query, key, and value weight matrices of the i-th head.

[0092] The output of the multi-head attention is combined with the output of all attention heads through a linear transformation, namely:

[0093] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O (18)

[0094] In formula (18), Concat(·) represents concatenation, W O Is a linear transformation matrix used to map the concatenated output back to the original dimension.

[0095] Then, the nonlinear transformation capability is increased through the feedforward neural network, and the expression is:

[0096] FFN(x)=max(0,x*W1+b1)W2+b2 (19)

[0097] In formula (19), W1 and W2 are linear transformation weight matrices, and b1 and b2 are bias vectors.

[0098] Again, add residual connections and layer normalization.

[0099] Repeat the above steps to stack two Transformer encoders based on the multi-head attention mechanism to obtain the CSI matrix and the features after preliminary processing by TRM, and obtain the communication features and perception features.

[0100] S5, build Figure 2The synaesthesia feature fusion module shown in the figure includes two parallel superposition encoders, a feature enhancement structure based on cross attention, and a multi-head attention fusion structure to interactively enhance communication features and perceptual features. The specific process includes:

[0101] S501. Mutual enhancement of features through cross attention. After the initial processing of CSI and TRM, the features are enhanced in parallel. One path uses the CSI features as the query matrix and key matrix, and the TRM features as the value matrix; the other path uses the TRM features as the query matrix and key matrix, and the CSI features as the value matrix. The expressions are:

[0102]

[0103]

[0104] S502: Stack the mutually enhanced CSI and TRM features. The expression is:

[0105] head ISAC =Concat(head com ,head sen ) (twenty two)

[0106] S6. Construct a decoding module, including two serially connected Transformer decoders and a fully connected layer. Each decoder includes a masked multi-head attention layer, a multi-head cross attention layer, and a feedforward layer.

[0107] The decoder used in the decoding module adds a masked multi-head attention layer and a cross multi-head attention layer to the encoder. The input is a randomly initialized beamforming matrix. The specific process includes:

[0108] S601, the masked multi-head attention layer is used to ensure that the current position can only access the previous position information. The expression of the masked attention layer output is:

[0109]

[0110] In formula (23), M is the mask matrix. If time step i cannot see time step j, then M[i, j] = -∞, otherwise it is 0.

[0111] S602, head ISAC As the query matrix and key matrix of the multi-head cross attention layer, the beamforming matrix after the masked multi-head attention layer is used as the value matrix of the multi-head cross attention layer. The expression for feature fusion of the multi-head cross attention layer is:

[0112] CrossMultiHead(Q ISAC ,K ISAC,V BF )=Concat(head1,head2,...,head h )W O (twenty four)

[0113] S7, according to the three modules constructed in steps S5-S7, establish Figure 1 The Transformer-based synaesthesia integrated beamforming model shown is trained with the optimization objective constructed in step S2, and the hyperparameters of the model are adjusted to adapt to different synaesthesia integrated scenario requirements.

[0114] S8. Combine the real and imaginary parts of the vector obtained in step S7 and normalize them to construct a final beamforming matrix and calculate the corresponding system indicators.

[0115] S9. Final performance evaluation is conducted using the test set. The model's performance in different scenarios is analyzed, such as the number of users, number of targets, and signal-to-noise ratio. The model's generalization and robustness are evaluated. The model utilizes offline training and online computation. During offline training, the weights of the integrated system evaluation indicators are modulated based on the emphasis on communication and perception. The model is trained using historical system information as a dataset. After model training is complete, the integrated synaesthesia beamforming matrix is ​​computed online.

[0116] Compared with the existing technology, the above method has the following advantages:

[0117] (1) The mutual information between the echo and target response matrices and the communication user reachable rate are selected to jointly construct a weighted integrated index. Using mutual information as a perception evaluation index can effectively suppress interference and improve the robustness and stability of the system. At the same time, mutual information and the communication reachable rate have the same dimension, making them more suitable for the design and research of integrated indicators. The weighted integrated index can adjust the size according to different system requirements, reflecting the combined performance of communication and perception, and flexibly adapting to more scenario requirements.

[0118] (2) Use a Transformer-based deep learning architecture for beamforming design. The Transformer architecture implements global information processing through a self-attention mechanism and focuses on multiple input features through a multi-head attention mechanism. It has excellent multimodal fusion and feature representation capabilities, powerful parallel computing capabilities, and excellent generalization performance, which can effectively solve the computational delay problem caused by traditional beamforming design problems.

[0119] (3) A feature fusion module based on the cross-attention mechanism was designed. After extracting the features of the communication channel state information and the perception target response matrix through the encoder, two parallel multi-head cross-attentions were used to achieve feature enhancement. The fusion was then reconstructed in the decoder to obtain the beamforming matrix. This fusion method increases the interaction between communication and perception, fully utilizes system information for beamforming design, and improves the efficiency and accuracy of the system.

[0120] Example 2

[0121] This embodiment provides an electronic device comprising a memory and a processor, wherein the processor is configured to execute a program stored in the memory, wherein the program comprises a number of instructions and can execute all or part of the steps of the method described in Example 1. The memory comprises a computer-readable storage medium, which can specifically be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code.

[0122] The above description of the embodiments is intended to facilitate understanding and use of the invention by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to these embodiments and apply the general principles described herein to other embodiments without requiring inventive effort. Therefore, the present invention is not limited to the above-described embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention, without departing from the scope of the present invention, should be within the scope of protection of the present invention.

Claims

1. A cross-attention mechanism-based synaesthesia integrated beamforming method, applied to millimeter-wave massive MIMO synaesthesia integrated system, characterized by: The following steps are involved: Obtain the channel state information matrix of the downlink communication user and the target response matrix of the potential sensing target, input them into the synaesthesia integrated beamforming model based on the Transformer architecture, and obtain the beamforming matrix; The synaesthesia integrated beamforming model based on the Transformer architecture includes an encoding module, a synaesthesia feature fusion module and a decoding module connected in sequence; The encoding module includes two stacked Transformer encoders based on a multi-head attention mechanism, which are used to perform preliminary feature extraction on the channel state information matrix and the target response matrix to obtain communication features and perception features; The synaesthesia feature fusion module includes two parallel feature enhancement structures based on a cross-attention mechanism, which are used to interactively enhance the communication feature and the perception feature; The decoding module includes a superimposed Transformer decoder based on a masked multi-head attention mechanism and a Transformer decoder based on a multi-head cross attention mechanism, which are used to further fuse the communication features and perception features after interaction enhancement, and obtain a reconstructed beamforming matrix based on the fusion results and a randomly initialized beamforming matrix as the final output.

2. The synaesthesia integrated beamforming method based on the cross-attention mechanism according to claim 1 is characterized in that: After the channel state information matrix and the target response matrix are both vectorized and position-coded, they are input into the synaesthesia integrated beamforming model based on the Transformer architecture.

3. The synaesthesia integrated beamforming method based on the cross-attention mechanism according to claim 2 is characterized in that: When vectorizing the channel state information matrix and the target response matrix, the real part and the imaginary part of each matrix are separated respectively, and the real number matrix is ​​reconstructed.

4. The synaesthesia integrated beamforming method based on the cross-attention mechanism according to claim 2, characterized in that: The channel state information matrix and the target response matrix are position-coded using the following formula: Among them, PE represents position encoding, pos represents the position index of input data, i represents dimension index, d model is an adjustable hyperparameter representing the model dimension.

5. The synaesthesia integrated beamforming method based on the cross-attention mechanism according to claim 1, characterized in that: In the synaesthesia feature fusion module, one feature enhancement structure uses the communication feature as the value matrix and the perception feature as the query matrix and key matrix; the other feature enhancement structure uses the perception feature as the value matrix and the communication feature as the query matrix and key matrix.

6. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 1, characterized in that: In the decoding module, the communication features and perception features after interactive enhancement are concatenated through the Concat function as the query matrix and key matrix of the Transformer decoder based on the multi-head cross attention mechanism; the randomly initialized beamforming matrix calculated by the Transformer decoder based on the masked multi-head attention mechanism is used as the value matrix of the Transformer decoder based on the multi-head cross attention mechanism; the Transformer decoder based on the multi-head cross attention mechanism performs feature fusion based on its value matrix, query matrix and key matrix to obtain a reconstructed beamforming matrix.

7. The synaesthesia integrated beamforming method based on the cross-attention mechanism according to claim 1, characterized in that: The Transformer architecture-based synaesthesia integrated beamforming model is trained using an offline training method. During the training process, a loss function is constructed based on a weighted integration index and a penalty term. The weighted integration index is the weighted sum of a communication evaluation index and a perception evaluation index. The communication evaluation index is the sum of the achievable rates of communication users, and the perception evaluation index is the mutual information between the echo signal received by the base station and the target response matrix.

8. The synaesthesia integrated beamforming method based on the cross-attention mechanism according to claim 7, characterized in that: The penalty term is used to constrain the total power of the system, and the penalty term introduces a penalty parameter determined based on penalty convergence theory.

9. The synaesthesia integrated beamforming method based on the cross attention mechanism according to claim 8, characterized in that: The specific expression of the loss function is as follows: Among them, w is the beamforming matrix to be optimized, K is the number of communication targets, R k is the sum of the achievable rates of the communication users, I(y S ;g S |X=WS) is the mutual information between the target response matrices, α k and α s R k and I(y S ;g S |X=WS) weight, λ is the penalty parameter, and P0 is the upper limit of the total system power.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images