Multi-agent SAC beam forming optimization method and system for secure communication

Abstract

The invention belongs to the technical field of communication security, and particularly relates to a multi-agent SAC beam forming optimization method and system for security communication. Comprising the steps of constructing a communication system model; considering that the malicious node has two intelligent behavior modes of sleep and eavesdropping, constructing an attack model; deriving a key generation rate and a data transmission rate, and generating a joint optimization problem of the key generation rate and the data transmission rate; a friendly node is set, channel information provided by the friendly node is utilized, and a malicious node eavesdropping channel and behavior are predicted through LSTM; a multi-agent SAC algorithm is introduced, eavesdropping channels and behaviors of malicious nodes predicted by LSTM are integrated, beam forming vectors in the joint optimization problem are cooperatively optimized, and an optimal beam forming strategy is obtained. According to the method, a unified beam forming optimization framework is constructed, dynamic evaluation and suppression of eavesdropping risks are realized under the condition of partial observable channels, and the joint security of two legal communication parties in key generation and data transmission is improved.

Description

Technical Field

[0001] This invention belongs to the field of communication security technology, and in particular relates to a multi-agent SAC beamforming optimization method and system for secure communication. Background Technology

[0002] With the evolution of sixth-generation wireless communication technology, we are entering a new era of the Internet of Things (IoT) that inherently demands security, elevating the importance of data security to an unprecedented level. Against this backdrop, physical layer key generation, as a potential alternative to key distribution in dynamic environments, shows great promise. This method fully utilizes the inherent randomness and reciprocity of wireless channels, enabling communicating parties to generate encryption keys based on shared channel state information. Meanwhile, multi-antenna systems are widely used in modern wireless communication, with beamforming technology used for directional signal transmission and reception, helping to improve the accuracy of channel estimation and thus providing crucial support for generating highly correlated keys between authorized communicating parties.

[0003] However, existing wireless communication systems typically treat physical layer key generation and secure data transmission as two separate problems, lacking a unified beamforming optimization framework. Traditional methods are significantly inadequate in the face of intelligent eavesdropping attacks. They are mostly based on the assumption that the eavesdropper's behavior is fixed and unchanging, making it difficult to effectively deal with the eavesdropper's dynamic switching between "sleep" and "eavesdropping" modes based on channel quality. This results in a significant decrease in system security performance when the eavesdropping channel conditions are favorable.

[0004] Another key challenge is that legitimate users typically cannot directly obtain the channel state information of eavesdroppers. This partial observability makes it difficult for beamforming designs to accurately assess potential eavesdropping risks, thus simultaneously limiting both the security of key generation and the reliability of data transmission.

[0005] Furthermore, traditional convex optimization methods heavily rely on precise mathematical models and static channel assumptions. In high-dimensional dynamic environments with multiple antennas and time-varying channels, they struggle to achieve real-time adaptive adjustments and cannot meet the requirements for joint optimization of key generation and data transmission. Existing solutions often focus on a single performance objective, failing to effectively balance security and communication efficiency dynamically. In addition, the lack of intelligent coordination mechanisms between Alice and Bob in bidirectional communication makes it difficult to achieve globally optimal beamforming strategies in complex wireless environments. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a multi-agent SAC beamforming optimization method and system for secure communication. By constructing a unified beamforming optimization framework, it achieves dynamic assessment and suppression of eavesdropping risks under partially observable channel conditions, and enhances the joint security of legitimate communication parties in key generation and data transmission through a multi-agent collaborative mechanism.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a multi-agent SAC beamforming optimization method for secure communication.

[0008] A multi-agent SAC beamforming optimization method for secure communication includes the following steps: Construct a communication system model for legitimate nodes to conduct bidirectional communication using multiple antennas and for malicious nodes to conduct eavesdropping using a single antenna; Considering that malicious nodes possess two intelligent behavior modes—sleep and eavesdropping—an attack model is constructed. Derive the key generation rate and data transmission rate, and generate a joint optimization problem for the key generation rate and data transmission rate; Set up friendly nodes and use the channel information provided by the friendly nodes to predict the channel eavesdropping behavior of malicious nodes through LSTM; The multi-agent SAC algorithm is introduced, which integrates the eavesdropping channels and behaviors of malicious nodes predicted by LSTM, and collaboratively optimizes the beamforming vector in the joint optimization problem to obtain the optimal beamforming strategy.

[0009] A second aspect of the present invention provides a multi-agent SAC beamforming optimization system for secure communication.

[0010] A multi-agent SAC beamforming optimization system for secure communication includes: The communication system construction module is configured to: construct a communication system model for legitimate nodes to conduct bidirectional communication with multiple antennas and malicious nodes to conduct eavesdropping with a single antenna; The attack model construction module is configured to: construct an attack model considering that malicious nodes have two intelligent behavior modes: sleep and eavesdropping. The joint optimization module is configured to derive the key generation rate and data transmission rate, and generate a joint optimization problem for the key generation rate and data transmission rate. The LSTM prediction module is configured to: set up friendly nodes, utilize the channel information provided by the friendly nodes, and use LSTM to predict the channel eavesdropping behavior of malicious nodes; The optimal strategy formation module is configured to: introduce the multi-agent SAC algorithm, integrate the eavesdropping channels and behaviors of malicious nodes predicted by LSTM, collaboratively optimize the beamforming vector in the joint optimization problem, and obtain the optimal beamforming strategy. A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the multi-agent SAC beamforming optimization method for secure communication as described in the first aspect of the present invention.

[0011] The fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the multi-agent SAC beamforming optimization method for secure communication as described in the first aspect of the present invention.

[0012] The above one or more technical solutions have the following beneficial effects: This invention provides a multi-agent SAC beamforming optimization method and system for secure communication. By constructing a unified beamforming optimization framework, it achieves dynamic assessment and suppression of eavesdropping risks under partially observable channel conditions, and enhances the joint security of legitimate communication parties in key generation and data transmission through a multi-agent collaborative mechanism. This invention can effectively counter intelligent eavesdropping attacks and achieve a dynamic balance between security and communication efficiency in time-varying multi-antenna channels.

[0013] This invention analyzes the behavior of eavesdroppers and constructs an attack model that enables malicious nodes to possess two intelligent behavioral modes: sleep and eavesdropping. When the quality of the eavesdropping channel is inferred to be poor, the malicious node can remain silent to conserve power. In contrast, when the quality of the eavesdropping channel is inferred to be good, the malicious node chooses to monitor the key generation process on a wireless fading channel. This comprehensively improves the system's security performance.

[0014] This invention addresses the partial observability problem by integrating an LSTM-based prediction module into the multi-agent SAC framework. Channel information provided by friendly nodes is used as input to train the LSTM network, which jointly predicts the channel and behavioral states of malicious nodes. These predictions are then used to form the comprehensive partial observable states of the SAC agents. Combining friendly nodes and the LSTM module enables informed decision-making without directly observing the eavesdropper's channel, enhancing robustness and performance in partially observable environments.

[0015] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0017] Figure 1 This is a schematic diagram of the communication system model in Example 1.

[0018] Figure 2 This is a flowchart illustrating the beamforming optimization implemented in Example 1.

[0019] Figure 3 This is a schematic diagram comparing the performance of the SAC algorithm in Example 1 with other Actor-Critic-based deep reinforcement learning algorithms (including TD3 and PPO).

[0020] Figure 4 This is a schematic diagram comparing LSTM prediction with fully observed and partially observable scenarios in Example 1. Detailed Implementation

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0023] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0024] Terminology Explanation: (1) SAC: Soft Actor-Critic; (2) TDD: Time Division Duplex; (3) LSTM: Long Short-Term Memory; (4) CSI: Channel State Information; (5) PPO: Proximal Policy Optimization; (6) TD3: Twin Delayed Deep Deterministic Policy Gradient; (7) KL Divergence: Kullback-Leibler Divergence.

[0025] Example 1 This embodiment discloses a multi-agent SAC beamforming optimization method for secure communication.

[0026] like Figure 1 As shown, the multi-agent SAC beamforming optimization method for secure communication includes the following steps: Construct a communication system model for legitimate nodes to conduct bidirectional communication using multiple antennas and for malicious nodes to conduct eavesdropping using a single antenna; Considering that malicious nodes possess two intelligent behavior modes—sleep and eavesdropping—an attack model is constructed. Derive the key generation rate and data transmission rate, and generate a joint optimization problem for the key generation rate and data transmission rate; Set up friendly nodes and use the channel information provided by the friendly nodes to predict the channel eavesdropping behavior of malicious nodes through LSTM; The multi-agent SAC algorithm is introduced, which integrates the eavesdropping channels and behaviors of malicious nodes predicted by LSTM, and collaboratively optimizes the beamforming vector in the joint optimization problem to obtain the optimal beamforming strategy.

[0027] This invention aims to propose an innovative beamforming optimization method using a multi-agent SAC algorithm to address the challenge of co-optimizing key security and data transmission in dynamic environments. This method constructs a unified beamforming optimization framework to dynamically assess and suppress eavesdropping risks under partially observable channel conditions, and enhances the joint security of legitimate communication parties in key generation and data transmission through a multi-agent collaborative mechanism. This invention effectively counters intelligent eavesdropping attacks, achieving a dynamic balance between security and communication efficiency in time-varying multi-antenna channels.

[0028] The specific technical solutions adopted in the embodiments of the present invention are as follows: (1) Construct a communication system model. The legitimate node Alice and Bob communicate bidirectionally with multiple antennas, while the malicious node Eve eavesdrops with a single antenna. Beamforming is used to enhance security and performance.

[0029] (2) Construct an attack model. The malicious node Eve has two intelligent behavior modes: sleep and eavesdropping.

[0030] (3) Derive the key generation rate and data transmission rate, and generate a joint optimization problem.

[0031] (4) Introduce the multi-agent SAC algorithm to collaboratively optimize the beamforming vector and balance exploration and utilization.

[0032] (5) Based on LSTM, using the channel information provided by the friendly node Fred, Eve’s eavesdropping channel and behavior are predicted by LSTM.

[0033] The technical solution of this embodiment will now be explained in detail with reference to the accompanying drawings.

[0034] Specifically, step (1) includes: Assuming the system model is as follows Figure 1 As shown, we consider a TDD two-way communication system employing multiple antennas. In this system, both the legitimate devices Alice and Bob are equipped with... Multiple antennas are used, and beamforming technology is employed to improve communication and physical layer key generation performance; while the passive eavesdropping node Eve is a single-antenna device that attempts to eavesdrop on the communication between the two.

[0035] In this embodiment, for ease of description, the two legitimate nodes are named legitimate devices Alice and Bob. Alice and Bob are respectively designated as the first and second legitimate nodes.

[0036] This represents the direct channel from Alice to Bob. This represents the direct channels from Alice to Eve and from Bob to Eve, where , Represents the key generation time slot. This represents the data transmission time slot. The signal received by Bob can be represented as:

[0037] in, and These are the beamforming vectors of Alice and Bob, with amplitudes of... Phase shift is , , It is Alice's transmitted signal. It follows the principle that the mean is zero and the variance is... Additive noise with a complex Gaussian distribution, It is the equivalent channel from Alice to Bob. Represents the Hermitian transpose of the beamforming vector of the first legitimate node; This represents the Hermitian transpose of the beamforming vector of the second legitimate node. Indicates a legitimate channel under the key generation time slot; This represents the direct channel between the malicious node and the two legitimate nodes (i.e., the direct channel from Alice to Eve and from Bob to Eve).

[0038] Since it is assumed that all links are mutual, the received signal from Bob to Alice can be represented as:

[0039] This represents Bob's transmitted signal. It follows the principle that the mean is zero and the variance is... Additive noise with a complex Gaussian distribution, It is the equivalent channel from Bob to Alice. It is the direct channel from Bob to Alice.

[0040] Similarly, the signals Eve eavesdropped on Alice and Bob , It can be represented as:

[0041]

[0042] This represents Alice's transmitted signal. This represents Bob's transmitted signal. and This represents the Hermitian transpose of the direct channels from Alice to Eve and from Bob to Eve. This represents the equivalent channel from Alice to Eve. This represents the equivalent channel from Bob to Eve.

[0043] We assume that all channels follow Rayleigh fading, where all channel gains... , and It is an independently distributed, circularly symmetric complex Gaussian random variable with non-zero mean and unit variance.

[0044] Specifically, step (2) includes: We believe the malicious node Eve is an intelligent attacker with two behaviors: a sleep mode and an eavesdropping mode. We call this an intelligent eavesdropping attack and represent these behaviors as follows: Used for eavesdropping Used for sleep. When the quality of the eavesdropping channel is inferred to be poor, a malicious Eve can remain silent to conserve power. In contrast, when the quality of the eavesdropping channel is inferred to be good, Eve chooses to monitor the key generation process on a wireless fading channel. We set the gain threshold of the eavesdropping channel to... Eve's status It can be defined as:

[0045] in, express The status of malicious nodes at all times. express The norm of the direct channel between a malicious node and two legitimate nodes at any given time.

[0046] Specifically, step (3) includes: We use CSI as the intrinsic random source for key generation. The process involves channel sounding, quantization, information coordination, and privacy amplification, with the key generation interval set to exceed the channel coherence time. This is to prevent key correlation.

[0047] Due to the channel reciprocity of all links in a TDD system, the key generation rate from Alice to Bob and from Bob to Alice is equal. Therefore, we assume that Bob is the active initiator of key generation and data transmission, and thus take the key generation rate on Alice's side as an example.

[0048] Key generation rate The theoretical bound is determined by conditional mutual information. Confirmed. When At this time, node Eve is in a sleep state, and the key generation rate can be given by the following formula:

[0049] Where the covariance matrix , and Defined as , Due to the reciprocity of the channel, we can... Rewritten as:

[0050] The determinant of the covariance matrix is:

[0051]

[0052] Among them, the variance of the legitimate channel We define the equivalent cross-correlation matrix. , Indicates statistical average. Indicates a legitimate channel under the key generation time slot. This represents the Hermitian transpose of a legitimate channel under the key generation time slot. For the sake of symbol simplicity without sacrificing generality, we normalize the variance of the noise to... Through further substitution, we obtain:

[0053] when At this time, node Eve is in an eavesdropping state, and the key generation rate must be determined by evaluating the complete mutual information. It can be specifically expressed as:

[0054] Similarly, due to the reciprocity of the channel, we can... Rewritten as:

[0055] The determinant of the covariance matrix is:

[0056]

[0057]

[0058] Among them, the variance of the eavesdropping channel Autocorrelation between legitimate and illegitimate channels We also defined the equivalent cross-correlation matrix. , This represents the Hermitian transpose of the eavesdropping channel. To mitigate the impact of a powerful eavesdropper like Eve, a security margin is introduced. ,therefore It can be formalized as follows:

[0059] in, This represents the variance of the eavesdropping channel.

[0060] Data transmission rate The theoretical upper limit is determined by the channel capacity. Confirmed, among which It is the channel bandwidth. This is the signal-to-noise ratio. Therefore, Alice's data transmission rate is given by the following formula:

[0061] This indicates a legitimate channel within a data transmission time slot.

[0062] On the one hand, our goal is to maximize the data transmission rate. This is to maintain consistency between metrics and key generation rate. On the other hand, the transceiver is also designed to maximize the key generation rate. It also carries the potential threat of being eavesdropped on by the intelligent Eve.

[0063] To this end, we optimized the beamforming vector symbol. and amplitude , and phase shift , To maximize the performance of both data transmission and key generation.

[0064] The optimization problem can be written as:

[0065] in, and It is a weighting factor that balances the importance of key transmission and key generation. This represents the maximum permissible power. The first two constraints simulate the beamforming vectors of Alice and Bob. and The power does not exceed the maximum allowable power, and the third constraint means that the intelligent Eve has only two possible behaviors to choose from.

[0066] Specifically, step (4) includes: To address the joint optimization problem of key generation and data transmission under intelligent eavesdropping attacks, we propose a multi-agent SAC framework based on LSTM adversary prediction. Alice observes the current malicious behavior from the perspective of the legitimate user. and CSI and These are the previous beamforming vectors. and The function. Next, Alice and Bob choose the beamforming vector. and Provide feedback for environmental observation. Then, configure the current beamforming vector. and After that, CSI from and Become and Therefore, it can be observed that Alice's actions generate a finite-state Markov decision process.

[0067] Research shows that reinforcement learning is a powerful tool for solving time-series optimization problems that satisfy Markov decision processes. Traditional convex optimization methods do not work effectively in such dynamically changing situations; therefore, we employ the SAC algorithm as a model-free reinforcement learning strategy to optimize beamforming. Alice and Bob are treated as a multi-agent system that perceives the dynamic environment around them in each time slot and adjusts the beamforming vector based on environmental feedback.

[0068] We define a Markov decision process as a triple. ,in , and These are the state space, action space, and reward. (Time slot) Alice and Bob's actions can be written It corresponds to the beamforming vector. and The elements in the text. Therefore, Alice and Bob's actions can be written as:

[0069] in, express The actions of two valid nodes at any given moment; express The beamforming vectors at both ends of the two legal nodes at time 1; The wireless environmental interaction between Alice, Bob, and Eve is determined by state. Indicated. In time slots In the context of Alice's observation and CSI, malicious behavior... The current environment information related to the operation of the previous time slot. Therefore, the time slot Alice's state can be given by the following formula:

[0070] in, They represent The equivalent legitimate channel, equivalent eavesdropping channel, and eavesdropping state at any given time; Alice's instant rewards in any time slot are provided by This means that it is set to be proportional to a weighted sum of the data transfer rate and the key generation rate. In this case, Alice's reward function is given by the following equation:

[0071] Specifically, step (4) includes: SAC is a model-free, non-policy Actor-Critic algorithm that combines the advantages of maximum entropy reinforcement learning and stable policy optimization. The objective function in SAC is defined as:

[0072] in It is a temperature parameter that controls the trade-off between exploration and development. It is a state The entropy of the strategy. We model Alice and Bob as two cooperative agents who jointly optimize their beamforming vectors. and Each agent uses its own network of participants. To generate actions and share two common critic networks. and To assess joint actions, and These are two action value functions The parameters. Any The loss function of the function is:

[0073] in It is a discount factor. Data collected from the past of the strategy Sampling from the middle, yes The state at any given moment, yes Momentary actions yes Momentary rewards yes The state at any given moment, From policy Mid-sampling Momentary actions, two target Q-networks Corresponding to the two Q-networks to enhance training stability, and These are the parameters of two target Q-networks.

[0074] At the same time, strategy The loss function can be derived from the KL divergence and simplifies to:

[0075] in and From respectively and Sampling.

[0076] Furthermore, to make the entropy regularization coefficient in the SAC algorithm adaptive, we ensure that the average entropy exceeds... To export Loss function:

[0077] in and From respectively and Mid-sampling, when the policy entropy is lower than the target value At times, the importance of increasing policy entropy is diminished; conversely, the focus shifts to enhancing value.

[0078] Specifically, step (5) includes: Another key issue is that legitimate users cannot observe the eavesdropper's CSI. To address this partial observability problem, we introduced a friendly node named Fred, deployed near Eve. This node, Fred, can legitimately communicate with Alice and Bob, providing channel measurements highly correlated with Eve's channel due to their spatial proximity. By utilizing historical channel data from Fred, we can more accurately predict complex-valued eavesdropping channels. And Eve's binary behavior state .

[0079] Specifically, we deploy a single antenna node, Fred, in the system. The channel between Fred and legitimate users Alice and Bob is represented as... and Therefore, the channel information provided by Fred can be used as input to train an LSTM network for jointly predicting Eve's channel and behavioral states, and then integrated into the multi-agent SAC framework. The input to the LSTM... Composed of a sequence of historical channel state vectors from Fred:

[0080] The LSTM processes the sequence and outputs a joint prediction containing the complex-valued eavesdropping channel and the behavioral state, as shown in the following formula:

[0081] in It is the sequence length; Indicates in The input vector at time step; Indicates from The first vector in the input sequence starting at time t; Indicates LSTM prediction The equivalent eavesdropping channel at any given moment. Indicates LSTM prediction A state of constant eavesdropping.

[0082] These predictions are then used to form the integrated partial observation state of the SAC agent, which is represented as follows:

[0083] Combining friendly nodes and LSTM modules enables informed decision-making without directly observing the eavesdropper's channel, enhancing robustness and performance in partially observable environments.

[0084] Finally, to verify the effectiveness of this invention, we conducted numerical simulations to evaluate the performance of the proposed multi-agent SAC algorithm with LSTM-based attack prediction. The simulation environment was implemented using PyTorch in Python 3.12.0. Both legitimate transceivers, Alice and Bob, were equipped with... Antenna. Therefore, the action dimension of each agent is... , representing the real and imaginary parts of their respective beamforming vectors. and The two agents learn collaboratively through centralized training with a distributed execution framework. The state dimension is set to... Including key generation and data transmission The real and imaginary parts of the equivalent channel, and the predicted real and imaginary parts of the eavesdropper channel. The SAC agent is built on a fully connected neural network architecture of Actor and Critic networks. Specifically, the actor and each critic network consists of two hidden layers, each containing Each neuron is subjected to layer normalization after each linear layer to stabilize training. Meanwhile, the LSTM-based attack prediction module is designed with a hidden size... The system consists of a single LSTM layer followed by a fully connected output layer. The LSTM module is trained in parallel with the SAC agent, using the same replay buffer and mini-batch sampling strategy, enabling joint optimization of beamforming and adversary prediction in an end-to-end manner.

[0085] like Figure 3 As shown, we first compare the proposed SAC algorithm with other Actor-Critic-based deep reinforcement learning algorithms (including TD3 and PPO) based on the average reward obtained during training. Numerical simulations are performed under specified environmental parameters. As shown in the figure above, SAC achieves fast convergence and high stability, producing approximately [missing data - likely a number of parameters]. The average return is [value missing]. This performance improvement stems from its entropy-regularized exploration strategy, which effectively balances exploration and development in complex environments. The small fluctuations in the reward observed after convergence are primarily due to environmental randomness, entropy regularization, and the inherent variability of the reward function. In contrast, both PPO and TD3 exhibit lower performance relative to SAC, highlighting the latter's advantages in tasks such as beamforming optimization and key generation.

[0086] Figure 4 In this paper, we compare fully observed and partially observable scenarios, demonstrating the effectiveness of LSTM-based prediction. In the partially observable scenario, the reward loss is controlled within... Within this range, it significantly outperforms the original SAC algorithm. Conversely, due to the lack of state information about Eve and the eavesdropping channel, the original SAC algorithm exhibits significant errors, with an average performance degradation of approximately [missing information]. .

[0087] Example 2 This embodiment discloses a multi-agent SAC beamforming optimization system for secure communication.

[0088] A multi-agent SAC beamforming optimization system for secure communication includes: The communication system construction module is configured to: construct a communication system model for legitimate nodes to conduct bidirectional communication with multiple antennas and malicious nodes to conduct eavesdropping with a single antenna; The attack model construction module is configured to: construct an attack model considering that malicious nodes have two intelligent behavior modes: sleep and eavesdropping. The joint optimization module is configured to derive the key generation rate and data transmission rate, and generate a joint optimization problem for the key generation rate and data transmission rate. The LSTM prediction module is configured to: set up friendly nodes, utilize the channel information provided by the friendly nodes, and use LSTM to predict the channel eavesdropping behavior of malicious nodes; The optimal strategy formation module is configured to: introduce the multi-agent SAC algorithm, integrate the eavesdropping channels and behaviors of malicious nodes predicted by LSTM, collaboratively optimize the beamforming vector in the joint optimization problem, and obtain the optimal beamforming strategy. Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0089] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps in the multi-agent SAC beamforming optimization method for secure communication as described in Embodiment 1 of this disclosure.

[0090] Example 4 The purpose of this embodiment is to provide an electronic device.

[0091] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the multi-agent SAC beamforming optimization method for secure communication as described in Embodiment 1 of this disclosure.

[0092] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0093] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0094] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A multi-agent SAC beamforming optimization method for secure communication, characterized in that, Includes the following steps: Construct a communication system model for legitimate nodes to conduct bidirectional communication using multiple antennas and for malicious nodes to conduct eavesdropping using a single antenna; Considering that malicious nodes possess two intelligent behavior modes—sleep and eavesdropping—an attack model is constructed. Derive the key generation rate and data transmission rate, and generate a joint optimization problem for the key generation rate and data transmission rate; Set up friendly nodes and use the channel information provided by the friendly nodes to predict the channel eavesdropping behavior of malicious nodes through LSTM; The multi-agent SAC algorithm is introduced, which integrates the eavesdropping channels and behaviors of malicious nodes predicted by LSTM, and collaboratively optimizes the beamforming vector in the joint optimization problem to obtain the optimal beamforming strategy.

2. The multi-agent SAC beamforming optimization method for secure communication as described in claim 1, characterized in that, The specific construction process of the communication system model is as follows: Two legitimate nodes are set up, each equipped with multiple antennas, and beamforming technology is used to improve communication and physical layer key generation performance; The malicious node is a single-antenna device attempting to eavesdrop on the communication between two legitimate devices. Assume that all channels follow Rayleigh fading, where all channel gains are identically distributed, circularly symmetric complex Gaussian random variables with non-zero mean and unit variance.

3. The multi-agent SAC beamforming optimization method for secure communication as described in claim 2, characterized in that, Considering that malicious nodes possess two intelligent behavior modes—sleep and eavesdropping—an attack model is constructed, specifically including: Using parameters to define the intelligent behavior patterns of malicious nodes express: This indicates malicious node eavesdropping behavior. This indicates malicious node sleep behavior; Set the gain threshold of the eavesdropping channel to The status of malicious nodes Defined as: ； in, express The status of malicious nodes at all times. express The norm of the direct channel between a malicious node and two legitimate nodes at any given time.

4. The multi-agent SAC beamforming optimization method for secure communication as described in claim 3, characterized in that, The key generation rate and data transmission rate are specifically as follows: when At that time, the malicious node is in a sleep state, and the key generation rate of the legitimate device is... It is given by the following formula: ； in, This represents the beamforming vector of the second legitimate node. Represents the Hermitian transpose of the beamforming vector of the second legitimate node; Represents the equivalent cross-correlation matrix. , Indicates statistical average. This represents the beamforming vector of the first legitimate node. Represents the Hermitian transpose of the beamforming vector of the first legitimate node; Indicates a legitimate channel under the key generation time slot. Represents the Hermitian transpose of a legitimate channel under the key generation time slot; when At that time, the malicious node Eve was in an eavesdropping state, and the key generation rate of the legitimate device Alice was... It is given by the following formula: ； in, Indicates the variance of the eavesdropping channel; Represents the equivalent cross-correlation matrix. , The Hermitian transpose of the eavesdropping channel; Indicates safety margin; Data transfer rate of the legitimate device Alice It is given by the following formula: ； in, Indicates channel bandwidth. This indicates a legitimate channel within a data transmission time slot.

5. The multi-agent SAC beamforming optimization method for secure communication as described in claim 4, characterized in that, The joint optimization problem is expressed as: ； in, and It is a weighting factor that balances the importance of key transmission and key generation; P represents the maximum allowed power.

6. The multi-agent SAC beamforming optimization method for secure communication as described in claim 1, characterized in that, By setting up friendly nodes and utilizing the channel information provided by these nodes, LSTM is used to predict the channel eavesdropping behavior of malicious nodes. Specifically, this includes: Deploy friendly nodes near malicious nodes; Friendly nodes can communicate with two legitimate nodes and provide channel measurement information related to the channel of malicious nodes; The LSTM network is trained using channel measurement information provided by friendly nodes as input. It consists of a sequence of historical channel state vectors from friendly nodes; LSTM processes the historical channel state vector sequence of friendly nodes and outputs a joint prediction containing complex-valued eavesdropping channel and behavioral state, as shown in the following formula: in It is the sequence length; Indicates in The input vector at time step; Indicates from The first vector in the input sequence starting at time t; Indicates LSTM prediction The equivalent eavesdropping channel at any given moment. Indicates LSTM prediction A state of constant eavesdropping.

7. The multi-agent SAC beamforming optimization method for secure communication as described in claim 5, characterized in that, The optimal beamforming strategy, specifically the acquisition process, includes: The changes in the actions of two legitimate nodes generate a finite-state Markov decision process. The Markov decision process is defined as a triplet. ,in , and These are the state space, action space, and reward; The two legitimate nodes are modeled as two cooperative agents that jointly optimize their beamforming vectors. and ; Each agent uses its own agent network to generate actions and shares two common critic networks to evaluate joint actions; Actions of two valid nodes Written as: ； in, express The actions of the first and second valid nodes at any given moment; express The beamforming vectors at both ends of the first and second valid nodes at time points; Two legitimate nodes in the time slot status It is given by the following formula: ； in, They represent The equivalent legitimate channel, equivalent eavesdropping channel, and eavesdropping state at any given time; The reward function for two valid nodes is given by the following formula: ； Use LSTM predictions to form a comprehensive partial observation state of the SAC agent.

8. A multi-agent SAC beamforming optimization system for secure communication, characterized in that, include: The communication system construction module is configured to: construct a communication system model for legitimate nodes to conduct bidirectional communication with multiple antennas and malicious nodes to conduct eavesdropping with a single antenna; The attack model construction module is configured to: construct an attack model considering that malicious nodes have two intelligent behavior modes: sleep and eavesdropping. The joint optimization module is configured to derive the key generation rate and data transmission rate, and generate a joint optimization problem for the key generation rate and data transmission rate. The LSTM prediction module is configured to: set up friendly nodes, utilize the channel information provided by the friendly nodes, and use LSTM to predict the channel eavesdropping behavior of malicious nodes; The optimal strategy formation module is configured to: introduce the multi-agent SAC algorithm, integrate the eavesdropping channels and behaviors of malicious nodes predicted by LSTM, collaboratively optimize the beamforming vector in the joint optimization problem, and obtain the optimal beamforming strategy.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the multi-agent SAC beamforming optimization method for secure communication as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the multi-agent SAC beamforming optimization method for secure communication as described in any one of claims 1-7.

Images