A beam steering method and system based on a phased array antenna
By constructing a joint state vector and utilizing graph neural networks, digital twin environments, and reinforcement learning guidance strategies, the beam guidance problem of phased array antennas in complex dynamic environments was solved, achieving beam control with high accuracy, stability, and anti-interference capabilities.
Patent Information
- Application Number
- CN202610319388.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-16
- Publication Date
- 2026-06-23
AI Technical Summary
Existing phased array antenna beamguiding methods lack the ability to jointly model terminal motion state, environmental obstruction changes, reflection propagation relationship and interference coupling relationship in complex dynamic environments, resulting in beam mismatch, frequent switching and reduced link stability, and lack of security constraints and closed-loop adaptive update capabilities.
By constructing a joint state vector, parallel pre-simulation of multi-candidate beam control actions is performed using graph neural networks and digital twin environments. Combined with reinforcement learning guidance strategies and safety constraint predictive control, global perception, accurate decision-making, and adaptive optimization of complex scenarios are achieved.
It improves the accuracy of beam guidance, link reliability, anti-interference capability, and adaptive capability, ensuring the stability and security of the system in complex environments.
Smart Images
Figure CN122268430A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of wireless communication and smart antenna control technology, and in particular to a beam guidance method and system based on a phased array antenna. Specifically, it relates to a phased array antenna beam guidance method and system that integrates digital twin pre-simulation, graph neural network interference modeling, reinforcement learning guidance decision-making, and safety constraint predictive control. Background Technology
[0002] Phased array antennas can achieve rapid beam scanning, directional radiation, and adaptive tracking by flexibly controlling the amplitude and phase of each element. Therefore, they are widely used in mobile communications, satellite communications, vehicle-mounted communications, low-altitude communications, radar sensing, and integrated air-space-ground networks. In complex dynamic environments, beamguiding performance directly affects link establishment speed, communication reliability, anti-interference capability, and system resource utilization efficiency.
[0003] Most existing beamguiding methods rely on preset codebook scanning, maximum received power search, local greedy tracking, or beam selection mechanisms based on single channel observations. While these methods are relatively simple to implement, they typically only use instantaneous received signal strength, signal-to-noise ratio, or reference signal measurements for decision-making, lacking the ability to jointly model terminal motion states, environmental obstruction changes, reflection propagation relationships, interference coupling relationships, and array hardware states. When communication scenarios involve multiple interference sources, dynamic obstructions, fast-moving terminals, or complex multipath propagation, traditional methods are prone to problems such as beam mismatch, frequent switching, increased sidelobe leakage, and decreased link stability.
[0004] Furthermore, while some existing methods incorporate learning models or prediction mechanisms, they typically suffer from the following shortcomings: First, they fail to adequately characterize the mapping relationship between the real propagation environment and beam control behavior, making it difficult to pre-simulate and assess the risks of candidate beam actions before decision-making. Second, they lack a structured representation of interference coupling and propagation correlation among multiple targets and entities, making it difficult to accurately reflect the complex relationships between targets, interfering bodies, reflectors, and obstructors. Third, some methods only pursue maximizing link gain or signal quality, failing to incorporate safety constraints such as array element power limitations, sidelobe suppression requirements, beam switching smoothness, and future link reliability into a unified decision-making framework, resulting in insufficient engineering feasibility of the obtained beam control actions. Fourth, they lack an online closed-loop correction mechanism for model mismatch caused by environmental changes, equipment errors, and propagation parameter drift, leading to poor model generalization ability and long-term operational stability.
[0005] Therefore, there is an urgent need to propose a new beamguide method and system based on phased array antennas to achieve joint modeling of the propagation environment, interference relationships and array execution process in complex dynamic scenarios, and to improve the accuracy, stability, anti-interference and adaptive capabilities of beamguide while meeting safety constraints. Summary of the Invention
[0006] The purpose of this invention is to at least address the problems in existing phased array antenna beamguiding methods, such as insufficient perception of complex dynamic environments, weak modeling ability of multi-entity interference relationships, lack of pre-evaluation mechanism for candidate beam actions, insufficient consideration of beam control safety constraints, and lack of closed-loop adaptive update capability for environmental changes and model mismatch. The invention provides a beamguiding method and system based on phased array antennas to improve the accuracy, link reliability, anti-interference capability, control safety, and online adaptive capability of beamguiding in complex scenarios.
[0007] To achieve the above objectives, this invention provides a beam guidance method based on a phased array antenna, executed by a processor, comprising: acquiring array state information, terminal state information, pilot measurement information, environmental perception information, and historical guidance feedback information of the phased array antenna, and constructing a joint state vector; constructing a scene interaction graph including target nodes, interference nodes, reflection nodes, and occlusion nodes based on the joint state vector, and extracting interference risk characterization parameters using a graph neural network; and performing parallel pre-simulation of control actions for multiple candidate beams in a digital twin environment based on the joint state vector, the interference risk characterization parameters, and array constraint parameters to obtain each candidate beam. The link quality prediction results, beam mismatch risk prediction results, constraint feasibility prediction results, and link reliability prediction results corresponding to the control actions are obtained. The joint state vector, the interference risk characterization quantity, and the prediction results corresponding to each candidate beam control action are input into the reinforcement learning guidance strategy model to obtain the target beam control action. The target beam control action is corrected by safety constraint prediction control to obtain the safe beam control action. The amplitude and phase of each element of the phased array antenna are controlled according to the safe beam control action to form the target guidance beam, and the graph neural network, the digital twin environment, and the reinforcement learning guidance strategy model are updated online according to the guidance feedback.
[0008] To achieve the aforementioned objective, this invention also provides a beam guidance system based on a phased array antenna, comprising: a state construction module for acquiring array state information, terminal state information, pilot measurement information, environmental perception information, and historical guidance feedback information of the phased array antenna, and constructing a joint state vector; a graph modeling module for constructing a scene interaction graph including target nodes, interference nodes, reflection nodes, and occlusion nodes based on the joint state vector, and extracting interference risk characterization quantities using a graph neural network; and a digital twin pre-simulation module for performing parallel pre-simulation of multiple candidate beam control actions in a digital twin environment based on the joint state vector, the interference risk characterization quantities, and array constraint parameters, to obtain the chain corresponding to each candidate beam control action. The system includes: a path quality prediction result, a beam mismatch risk prediction result, a constraint feasibility prediction result, and a link reliability prediction result; a guidance decision module, used to input the joint state vector, the interference risk characterization quantity, and the prediction results corresponding to each candidate beam control action into the reinforcement learning guidance strategy model to obtain the target beam control action; a safety control module, used to perform safety constraint prediction control correction on the target beam control action to obtain a safe beam control action; an array execution module, used to control the amplitude and phase of each element of the phased array antenna to form the target guidance beam according to the safe beam control action; and a closed-loop update module, used to update the graph neural network, the digital twin environment, and the reinforcement learning guidance strategy model online based on guidance feedback.
[0009] Compared with the prior art, the present invention has at least the following beneficial effects:
[0010] First, by constructing a joint state vector, this invention unifies the representation of array state, terminal state, link observation information, environmental perception information, and historical guidance feedback information, thereby improving the global perception capability of beam guidance for complex dynamic scenarios and overcoming the problem that traditional methods rely solely on a single received signal strength or a single measurement quantity for guidance.
[0011] Second, by constructing a scene interaction graph including target nodes, interference nodes, reflection nodes, and occlusion nodes, and by using graph neural networks to extract interference coupling relationships and propagation correlation features, this invention can more accurately reflect the complex influence relationships between multiple entities, thereby improving the anti-interference capability and decision accuracy of beamguide in complex interference environments.
[0012] Third, by performing parallel pre-simulation of multiple candidate beam control actions in a digital twin environment, this invention obtains link quality prediction results, beam mismatch risk prediction results, and constraint feasibility prediction results before actual execution. This enables the beam guidance process to have forward-looking prediction and risk assessment capabilities, improves the effectiveness of beam selection, and reduces guidance overhead and mismatch risk caused by blind trial and error.
[0013] Fourth, this invention achieves adaptive optimization of beam control actions in dynamic scenarios by inputting the joint state vector, interference risk representation quantity, and prediction results output by the digital twin environment into the reinforcement learning guidance strategy model, thereby improving the intelligent decision-making and continuous optimization capabilities of beam guidance in complex time-varying environments.
[0014] Fifth, by introducing safety constraint predictive control, this invention integrates array element power constraints, sidelobe constraints, beam switching rate constraints, and link reliability constraints into the target beam control action correction process, thereby avoiding the engineering unexecutability problem caused by only pursuing local link gain and improving the safety, feasibility, and stability of the output beam control action.
[0015] Sixth, this invention achieves closed-loop adaptive correction of the model to environmental changes, propagation drift, and hardware mismatch by updating the reinforcement learning guidance strategy model, graph neural network, and propagation parameters in the digital twin environment online based on real guidance feedback, thereby improving the robustness, generalization ability, and guidance accuracy of the system during long-term operation.
[0016] Seventh, this invention organically couples digital twin pre-simulation, graph neural network interference modeling, reinforcement learning beamguide, and safety constraint predictive control to construct a closed-loop technical link of "graph modeling - twin pre-simulation - intelligent decision-making - safety correction - feedback update". This enables the simultaneous consideration of beamguide accuracy, link reliability, anti-interference capability, control security, and online self-evolution capability in complex dynamic communication scenarios, and has high engineering application value. Attached Figure Description
[0017] Figure 1 This is a flowchart of the beam guidance method based on a phased array antenna provided by the present invention. Detailed Implementation
[0018] The invention will be further described below with reference to the accompanying drawings.
[0019] Figure 1 This is a flowchart of the beam guidance method based on a phased array antenna provided by the present invention. Figure 1 As shown, this invention provides a beam guidance method based on a phased array antenna, executed by a processor calling instructions from memory. The method is designed for dynamic communication scenarios, unifying the modeling of array state, terminal state, environmental awareness information, and historical guidance feedback. It extracts interference coupling relationships using a graph neural network, pre-simulates candidate beam control actions using a digital twin environment, generates target beam control actions using a reinforcement learning guidance strategy model, corrects the actions using safety constraint predictive control, controls each element of the phased array antenna to form the target guidance beam, and finally updates each model in a closed loop based on real feedback, specifically including steps S1 to S7.
[0020] S1, Joint State Construction
[0021] Specifically, it includes:
[0022] The system acquires array state information, terminal state information, pilot measurement information, environmental perception information, and historical guidance feedback information from the phased array antenna. It then performs time alignment, scale normalization, and feature stitching on this heterogeneous data to construct a joint state vector for subsequent graph modeling, twin simulation, and guidance decision-making. .
[0023] The array status information includes at least the health status of array elements, the upper limit of array element power, the phase offset of array elements, the amplitude offset of array elements, and the current array weighting vector; the terminal status information includes at least the terminal position, velocity, heading angle, and attitude change information; the pilot measurement information includes at least the received power, signal-to-noise ratio, pilot correlation peak value, multipath delay spread, and angle estimation information; the environmental perception information includes at least the distribution of reflectors, distribution of obstructions, distribution of potential interference sources, and scene geometric boundary information; the historical guidance feedback information includes at least the beam control actions of the previous moment or several moments before, the trend of link quality changes, the trend of bit error rate changes, and the trend of azimuth deviation changes.
[0024] The joint state vector is represented as:
[0025] ,
[0026] in: Indicates time The joint state vector, Indicates time Array status information, Indicates time Terminal status information, Indicates time Pilot measurement information, Indicates time Environmental perception information, Indicates time Historical guidance and feedback information.
[0027] In an alternative embodiment, it can be , , , and The state inputs are encoded into fixed-length vectors and then concatenated to form a unified dimension. For missing observations, mask vectors and nearest-neighbor time-interpolation can be used to fill in the missing observations in order to maintain the stability of the state structure.
[0028] In this step, the present invention constructs a joint state vector by unifying array state information, terminal state information, pilot measurement information, environmental perception information, and historical guidance feedback information. This provides a unified and time-consistent input basis for subsequent interference relationship modeling, candidate beam action pre-simulation, and beam guidance decision-making, thereby improving the system's overall perception capability of complex dynamic scenes and avoiding the problem of insufficient information caused by traditional methods that rely solely on the strength of a single received signal for guidance.
[0029] S2. Scene Interaction Graph Construction and Graph Neural Network Interference Modeling
[0030] Specifically, it includes:
[0031] Joint state vector based on S1 output Build scene interaction graph The scene interaction graph includes target nodes, interference nodes, reflection nodes, and occlusion nodes, and uses edge relationships to represent the interference associations, reflection associations, occlusion associations, and reachability associations between nodes. Subsequently, the scene interaction graph is input into a graph neural network to extract the interference coupling relationships and propagation association features between multiple entities in the scene, obtaining the interference risk representation. .
[0032] The scene interaction diagram is represented as follows:
[0033]
[0034] in: Indicates time Scene interaction diagram, Represents a set of nodes. Denotes the set of edges. This represents the set of node features and edge features.
[0035] Wherein, the set of nodes It includes at least: target nodes representing the target communication object, interference nodes representing potential external interference emitters, reflection nodes representing specular reflection or scattering propagation media, and blocking nodes representing obstruction of the propagation path; the edge set It includes at least the interference edge between the target node and the interfering node, the reflection edge between the target node and the reflecting node, the occlusion edge between the target node and the occluding node, and the propagation influence edge between the interfering node and the reflecting node and the occluding node.
[0036] The graph neural network performs message passing and feature aggregation on the scene interaction graph to obtain the interference risk representation quantity. , can be represented as:
[0037]
[0038] in: Indicates time Interference risk characterization quantity, The parameter is Graph neural networks.
[0039] In one embodiment, the graph neural network may employ a graph convolutional network, a graph attention network, a message-passing neural network, or a combination thereof. The initial features of a node may be derived from a joint state vector. The relevant sub-vectors are mapped to obtain features, such as mapping terminal orientation and velocity to target node features, mapping interference source power and direction to interference node features, mapping reflection coefficient and normal information to reflection node features, and mapping occlusion size and material attenuation information to occlusion node features. Edge features can be calculated from the relative distance between nodes, relative orientation, occlusion degree, and propagation reachability.
[0040] In this invention, the graph neural network can be trained using a combination of offline supervised training and online incremental updates. During the offline training phase, historical scene samples are collected to construct a historical scene interaction graph and its corresponding real interference level labels, interference event labels, or link degradation labels. The graph neural network is then trained to predict interference risk representation results from the graph structure. During the online operation phase, based on real link feedback after real-time guidance, the graph neural network parameters are updated incrementally with small steps to adapt to changes in the environmental structure.
[0041] In one embodiment, the training loss of a graph neural network can be defined as:
[0042]
[0043] in: This represents the training loss of a graph neural network. Represents the number of training samples. Indicates the first The predicted interference risk value for each sample Indicates the first The true risk value of interference for each sample.
[0044] In this step, the present invention uses a joint state vector as the basis. Building a scene interaction graph And use graph neural networks to extract interference risk characterization quantities. This incorporates the complex interactions between target nodes, interference nodes, reflection nodes, and occlusion nodes into a unified graph structure representation, thereby improving the system's ability to model complex interference coupling and propagation correlations and enhancing the accuracy of beamguide decision-making in dynamic multi-entity scenarios.
[0045] S3, Parallel Pre-simulation of Candidate Beam Control Actions in Digital Twin Environment
[0046] Specifically, it includes:
[0047] Joint state vector based on S1 output Interference risk characterization quantity output by S2 In addition to array constraint parameters, multiple candidate beam control actions are generated in the digital twin environment, and each candidate beam control action is pre-simulated in parallel to obtain the link quality prediction results, beam mismatch risk prediction results, constraint feasibility prediction results, and link reliability prediction results corresponding to each candidate beam control action.
[0048] The set of candidate beam control actions can be represented as:
[0049] ,
[0050] in: Indicates time The set of candidate beam control actions, Indicates time The One candidate beam control action, This indicates the number of candidate beam control actions.
[0051] The candidate beam control action This can include low-dimensional encoding of target pointing angle, beamwidth, null direction, power allocation factor, or array control parameters. The digital twin environment is based on the current state. Interference risk characterization quantity and candidate beam control actions Calculate the propagation and link response of candidate actions in the short time domain in the future, and output the prediction result vector:
[0052] ,
[0053] in: Indicates time The The prediction result vector corresponding to each candidate beam control action. Indicates the first Link quality prediction results corresponding to each candidate beam control action. Indicates the first Beam mismatch risk prediction results for each candidate beam control action Indicates the first Constraint feasibility prediction results for each candidate beam control action Indicates the first Link reliability prediction results corresponding to each candidate beam control action.
[0054] In one embodiment, the digital twin environment may include an array response submodule, a propagation path submodule, an obstruction attenuation submodule, a reflection propagation submodule, and an interference superposition submodule, which are respectively used to approximate the phased array antenna radiation, propagation loss, obstacle obstruction, environmental reflection, and interference superposition processes in a real physical scenario. For each candidate beam control action... Digital twin environments can all output corresponding prediction result vectors. .
[0055] Digital twin environments can be constructed using a "mechanism initialization + data calibration" approach. First, an initial twin environment is established based on array structure parameters, geometric propagation models, and electromagnetic approximation models. Then, data-driven calibration of propagation parameters is performed using historical real-world link samples. Finally, during the online phase, the twin environment parameters are continuously adjusted based on current real-world feedback.
[0056] In one embodiment, the training loss of a digital twin environment can be defined as:
[0057]
[0058] in: This represents the training loss of the digital twin environment. Represents the number of training samples. and They represent the first The predicted link quality and the actual link quality of each sample. and They represent the first Predicted beam mismatch risk and actual beam mismatch risk for each sample and They represent the first Predicted link reliability and actual link reliability for each sample. , and This represents the loss weighting coefficient.
[0059] In this step, the present invention controls the set of candidate beam control actions in a digital twin environment. Perform parallel pre-simulation and output the prediction result vector. This allows the system to predict the link quality, beam mismatch risk, constraint feasibility, and link reliability corresponding to different candidate actions before actual execution. This transforms the traditional passive guidance method of "execute first, then feedback" into an active guidance method of "pre-rehearse first, then execute," reducing the link jitter and mismatch risks caused by blind trial and frequent switching.
[0060] S4. Reinforcement learning-guided strategy model generates target beam control actions.
[0061] Specifically, it includes:
[0062] The joint state vector output by S1 Interference risk characterization quantity output by S2 and the set of candidate action prediction result vectors output by S3 Common input reinforcement learning guided policy model, output time Target beam control action .
[0063] In one embodiment, the reinforcement learning-guided policy model may consist of a policy network and a value network. The policy network controls actions based on the current input-output target beam, while the value network estimates the long-term reward in the current state to assist in policy optimization. The policy network can be represented as:
[0064]
[0065] in: Indicates time Target beam control actions, The parameter is The reinforcement learning-guided strategy model; Represents a set The total number of elements in the text.
[0066] In one embodiment, the value estimate can be expressed as:
[0067]
[0068] in: Indicates time State value estimation, The parameter is The value network Indicates time The interference risk characterization quantity.
[0069] The immediate reward following guidance can be defined as:
[0070]
[0071] in: Indicates time The reward value, Indicates time The actual link quality metrics, Indicates time The cost of beam mismatch Indicates time The cost of sidelobe leakage, Indicates time The cost of beam switching disturbance, , , and This represents the corresponding weight coefficient, and all of them are real numbers greater than 0.
[0072] in, It can be calculated from the received power, signal-to-noise ratio, and bit error rate. It can be calculated from the angular deviation between the target direction and the actual beam main lobe direction, as well as the degree of degradation in backhaul quality. It can be calculated from the sidelobe energy leakage level; Actions can be controlled by the current target beam. The change in amplitude between the target beam control action and the previous moment is calculated.
[0073] In this invention, the reinforcement learning-guided strategy model can be trained using a combination of offline pre-training and online fine-tuning. During the offline pre-training phase, historical interaction data or data generated from digital twin environment simulations are used to construct trajectory samples. This is used to train the policy network and value network; during the online operation phase, the model parameters are updated continuously based on new samples obtained from real-time interaction. This is the joint state vector at time t+1.
[0074] In one embodiment, an Actor-Critic or PPO training framework can be used; correspondingly, the policy update objective can be expressed as:
[0075]
[0076] in: This represents the training loss of the reinforcement learning-guided policy model. Represents the number of training samples. Indicates input Take action under the condition The strategy probability, Indicates the first The policy input for each sample consists of a joint state vector, a disturbance risk representation, and candidate action prediction results. Indicates the first The advantage function estimate for each sample.
[0077] Value network loss can be expressed as:
[0078]
[0079] in: This represents the training loss of the value network. Indicates the first The value network output of each sample Indicates the first The target value of each sample.
[0080] The input for S4 is , and The output is the target beam control action. The aforementioned This will serve as input to S5 security constraint predictive control to prevent actions directly output by the policy network from being infeasible at the physical execution level.
[0081] In this step, the present invention uses the joint state vector Interference risk characterization quantity and candidate action prediction result vector set The common input is a reinforcement learning-guided policy model, which outputs target beam control actions. This enables the system to adaptively optimize beamguide decisions based on environmental conditions, interference coupling, and the future effects of candidate actions, thereby improving the accuracy, continuity, and long-term benefits of beamguide in complex dynamic scenarios.
[0082] S5. Safety Constraint Predictive Control and Safety Beam Control Action Generation
[0083] Specifically, it includes:
[0084] Control the target beam output from S4. The input safety constraint prediction control module projects or corrects the target beam control action within the feasible region to obtain a safe beam control action that satisfies power constraints, sidelobe constraints, beam switching rate constraints, and link reliability constraints. .
[0085] Safety constraints can be expressed as:
[0086]
[0087] in: Indicates time The The transmission power of each array element Indicates the first The maximum permissible transmit power of each array element This indicates the total number of array elements.
[0088]
[0089] in: Indicates time The sidelobe level, This indicates the maximum permissible sidelobe level.
[0090]
[0091] in: Indicates time Target beam control actions, Indicates time Target beam control actions, This indicates the beam switching rate threshold.
[0092]
[0093] in: Represents probability. Indicates future time The predicted link quality value, Indicates the lower limit of link quality. Indicates the reliability threshold, and .
[0094] In one embodiment, safety constraint predictive control can obtain the safety beam control action by solving the following modified optimization problem:
[0095]
[0096] in: Indicates time Safety beam control actions, Represents the optimization variable. Indicates time Target beam control actions.
[0097] The constraints of the aforementioned optimization problem are the power constraint, sidelobe constraint, beam switching rate constraint, and link reliability constraint. That is, while preserving as much of the original action intent of the reinforcement learning guidance strategy model as possible, it undergoes minimum deviation correction to obtain physically executable and engineering-safe safe beam control actions. .
[0098] The input to S5 is the target beam control action. In addition to the constraint feasibility prediction results and link reliability prediction results from the S3 pre-simulation results, the output is the safe beam control action. The aforementioned It will be used as the direct input for execution of the S6 array.
[0099] This invention controls the target beam action. Perform safety constraint predictive control correction to obtain the safety beam control action. This integrates element power, sidelobe level, beam switching smoothness, and future link reliability into the constraint screening process before action execution, thereby avoiding reinforcement learning models from pursuing only local rewards and outputting physically unexecutable or engineering-unsafe actions, and improving the feasibility and operational stability of beam guidance.
[0100] S6. Acquisition of Phased Array Antenna Array Execution and Guidance Results
[0101] Specifically, it includes:
[0102] Based on the safety beam control action output by S5 Generate array weighted vector It controls the amplitude and phase of each element of the phased array antenna to form a target guidance beam; then it collects the received power, signal-to-noise ratio, bit error rate, azimuth deviation and return quality indicators after guidance, as the real feedback input for S7 closed-loop update.
[0103] The array weighted vector can be represented as:
[0104]
[0105] in: Indicates time The array of complex weighted vectors, Indicates time The The weighted values of each array element, Indicates the total number of array elements. This indicates transpose.
[0106] In one embodiment, secure beam control action With array weighted vector The mappings can be obtained through lookup table mappings, analytical beamforming function mappings, or neural network decoding mappings. If... If expressed in terms of target pointing angle, beamwidth, and null direction, the complex weights of each array element can be further calculated. ;like Since it is itself a low-dimensional encoding of the array control parameters, the array weighting vector can be directly recovered by the decoder. .
[0107] In one embodiment, the array pattern can be represented by the array factor as follows:
[0108]
[0109] in: Indicates time In direction The array factor at that location, Indicates time The The weighted values of each array element, Represents the imaginary unit. Indicates the operating wavelength. Indicates the distance between adjacent array elements. Indicates spatial orientation angle.
[0110] The actual feedback results after guidance should include at least: received power. Signal-to-noise ratio Bit error rate Orientation deviation and return quality indicators Among them, the aforementioned Will participate in the S4 reward value The calculation, It can be used to estimate beam mismatch cost , It can be used to evaluate the stability of the link after beamguide.
[0111] The input to S6 is the safety beam control action. The output is an array of complex weighted vectors. and real feedback results The actual feedback results will be directly used as input for the S7 closed-loop update to correct the model and environmental parameters.
[0112] This invention controls actions based on a secure beam. Generate array weighted vector It controls each element of the phased array antenna to perform beamforming, while collecting real guidance feedback results, thereby achieving a consistent mapping from "decision-level actions" to "physical array execution" and providing real measurement basis for subsequent model updates. This improves the accuracy of beam control implementation and enhances the verifiability of the system's closed loop.
[0113] S7, Closed-loop feedback update and self-evolving beamguide control
[0114] Specifically, it includes:
[0115] Based on the actual feedback results collected by S6 Construct reward value The model correction signal is used to update the propagation parameters in the reinforcement learning guidance strategy model, graph neural network, and digital twin environment online to form a closed-loop beamguide control.
[0116] First, construct the beam mismatch cost based on real feedback. Side lobe leakage cost and beam switching disturbance cost .in, It can be caused by azimuth deviation and return quality indicators calculate, It can be estimated from the sidelobe energy of the array pattern. Actions can be controlled by the current target beam. The difference between the target beam control action and the previous action is calculated. Then, the reward value is calculated based on the reward function defined in S4. .
[0117] Secondly, based on the prediction result vector output by the digital twin environment in S3 Based on the differences between the actual feedback results in S6, a propagation parameter correction signal is constructed to update the path loss parameters, reflection parameters, occlusion attenuation parameters, and array element error parameters in the digital twin environment; based on the interference risk characterization output by the graph neural network in S2... The deviation from the actual link degradation performance is used to construct the incremental update signal of the graph neural network; based on the policy execution result and reward value... We construct online update samples for reinforcement learning-guided strategy models and value networks.
[0118] In one embodiment, the online propagation parameter update of the digital twin environment can employ gradient descent, recursive least squares, or Kalman filtering; the graph neural network can use mini-batch incremental updates; and the reinforcement learning guided policy model can use empirical replay and small-step policy updates. Real-time interactive samples can be represented as:
[0119]
[0120] in: Indicates time Interaction samples, Indicates time The joint state vector, Indicates time Interference risk characterization quantity, Indicates time The set of candidate action prediction result vectors, Indicates time Target beam control actions, Indicates time Safety beam control actions, Indicates time The reward value, Indicates time The joint state vector.
[0121] In one embodiment, after online updates, the reinforcement learning guided policy model parameters Value network parameters Graph Neural Network Parameters Furthermore, the parameters of the digital twin environment can be adjusted to better suit the current dynamic scenario, so that the beam guidance decision at the next moment can inherit the real feedback experience of the current moment.
[0122] This invention constructs reward values by utilizing real guidance feedback results. The system continuously corrects policy bias, scenario modeling bias, and propagation prediction bias by updating the reinforcement learning guidance policy model, graph neural network, and propagation parameters in the digital twin environment online. This reduces the mismatch between the model and the real environment, improves long-term operational stability, and enhances the system's adaptability and robustness in complex dynamic scenarios.
[0123] The data coupling relationship between the steps in this embodiment is as follows:
[0124] S1 outputs the joint state vector. The Simultaneous input to S2, S3, and S4 forms the unified state basis of the entire method; S2 is based on... Building a scene interaction graph and output interference risk characterization quantity. The Inputs to S3 and S4 are used to characterize the multi-entity interference coupling state; S3 is based on , and candidate beam control action set Output prediction result vector set The set of predicted result vectors is input to S4 to assist the reinforcement learning guidance policy model in making action decisions, while a portion of the predicted results is input to S5 as a basis for safety constraint judgment; S4 is based on , and Output target beam control action The Input to S5; S5 to Perform safety corrections and output safety beam control actions. The Input to S6; S6 according to Generate array weighted vector The control array executes and outputs real feedback results. The actual feedback result is input to S7; S7 constructs a reward value based on the actual feedback result, the prediction result, and the historical actions. The correction signal is used to update the model parameters and then fed back to the next time step S1-S4 to form a closed loop.
[0125] Thus, this implementation method forms a continuous technical chain of "state construction - graph modeling - twin pre-simulation - strategy guidance - security correction - array execution - feedback update".
[0126] This implementation method uses the joint state vector Scene interaction diagram Interference risk characterization quantity Candidate beam control action set Prediction result vector Target beam control action Safety beam control action and array weighted vector By integrating all aspects into a single closed-loop beamguide framework, unified modeling, pre-simulation decision-making, safety control, and feedback updates for the phased array antenna beamguide process in complex dynamic scenarios are achieved. This technical solution improves the accuracy, reliability, safety, and continuous adaptive capability of beamguide in complex interference, dynamic obstruction, and rapid movement scenarios.
[0127] According to one embodiment of the present invention, a beam guidance system based on a phased array antenna is also provided, comprising: a state construction module, used to acquire array state information, terminal state information, pilot measurement information, environmental perception information, and historical guidance feedback information of the phased array antenna, and construct a joint state vector; a graph modeling module, used to construct a scene interaction graph including target nodes, interference nodes, reflection nodes, and occlusion nodes based on the joint state vector, and extract interference risk characterization quantities using a graph neural network; and a digital twin pre-simulation module, used to perform parallel pre-simulation of multiple candidate beam control actions in a digital twin environment based on the joint state vector, the interference risk characterization quantities, and array constraint parameters, to obtain the links corresponding to each candidate beam control action. The system includes: quality prediction results, beam mismatch risk prediction results, constraint feasibility prediction results, and link reliability prediction results; a guidance decision module, used to input the joint state vector, the interference risk characterization quantity, and the prediction results corresponding to each candidate beam control action into the reinforcement learning guidance strategy model to obtain the target beam control action; a safety control module, used to perform safety constraint prediction control correction on the target beam control action to obtain a safe beam control action; an array execution module, used to control the amplitude and phase of each element of the phased array antenna to form the target guidance beam according to the safe beam control action; and a closed-loop update module, used to update the graph neural network, the digital twin environment, and the reinforcement learning guidance strategy model online based on guidance feedback.
[0128] The present invention has been described in detail above with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions, simple transformations, combinations, adjustments, or conventional improvements made by those skilled in the art to the technical solutions of the present invention without departing from the concept and essence of the present invention shall fall within the scope of protection of the present invention. The technical features in the various embodiments of the present invention can be combined with each other to form new embodiments without conflict; all technical solutions directly derived or conceived based on the content of the present invention specification and drawings shall be regarded as the content disclosed in the present invention.
Claims
1. A beamguiding method based on a phased array antenna, characterized in that, Executed by the processor, the process includes: acquiring array state information, terminal state information, pilot measurement information, environmental perception information, and historical guidance feedback information of the phased array antenna, and constructing a joint state vector; constructing a scene interaction graph including target nodes, interference nodes, reflection nodes, and obstruction nodes based on the joint state vector, and extracting interference risk characterization parameters using a graph neural network; and performing parallel pre-simulation of multiple candidate beam control actions in a digital twin environment based on the joint state vector, the interference risk characterization parameters, and array constraint parameters, to obtain the link quality prediction results and wave count prediction results corresponding to each candidate beam control action. The beam mismatch risk prediction results, constraint feasibility prediction results, and link reliability prediction results are used to obtain the target beam control action. The joint state vector, the interference risk characterization quantity, and the prediction results corresponding to each candidate beam control action are input into the reinforcement learning guidance strategy model to obtain the target beam control action. The target beam control action is then corrected by safety constraint prediction control to obtain the safe beam control action. The amplitude and phase of each element of the phased array antenna are controlled according to the safe beam control action to form the target guidance beam. The graph neural network, the digital twin environment, and the reinforcement learning guidance strategy model are updated online based on the guidance feedback.
2. The beamguiding method based on a phased array antenna according to claim 1, characterized in that, The joint state vector is: in, For a moment The joint state vector, For a moment Array status information, For a moment Terminal status information, For a moment Pilot measurement information, For a moment Environmental perception information, For a moment Historical guidance and feedback information.
3. The beamguiding method based on a phased array antenna according to claim 1, characterized in that, The scene interaction diagram is as follows: in, For a moment Scene interaction diagram, For a set of nodes, Let be the set of edges. The graph neural network outputs the following disturbance risk representation quantity: (The graph features are a set of node features and edge features.) in, For a moment Interference risk characterization quantity, For parameters Graph neural networks.
4. The beamguiding method based on a phased array antenna according to claim 1, characterized in that, The multiple candidate beam control actions constitute a candidate beam control action set: in, For a moment The set of candidate beam control actions, For a moment The One candidate beam control action, The number of candidate beam control actions; the digital twin environment for the first The output prediction vector of each candidate beam control action: in, For a moment The The prediction result vector corresponding to each candidate beam control action. For the link quality prediction results, The results of beam mismatch risk prediction To constrain the feasibility prediction results, This is the link reliability prediction result.
5. The beamguiding method based on a phased array antenna according to claim 1, characterized in that, The expression for the target beam control action output by the reinforcement learning guidance strategy model is: in, For a moment Target beam control actions, For parameters The reinforcement learning guided strategy model, For a moment The set of prediction result vectors corresponding to each candidate beam control action.
6. The beamguiding method based on a phased array antenna according to claim 1, characterized in that, The reinforcement learning guidance strategy model is trained based on a reward function, which is: in, For a moment The reward value, For a moment The actual link quality metrics, For a moment The cost of beam mismatch For a moment The cost of sidelobe leakage, For a moment The cost of beam switching disturbance, , , and These are weighting coefficients, and all are greater than 0.
7. The beamguiding method based on a phased array antenna according to claim 1, characterized in that, The safety constraint predictive control satisfies the following conditions: ， ， ， ， in, For a moment No. The transmission power of each array element For the first The maximum permissible transmit power of each array element The total number of array elements. For a moment The sidelobe level, For the maximum permissible sidelobe level, and They are time points and time Target beam control actions, The beam switching rate threshold, For the future The predicted link quality value, This represents the lower limit of link quality. The reliability threshold and satisfying .
8. The beamguiding method based on a phased array antenna according to claim 7, characterized in that, The secure beam control action is obtained through the following optimizations: in, For a moment Safety beam control actions, To optimize variables, For a moment The target beam control action; the optimization satisfies the power constraint, sidelobe constraint, beam switching rate constraint and link reliability constraint as described in claim 7.
9. The beamguiding method based on a phased array antenna according to claim 1, characterized in that, Generate array weighted vectors based on the aforementioned safe beam control actions: in, For a moment The array of complex weighted vectors, For a moment No. The weighted values of each array element, The total number of array elements. It is a transpose; and the amplitude and phase of each element of the phased array antenna are controlled based on the array weighted vector to form a target guiding beam.
10. A beamguiding system based on a phased array antenna, characterized in that, include: The state construction module is used to acquire array state information, terminal state information, pilot measurement information, environmental perception information and historical guidance feedback information of the phased array antenna, and construct a joint state vector; the graph modeling module is used to construct a scene interaction graph including target nodes, interference nodes, reflection nodes and occlusion nodes based on the joint state vector, and use graph neural networks to extract interference risk characterization quantities. The twin pre-simulation module is used to perform parallel pre-simulation of multiple candidate beam control actions in a digital twin environment based on the joint state vector, the interference risk characterization quantity, and array constraint parameters, to obtain the link quality prediction result, beam mismatch risk prediction result, constraint feasibility prediction result, and link reliability prediction result corresponding to each candidate beam control action; the guidance decision module is used to input the joint state vector, the interference risk characterization quantity, and the prediction results corresponding to each candidate beam control action into the reinforcement learning guidance strategy model to obtain the target beam control action; The safety control module is used to perform safety constraint prediction control correction on the target beam control action to obtain a safe beam control action. The array execution module is used to control the amplitude and phase of each element of the phased array antenna to form a target guidance beam according to the security beam control action; the closed-loop update module is used to update the graph neural network, the digital twin environment and the reinforcement learning guidance strategy model online according to the guidance feedback.