Satellite antenna automatic pointing method based on PPO reinforcement learning

Abstract

The application provides a satellite antenna automatic pointing method based on PPO reinforcement learning, comprising: initializing an antenna controller, a sensor group and a satellite orbit prediction module, loading a pre-trained PPO reinforcement learning model, defining a state space and a continuous action space; based on real-time antenna angle and signal strength data obtained by the sensor group, combining theoretical position data calculated by the satellite orbit prediction module, performing data fusion processing to generate a dynamic state vector; based on the dynamic state vector, performing policy inference processing through an Actor network to generate a continuous action, and performing safety evaluation processing on the continuous action through a Critic network to obtain a safety decision result; based on the obtained state change data after the antenna rotates, performing dynamic reward calculation to obtain a reward value, and storing the reward value and the state change data into an experience pool. By using the method, high-precision pointing control can be realized in a dynamic environment through real-time policy optimization of the continuous action space.

Description

Technical Field

[0001] This invention belongs to the field of communication technology, and in particular relates to an automatic satellite antenna alignment method based on PPO reinforcement learning. Background Technology

[0002] With the rapid development of satellite communication technology, automatic satellite alignment technology has become a core component in ensuring stable signal transmission. This field has seen the emergence of mainstream solutions such as gradient scanning, rule-based satellite tracking algorithms, and heatmap-based classification reinforcement learning methods. For example, traditional scanning techniques achieve coarse and fine satellite alignment in stages through a preset path (such as the hierarchical scanning mechanism proposed in CN104659488B), while emerging reinforcement learning methods (such as the DQN algorithm proposed in CN115081519B) rely on discrete action spaces and pre-stored signal heatmaps for decision-making. Traditional techniques typically employ a fixed process: first, the antenna angle is initialized through theoretical calculations; then, periodic scanning or discrete angle adjustments are used to approximate the signal peak value; finally, a preset threshold is used to determine if satellite alignment is complete.

[0003] However, existing automatic satellite alignment methods have fundamental bottlenecks: their rule-based frameworks cannot dynamically learn environmental changes (such as satellite drift and weather disturbances), resulting in satellite alignment accuracy being limited by discrete actions. Furthermore, scanning algorithms converge slowly, while reinforcement learning schemes require a large amount of pre-stored heatmap data, have poor real-time performance, and are difficult to adapt to sudden interference. Summary of the Invention

[0004] Therefore, it is necessary to provide an automatic satellite antenna alignment method based on PPO reinforcement learning to address the above-mentioned technical problems. This method can achieve high-precision satellite alignment control in dynamic environments through real-time strategy optimization in continuous action space.

[0005] Firstly, this application provides an automatic satellite antenna alignment method based on PPO reinforcement learning, including:

[0006] Initialize the antenna controller, sensor group and satellite orbit prediction module, load the pre-trained PPO reinforcement learning model, and define the state space and continuous action space;

[0007] Based on the real-time antenna angle and signal strength data acquired by the sensor group, combined with the theoretical position data calculated by the satellite orbit prediction module, data fusion processing is performed to generate a dynamic state vector.

[0008] Based on dynamic state vectors, policy reasoning is performed through the Actor network to generate continuous actions, and the safety assessment of the continuous actions is performed through the Critic network to obtain safety decision results. The safety decision results are used to instruct the target antenna to adjust its angle.

[0009] Based on the acquired state change data after the antenna rotates, a dynamic reward is calculated to obtain the reward value. The reward value and state change data are stored in the experience pool, which is used to optimize the PPO reinforcement learning model online.

[0010] In one embodiment, based on real-time antenna angle and signal strength data acquired by the sensor array, and combined with theoretical position data calculated by the satellite orbit prediction module, data fusion processing is performed to generate a dynamic state vector, including:

[0011] The system acquires the real-time antenna azimuth and elevation angles collected by the sensor array, and obtains the theoretical azimuth and elevation angles calculated by the satellite orbit prediction module.

[0012] The deviation between the real-time antenna azimuth and the theoretical azimuth is calculated and processed to generate azimuth deviation data. The deviation between the real-time antenna elevation and the theoretical elevation is also calculated and processed to generate elevation deviation data.

[0013] Disturbance features are extracted from the wind speed data collected by the sensor array to generate environmental disturbance parameters;

[0014] A multi-source fusion processing is performed on azimuth deviation data, pitch deviation data, signal strength data, and environmental disturbance parameters to generate a dynamic state vector.

[0015] In one embodiment, based on a dynamic state vector, a policy reasoning process is performed through an Actor network to generate continuous actions, and a security assessment process is performed on the continuous actions through a Critic network to obtain a security decision result, including:

[0016] The dynamic state vector is input into the fully connected layer of the Actor network for feature abstraction to obtain a higher-order state representation.

[0017] Based on higher-order state representation, action probability distribution is modeled through Gaussian distribution sampling layer to generate continuous action sampling space;

[0018] Variance constraint processing is applied to the continuous motion sampling space, and the azimuth and pitch angle adjustments are output as continuous motions.

[0019] The dynamic state vector is input into the value assessment layer of the Critic network to predict the state value and obtain the expected return estimate.

[0020] Based on continuous actions, the trajectory deviation is calculated and processed through the action impact analysis layer to generate a risk coefficient.

[0021] The expected return valuation and risk coefficient are weighted and integrated to output a safe decision result that includes the priority of action execution.

[0022] In one embodiment, based on the acquired state change data after antenna rotation, a dynamic reward calculation is performed to obtain a reward value, and the reward value and state change data are stored in an experience pool, including:

[0023] The base reward value is obtained by calculating the gain reward on the state change data;

[0024] The satellite drift direction vector is obtained by extracting the direction from the satellite velocity vector output by the satellite orbit prediction module.

[0025] The azimuth and elevation adjustments after antenna rotation are vectorized to generate the actual antenna rotation direction vector.

[0026] Cosine similarity calculation is performed between the satellite drift direction vector and the antenna actual rotation direction vector to generate trajectory matching rewards;

[0027] The efficiency penalty value is obtained by penalizing the antenna rotation time using a preset time decay factor.

[0028] A reward value is generated by combining the base reward value, trajectory matching reward value, and efficiency penalty value.

[0029] The reward value, state change data and corresponding dynamic state vector are associated and stored to generate experience data units.

[0030] Priority assignment is performed on the experience data units to obtain priority-bearing experience data, and the priority-bearing experience data is stored in the experience pool.

[0031] In one embodiment, online optimization of the PPO reinforcement learning model includes:

[0032] Prioritize and sort the experience data stored in the experience pool to generate a high-value sample set;

[0033] Based on a high-value sample set, the policy gradient is calculated through importance sampling to obtain the policy update gradient;

[0034] A pruning objective function is used to constrain the gradient of the policy update, and the parameters of the Actor network and the Critic network are updated online.

[0035] In one embodiment, the antenna controller, sensor group, and satellite orbit prediction module are initialized, a pre-trained PPO reinforcement learning model is loaded, and the state space and continuous action space are defined, including:

[0036] Based on the obtained satellite antenna control requirements, the dimensions of the state space are determined;

[0037] Based on the constraints of the antenna's mechanical structure, the boundary range of the continuous operation space is determined;

[0038] The continuous action space is initialized with a Gaussian distribution to generate initial values ​​for the mean and variance of the action probability distribution.

[0039] Secondly, this application also provides an automatic satellite antenna alignment device based on PPO reinforcement learning, comprising:

[0040] The initialization module is used to initialize the antenna controller, sensor group and satellite orbit prediction module, load the pre-trained PPO reinforcement learning model, and define the state space and continuous action space.

[0041] The data fusion module is used to perform data fusion processing based on the real-time antenna angle and signal strength data obtained by the sensor group and the theoretical position data calculated by the satellite orbit prediction module to generate a dynamic state vector.

[0042] The dual-support neural network module is used to generate continuous actions based on dynamic state vectors through the Actor network for policy reasoning, and to perform safety assessment on the continuous actions through the Critic network to obtain safety decision results. The safety decision results are used to instruct the target antenna to adjust its angle.

[0043] The reinforcement learning module is used to dynamically calculate the reward value based on the acquired state change data after the antenna rotates. The reward value and state change data are stored in the experience pool, which is used to optimize the PPO reinforcement learning model online.

[0044] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described automatic satellite antenna alignment method based on PPO reinforcement learning.

[0045] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described automatic satellite antenna alignment method based on PPO reinforcement learning.

[0046] The aforementioned automatic satellite antenna alignment method based on PPO reinforcement learning initializes the antenna controller, sensor array, and satellite orbit prediction module, loads a pre-trained PPO reinforcement learning model, and defines a state space and a continuous action space to provide a basic framework for generating continuous actions. Based on real-time antenna angle and signal strength data acquired by the sensor array, combined with theoretical position data calculated by the satellite orbit prediction module, data fusion processing is performed to generate a dynamic state vector to capture dynamic environmental changes in real time. Based on this dynamic state vector, policy reasoning is performed through an Actor network to generate continuous actions, and a Critic network performs safety assessment on these actions to obtain a safety decision result, achieving high-precision satellite alignment angle adjustment. Based on the acquired state change data after antenna rotation, dynamic reward calculation is performed to obtain a reward value. The reward value and state change data are stored in an experience pool for online optimization of the PPO reinforcement learning model. This technical method can achieve high-precision satellite alignment control in a dynamic environment through real-time policy optimization in the continuous action space. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A flowchart illustrating an automatic satellite antenna alignment method based on PPO reinforcement learning provided in an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of a satellite antenna automatic alignment device based on PPO reinforcement learning, provided as an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] First, a brief introduction to the terms used in the embodiments of this application will be given.

[0052] Proximal Policy Optimization (PPO) reinforcement learning is an advanced policy optimization algorithm that achieves efficient policy iteration while ensuring training stability by introducing a proximal optimization mechanism and trust region constraints. The algorithm employs an Actor-Critic dual-network architecture, where the Actor network generates continuous action policies, and the Critic network evaluates state values ​​to guide policy updates. Its core innovation lies in limiting the magnitude of policy updates to avoid drastic fluctuations during training, ensuring the monotonicity of policy improvement and convergence reliability. PPO reinforcement learning is particularly suitable for control problems in high-dimensional continuous action spaces, and can autonomously adapt to dynamic environments through experience replay and online optimization mechanisms, achieving excellent generalization performance and robustness while maintaining algorithmic simplicity.

[0053] Automatic satellite alignment refers to the technology that enables an antenna to autonomously align itself with a target satellite through a closed-loop control system. This technology dynamically generates control commands based on real-time acquired antenna azimuth, elevation, and signal strength data, combined with satellite orbit prediction information, to drive the antenna rotation and ultimately maximize the received signal strength. Its core lies in continuously optimizing pointing accuracy through a feedback mechanism, overcoming the influence of dynamic factors such as satellite orbit perturbations and environmental interference, and ensuring the stability and transmission quality of the communication link. Compared to manual operation, this system significantly improves alignment efficiency, accuracy, and environmental adaptability, and is a key supporting technology for modern satellite communication systems.

[0054] The satellite orbit prediction module is a prediction system based on orbital dynamics models. By analyzing the satellite's orbital parameters (such as Kepler elements) and perturbation models, it accurately calculates the target satellite's spatial position and trajectory at future moments. This module solves the perturbed motion equations using numerical integration methods, outputting key parameters such as the satellite's right ascension, declination, azimuth, and elevation angles in real time, and integrating time system calibration and error compensation mechanisms. Its core function is to provide a high-precision target pointing reference for the antenna control system, overcoming the influence of dynamic factors such as Earth's rotation and orbital perturbations, ensuring the spatiotemporal consistency of satellite operation, and serving as a fundamental support unit for maintaining the stability of satellite communication links.

[0055] Based on the above definitions, the implementation environment of an automatic satellite antenna alignment method based on PPO reinforcement learning provided in this application embodiment will be described. Schematic, the implementation environment includes: a terminal, a sensor group, a satellite orbit prediction module, a processor, and a memory. The terminal is connected to the processor, sensing devices, and memory via a network. The sensor group includes, but is not limited to, angle sensors, signal strength sensors, environmental sensors, positioning devices, and vibration sensors. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), or an artificial intelligence chip (such as an NPU or TPU). The memory can be a distributed cloud storage system or a local server cluster, without limitation here.

[0056] Based on the above explanations of terms and implementation environments, the application scenarios of the embodiments of this application are described. The automatic satellite antenna alignment method based on PPO reinforcement learning provided in the embodiments of this application can be applied to scenarios including but not limited to the following:

[0057] In mobile platform satellite communication scenarios, such as satellite communication terminals on vehicle-mounted, ship-mounted, and airborne mobile platforms, traditional satellite pointing methods struggle to maintain high-precision pointing while the platform is in continuous motion. This solution utilizes a sensor array to collect real-time data on the platform's attitude, position, and signal strength. Combined with the dynamic strategy optimization capabilities of the PPO algorithm, it achieves autonomous tracking and compensation of the satellite orbit during movement, ensuring communication stability in mobile environments.

[0058] In communication missions that require frequent switching between different satellite orbits (such as global data relay), this solution utilizes the intelligent decision-making characteristics of reinforcement learning. Through a collaborative mechanism of pre-trained models and online optimization, it quickly generates the optimal pointing switching path. Based on multi-target location data provided by the satellite orbit prediction module, it dynamically adjusts the antenna action sequence, significantly improving the efficiency of cross-satellite switching.

[0059] For unmanned ground stations in remote areas used for power supply and meteorological monitoring, this solution overcomes the difficulties of manual maintenance through automated satellite alignment and continuous learning mechanisms. By accumulating environmental data and satellite alignment experience over a long period, the solution gradually optimizes control strategies to adapt to long-term changing factors such as seasonal changes and equipment aging, ensuring the long-term stable operation of the base station.

[0060] This is merely an illustrative example; the automatic satellite antenna alignment method based on PPO reinforcement learning provided in this application embodiment can also be applied to other application scenarios. It is only used as an example and is not intended to limit the specific application scenarios.

[0061] In one exemplary embodiment, such as Figure 1As shown, an automatic satellite antenna alignment method based on PPO reinforcement learning is provided. This embodiment illustrates the application of this method to a terminal in the aforementioned implementation environment. It is understood that this method can also be applied to a server, and also to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps 101 to 104:

[0062] Step 101: Initialize the antenna controller, sensor group and satellite orbit prediction module, load the pre-trained PPO reinforcement learning model, and define the state space and continuous action space.

[0063] Specifically, initialization operations are performed on the antenna controller, sensor array, and satellite orbit prediction module. For example, the initial angle and motion parameters of the antenna controller can be configured through a software interface, a pre-trained PPO reinforcement learning model can be loaded from local or remote memory, and a state space and continuous action space can be defined. The state space may include dimensions such as azimuth, elevation, and signal strength, while the action space defines the continuous adjustments in azimuth and elevation directions. The data for this step can come from the initial settings of the hardware components and / or stored model files. The processing includes parameter configuration and spatial dimension programming to ensure a stable foundation for subsequent real-time control, eliminate initial deviations, provide a scalable control environment, and avoid instability during startup. For example, the antenna controller can be initialized using a preset zero-point calibration; model loading can be achieved through API calls; and spatial dimensions can be defined using a programming language to set multi-dimensional arrays.

[0064] Step 102: Based on the real-time antenna angle and signal strength data acquired by the sensor group, and combined with the theoretical position data calculated by the satellite orbit prediction module, data fusion processing is performed to generate a dynamic state vector.

[0065] Specifically, based on the real-time antenna angle and signal strength data acquired by the sensor array, such as azimuth and elevation angles collected by the encoder and signal strength measured by the RSSI sensor, and combined with the theoretical position data calculated by the satellite orbit prediction module, such as the predicted azimuth and elevation angles of the satellite, data fusion processing is performed to generate a dynamic state vector. Furthermore, the processing includes fusing real-time measured values ​​with theoretical predicted values, for example, using the Kalman filter algorithm to reduce noise interference, or using the weighted average method to integrate multi-source information to generate a vector that comprehensively reflects the dynamics of the environment.

[0066] Step 103: Based on the dynamic state vector, policy reasoning is performed through the Actor network to generate continuous actions, and the continuous actions are then subjected to security assessment through the Critic network to obtain security decision results. The security decision results are used to instruct the target antenna to adjust its angle.

[0067] Specifically, the Actor network extracts state features through a multi-layer neural network, outputting continuous control values ​​for azimuth and pitch adjustments. For example, the adjustment accuracy reaches 0.01°, achieving smooth tracking that traditional discrete actions cannot. Furthermore, the Critic network receives this continuous action and performs a safety assessment: it predicts signal stability over the next 10 seconds through action trajectory simulation, evaluates structural risks using a mechanical stress model, and finally outputs a safety decision result including execution priorities. This mechanism ensures tracking accuracy while preventing mechanical overload accidents, addressing equipment damage caused by sudden action changes in reinforcement learning.

[0068] Step 104: Based on the obtained state change data after the antenna rotation, perform dynamic reward calculation to obtain the reward value, and store the reward value and state change data in the experience pool. The experience pool is used to optimize the PPO reinforcement learning model online.

[0069] Specifically, after the antenna adjusts its angle according to the safety decision, the sensor array acquires the new antenna angle and signal strength data again, along with the satellite orbit prediction module's recalculated theoretical satellite position data. This data is compared with the data before the adjustment to obtain state change data. Based on a preset reward function (e.g., positive reward factors like increased signal strength and reduced angle deviation, and negative reward factors like excessive adjustment and signal strength fluctuations), a dynamic reward value is calculated. The reward value and state change data are stored in an experience pool. Once the experience pool accumulates a certain amount of data, the reinforcement learning model is updated and optimized online using the PPO algorithm's optimization strategy to continuously improve the model's policy reasoning and safety assessment performance. For example, the reward function can be set as the difference between the increment of signal strength and the sum of the squares of the antenna adjustment angle. A higher positive reward value is obtained when the signal strength increases and the adjustment angle is appropriate; conversely, a lower or negative reward value is obtained. In this way, the model can continuously learn better satellite alignment strategies and safety assessment methods, improving the accuracy and stability of satellite alignment.

[0070] In one possible embodiment, the antenna controller, high-precision angle sensor, and satellite orbit prediction module can be initialized and configured. A pre-trained PPO reinforcement learning model is loaded, and a four-dimensional state space including azimuth, elevation, signal strength, and their gradients, as well as a two-dimensional continuous action space of azimuth and elevation, are defined. Subsequently, the antenna pointing angle and signal strength data are collected in real time through the sensor group, and the theoretical position information calculated by the satellite orbit prediction module is received synchronously. A dynamic state vector is generated using data fusion technology. This vector is input into an Actor network consisting of two 256-node fully connected layers for policy reasoning, and outputs continuous adjustment actions. At the same time, a Critic network with the same architecture is used to evaluate the safety boundary of the actions and generate a safety decision command to drive the antenna rotation. Finally, based on the state change data after the antenna rotation, the dynamic reward value is calculated by combining the signal strength increase, action amplitude, and convergence state. The reward value and state data are stored in the experience pool, driving the PPO model to update parameters online through a near-end policy optimization mechanism. This embodiment achieves high-precision satellite alignment control of ±0.1 degrees in both azimuth and elevation dimensions in a dynamic environment through closed-loop control and real-time strategy iteration, significantly improving convergence speed and environmental adaptability, and providing autonomous optimization capabilities for scenarios such as emergency communication and mobile platforms.

[0071] The aforementioned automatic satellite antenna alignment method based on PPO reinforcement learning initializes the antenna controller, sensor array, and satellite orbit prediction module, loads a pre-trained PPO reinforcement learning model, and defines a state space and a continuous action space to provide a basic framework for generating continuous actions. Based on real-time antenna angle and signal strength data acquired by the sensor array, combined with theoretical position data calculated by the satellite orbit prediction module, data fusion processing is performed to generate a dynamic state vector to capture dynamic environmental changes in real time. Based on this dynamic state vector, policy reasoning is performed through an Actor network to generate continuous actions, and a Critic network performs safety assessment on these actions to obtain a safety decision result, achieving high-precision satellite alignment angle adjustment. Based on the acquired state change data after antenna rotation, dynamic reward calculation is performed to obtain a reward value. The reward value and state change data are stored in an experience pool for online optimization of the PPO reinforcement learning model. This technical method can achieve high-precision satellite alignment control in a dynamic environment through real-time policy optimization in the continuous action space.

[0072] In one embodiment, data fusion processing is performed based on real-time antenna angle and signal strength data acquired by the sensor array, combined with theoretical position data calculated by the satellite orbit prediction module, to generate a dynamic state vector, which may include:

[0073] The system acquires the real-time antenna azimuth and elevation angles collected by the sensor array, and obtains the theoretical azimuth and elevation angles calculated by the satellite orbit prediction module.

[0074] Specifically, the angle sensors in the sensor array measure the antenna's current azimuth and elevation angles in real time. These sensors can be high-precision encoders or inertial measurement units. Simultaneously, the satellite orbit prediction module calculates the theoretical azimuth and elevation angles using an orbital mechanics model based on the satellite's orbital parameters and time information. For example, the orbit prediction module can employ an improved Kepler orbital model, combined with satellite ephemeris data and factors such as Earth's rotation, to accurately calculate the satellite's theoretical position at any given time, thereby obtaining the theoretical azimuth and elevation angles.

[0075] The deviation between the real-time antenna azimuth and the theoretical azimuth is calculated and processed to generate azimuth deviation data. Similarly, the deviation between the real-time antenna elevation and the theoretical elevation is calculated and processed to generate elevation deviation data.

[0076] Specifically, the deviation between the real-time antenna azimuth and the theoretical azimuth is calculated to generate azimuth deviation data; simultaneously, the real-time elevation angle and the theoretical elevation angle are processed in the same way to generate elevation deviation data. The processing can directly quantify the deviation between the actual pointing and the ideal target using arithmetic subtraction, for example, by characterizing the tracking error through the absolute value of the angle difference. This step significantly improves the intuitiveness of the state representation and the control sensitivity by transforming abstract spatial position differences into calculable numerical indicators. Furthermore, a nonlinear coupling model can be used to handle the relationship between the real-time antenna pointing angle and the theoretical position: a spatiotemporal variation curve of the theoretical pointing is constructed based on the satellite orbital dynamics equations, and then the tangential deviation of the measured angle relative to the theoretical curve is calculated using the Lie derivative, generating a dynamic error field for the azimuth and elevation angles. The processing integrates kinematic constraints and signal attenuation characteristics; for example, the signal strength gradient is introduced as a weighting factor to automatically enhance the sensitivity of deviation calculation in weak signal regions. The data for this step comes from the continuous trajectory function of orbit prediction and real-time sensor sampling values. The effect is to transform the static angle difference into a dynamic error vector with direction sensitivity, which significantly improves the environmental adaptability and anti-interference capability of satellite control.

[0077] Disturbance features are extracted from the wind speed data collected by the sensor array to generate environmental disturbance parameters.

[0078] Specifically, the wind speed sensors in the sensor array collect environmental wind speed data in real time. This data reflects the environmental disturbances to the antenna. Through signal processing algorithms, such as wavelet transform or Fourier transform, feature parameters from the wind speed data, such as the average value, variance, and frequency components of the wind speed, are extracted to generate environmental disturbance parameters. Furthermore, deep learning algorithms, such as LSTM networks, can be used to analyze the wind speed time series and extract more complex disturbance features. For example, an LSTM network can be trained on wind speed data from a past period to predict future wind speed trends and incorporate this prediction into the environmental disturbance parameters.

[0079] A multi-source fusion processing is performed on azimuth deviation data, pitch deviation data, signal strength data, and environmental disturbance parameters to generate a dynamic state vector.

[0080] Specifically, multi-source data can be integrated using methods such as weighted fusion, Kalman filter fusion, or neural network fusion. For example, the Kalman filter fusion method dynamically adjusts the weights of each data point based on their reliability and correlation to obtain an optimal fusion result. Furthermore, a multilayer perceptron (MLP) neural network can be used, taking each data point as input features and performing nonlinear transformations to generate a comprehensive dynamic state vector. This dynamic state vector can comprehensively reflect the real-time relative state between the antenna and the satellite, as well as the environmental impact on the antenna, providing more accurate information for subsequent satellite control.

[0081] In one embodiment, based on a dynamic state vector, a policy reasoning process is performed through an Actor network to generate continuous actions, and a security assessment process is performed on the continuous actions through a Critic network to obtain a security decision result, including:

[0082] The dynamic state vector is input into the fully connected layer of the Actor network for feature abstraction to obtain a higher-order state representation.

[0083] Specifically, the dynamic state vector can be input into the fully connected layer of the Actor network for feature abstraction. For example, through a two-layer 256-dimensional neural network nonlinear transformation, the correlation between satellite motion patterns and environmental disturbances implicit in the state can be extracted. By combining weight matrices and activation functions, the effect is to generate high-order state features with strong representational capabilities, significantly improving the physical perception depth of policy reasoning. For instance, the fully connected layer can use the ReLU activation function to enhance nonlinear expressive power; feature abstraction can be combined with residual connections to avoid gradient vanishing.

[0084] Based on higher-order state representation, action probability distribution is modeled through a Gaussian distribution sampling layer to generate a continuous action sampling space.

[0085] For example, action probability distribution modeling can be performed using a Gaussian sampling layer based on higher-order state representations, such as constructing the probability distributions of azimuth and pitch angle adjustments using a mean-variance parameterization method. The processing may include calculating the action mean and adaptive variance to generate a continuous action sampling space that balances exploration and exploitation. This step, through a probabilistic sampling mechanism, achieves a balance between precise control and exploration of the unknown environment in action generation. For instance, the Gaussian sampling layer can integrate reparameterization techniques; the adaptive variance can decay as training progresses to improve convergence stability.

[0086] Variance constraints are applied to the continuous motion sampling space, and the azimuth and pitch adjustments are output as continuous motions.

[0087] Specifically, variance constraint processing aims to limit the fluctuation range of actions and prevent excessive actions from causing system instability. For example, upper limits can be set for the variance of azimuth and elevation adjustments. Optimization algorithms such as the Lagrange multiplier method are used to constrain the variance while ensuring the rationality of the actions. After processing, the optimal azimuth and elevation adjustments that meet the variance constraints are selected from the continuous action sampling space as the final continuous actions. These actions guide the precise adjustment of the antenna, ensuring that the antenna rapidly approaches the target satellite position within a safe range.

[0088] The dynamic state vector is input into the value assessment layer of the Critic network to predict the state value and obtain the expected return estimate.

[0089] Specifically, the value assessment layer typically contains several neurons that comprehensively evaluate the input dynamic state vector to calculate the value of the current state. For example, methods such as linear regression or nonlinear fitting can be used to learn the mapping relationship between states and values ​​based on historical data and a pre-set reward mechanism. The expected return valuation reflects the expected long-term returns that may be obtained by performing different actions in the current state, providing a crucial basis for subsequent risk assessment.

[0090] Based on continuous actions, the trajectory deviation is calculated and processed through the action impact analysis layer to generate a risk coefficient.

[0091] Specifically, the action impact analysis layer simulates the antenna's trajectory after continuous adjustments and compares it with the ideal trajectory. For example, a geometric or dynamic model can be used to calculate the degree of deviation between the actual trajectory and the ideal trajectory, i.e., the trajectory deviation. Based on the trajectory deviation and a preset risk threshold, a risk coefficient is generated to measure the potential safety risk of the action. If the trajectory deviation is too large, the risk coefficient increases accordingly, indicating that the action carries a high safety risk.

[0092] The expected return valuation and risk coefficient are weighted and integrated to output a safe decision result that includes the priority of action execution.

[0093] Specifically, the expected return valuation and risk coefficient are weighted and fused to output a safety decision result that includes the priority of action execution. The weighted fusion process assigns different weights to the expected return valuation and risk coefficient based on their importance. For example, the weight of the expected return valuation can be set to 0.7, and the weight of the risk coefficient to 0.3. A comprehensive score for each consecutive action is calculated through weighted summation. Based on the comprehensive score, the execution priority of the actions is determined, and a safety decision result is output, prioritizing actions with high returns and low risks to ensure the safety and effectiveness of antenna adjustment.

[0094] In one embodiment, based on the acquired state change data after antenna rotation, a dynamic reward calculation is performed to obtain a reward value, and the reward value and state change data are stored in an experience pool, including:

[0095] The base reward value is obtained by performing gain reward calculation on the state change data.

[0096] The satellite drift direction vector is obtained by extracting the direction from the satellite velocity vector output by the satellite orbit prediction module.

[0097] The azimuth and elevation adjustments after antenna rotation are vectorized to generate the actual antenna rotation direction vector.

[0098] The cosine similarity between the satellite drift direction vector and the antenna's actual rotation direction vector is calculated to generate a trajectory matching reward.

[0099] An efficiency penalty value is obtained by penalizing the antenna rotation time using a preset time decay factor.

[0100] The reward value is generated by combining the base reward value, trajectory matching reward value, and efficiency penalty value.

[0101] The reward value, state change data, and corresponding dynamic state vector are associated and stored to generate experience data units.

[0102] Priority assignment is performed on the experience data units to obtain priority-bearing experience data, and the priority-bearing experience data is stored in the experience pool.

[0103] Specifically, this embodiment calculates a gain reward based on the state change data after antenna rotation, for example, generating a basic reward value based on the signal strength improvement. Simultaneously, it extracts the direction based on the satellite velocity vector output by the satellite orbit prediction module, for example, obtaining the real-time satellite drift direction vector through vector decomposition. It then synthesizes the azimuth and elevation angle adjustments after execution to generate the actual antenna rotation direction vector. By calculating the cosine similarity between the satellite drift direction vector and the actual antenna rotation direction vector, it generates a trajectory matching reward representing motion consistency, for example, assigning a positive reward when both are in the same direction. A preset time decay factor is introduced to penalize the rotation time, for example, increasing the penalty value linearly with the adjustment time. The basic reward value, trajectory matching reward, and efficiency penalty value are fused, for example, using a weighted summation formula to generate a comprehensive reward value. The reward value, state change data, and corresponding dynamic state vector are associated and stored to generate an experience data unit with priority assignment, for example, setting sampling weights based on the reward value fluctuation amplitude, and storing it in the experience pool through a tree structure index.

[0104] This embodiment breaks through the limitations of traditional reward mechanisms through the innovative design of motion consistency reward and priority storage: trajectory matching reward introduces satellite dynamic characteristics into the optimization target, so that the action decision actively conforms to the orbit change law; time decay penalty enhances convergence efficiency; priority storage focuses on high-value experience, and the three work together to drive the model to achieve faster convergence and higher precision intelligent control in complex dynamic environments.

[0105] In one embodiment, online optimization of the PPO reinforcement learning model includes:

[0106] Prioritize and sort the experience data stored in the experience pool to generate a high-value sample set.

[0107] Based on a high-value sample set, the policy gradient is calculated through importance sampling to obtain the policy update gradient.

[0108] A pruning objective function is used to constrain the gradient of the policy update, and the parameters of the Actor network and the Critic network are updated online.

[0109] Specifically, this embodiment prioritizes the experience data stored in the experience pool to generate a high-value sample set. The experience data originates from information such as state changes and reward values ​​during antenna adjustment, and is sorted according to its priority. For example, priority can be determined based on the magnitude of the reward value and the significance of the state change; data with higher reward values ​​and more significant state changes have higher priority. Through selection sorting algorithms, high-priority data is selected to form a high-value sample set, ensuring that subsequent optimization processes focus on key data. Based on the high-value sample set, policy gradient calculation is performed through importance sampling to obtain the policy update gradient. Importance sampling is used to measure the contribution of different samples to policy updates. For example, the Advantages value can be used to represent the potential contribution of each sample; samples with larger Advantages values ​​have greater weight in policy gradient calculation. Specifically, the Advantages value of each sample is first calculated, and then the samples are weighted according to this value. The policy update gradient is calculated using optimization algorithms such as gradient descent, reflecting the direction and degree of policy optimization. A pruning objective function is used to constrain the magnitude of the policy update gradient, and the parameters of the Actor network and the Critic network are updated online. The pruning objective function is used to limit the policy update gradient from being too large or too small, preventing instability during model training. For example, a pruning range can be set; if the policy update gradient exceeds this range, it is pruned to within that range. By updating the parameters of the Actor network with gradients, the policy generation process is adjusted; simultaneously, the Critic network is updated to improve its accuracy in evaluating state value, achieving overall model optimization and continuously improving the antenna adjustment strategy to adapt to dynamically changing environments. This embodiment overcomes the efficiency bottleneck of traditional experience replay through a collaborative optimization mechanism of priority-driven and gradient pruning: high-value samples accelerate key policy learning, importance sampling ensures the statistical rationality of data utilization, and gradient pruning maintains training stability at the algorithm level. The three work together to achieve a balance between efficiency and robustness in online optimization, providing core support for continuous performance improvement in dynamic environments.

[0110] In one embodiment, the antenna controller, sensor group, and satellite orbit prediction module are initialized, a pre-trained PPO reinforcement learning model is loaded, and the state space and continuous action space are defined, including:

[0111] Based on the obtained satellite antenna control requirements, the dimensions of the state space are determined.

[0112] Based on the constraints of the antenna's mechanical structure, the boundary range of the continuous operation space is determined.

[0113] The continuous action space is initialized with a Gaussian distribution to generate initial values ​​for the mean and variance of the action probability distribution.

[0114] Specifically, this embodiment determines the dimensions of the state space based on satellite antenna control requirements, such as defining a four-dimensional state space by comprehensively considering key parameters like azimuth, elevation, signal strength, and gradient changes. It further determines the boundary range of the continuous action space based on antenna mechanical structure constraints, for example, limiting azimuth adjustment to ±5 degrees and elevation to ±3 degrees through structural analysis to avoid overload risks. Specifically, the continuous action space is initialized using a Gaussian distribution, for example, setting the action mean to zero and the variance to 0.5 times the boundary value, generating initial parameters for the action probability distribution. This step employs a dual-constraint design: the state dimension definition precisely matches control requirements, the action boundary setting conforms to mechanical limits, and Gaussian initialization balances exploration and stability. For example, the state dimension can be derived through satellite communication protocols; the action boundary can be calculated using the motor torque curve; and the Gaussian parameter initialization can employ heuristic search optimization to provide a physically interpretable initial policy framework for reinforcement learning, avoiding mechanical overstepping or ineffective exploration in the early stages of training.

[0115] This embodiment lays a solid foundation for the training and application of the PPO reinforcement learning model through precise definition of the state space and continuous action space and reasonable initialization processing. It not only ensures the effectiveness and safety of the model during the training process, but also improves the model's adaptability to satellite antenna control requirements and satellite alignment accuracy. At the same time, it accelerates the training convergence speed of the model and improves the intelligence level and reliability of the entire satellite antenna automatic alignment system.

[0116] In summary, the automatic satellite antenna alignment method based on PPO reinforcement learning provided in this application establishes a control framework during the initialization phase. It defines a multi-dimensional state space (such as azimuth, elevation, and signal strength) based on satellite communication requirements, sets the physical boundaries of the continuous action space in conjunction with antenna mechanical constraints, and initializes action probability parameters using a Gaussian distribution. In the dynamic control loop, it fuses real-time sensor data with theoretical orbit prediction values ​​to generate a dynamic state vector. A strategy reasoning and safety assessment are achieved through an Actor-Critic dual-network collaborative mechanism: the Actor network abstracts high-order features based on a fully connected layer and outputs continuously adjusted actions after Gaussian sampling; the Critic network combines value prediction and trajectory deviation analysis to generate a risk-reward balanced safety decision. Based on the state changes after action execution, the reward value is calculated through dynamic fusion of gain reward, trajectory matching reward, and efficiency penalty. Priority-marked experience data is stored in an experience pool, driving the PPO model to achieve online optimization through importance sampling and gradient pruning. This technical solution overcomes the inherent limitations of traditional control methods by using real-time strategy iteration and dynamic environment adaptation mechanisms in continuous action space: it achieves high-precision pointing control in both azimuth and pitch dimensions under complex scenarios such as satellite orbit perturbation and wind interference, significantly improving convergence speed, environmental adaptability and system stability, and providing satellite communication systems with autonomously optimized, safe and reliable core control capabilities.

[0117] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0118] Based on the same inventive concept, this application also provides a PPO reinforcement learning-based automatic satellite antenna alignment device 10 for implementing the aforementioned PPO reinforcement learning-based automatic satellite antenna alignment method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the PPO reinforcement learning-based automatic satellite antenna alignment device 10 provided below can be found in the above-described limitations of the PPO reinforcement learning-based automatic satellite antenna alignment method, and will not be repeated here.

[0119] In one exemplary embodiment, such as Figure 2 As shown, a satellite antenna automatic alignment device 10 based on PPO reinforcement learning is provided, comprising:

[0120] Initialization module 11 is used to initialize the antenna controller, sensor group and satellite orbit prediction module, load the pre-trained PPO reinforcement learning model, and define the state space and continuous action space.

[0121] The data fusion module 12 is used to perform data fusion processing based on the real-time antenna angle and signal strength data obtained by the sensor group and the theoretical position data calculated by the satellite orbit prediction module to generate a dynamic state vector.

[0122] The dual-support neural network module 13 is used to generate continuous actions by performing policy reasoning processing through the Actor network based on dynamic state vectors, and to perform safety assessment processing on the continuous actions through the Critic network to obtain safety decision results. The safety decision results are used to instruct the target antenna to adjust its angle.

[0123] The reinforcement learning module 14 is used to dynamically calculate the reward value based on the acquired state change data after the antenna rotates, and store the reward value and state change data in the experience pool. The experience pool is used to optimize the PPO reinforcement learning model online.

[0124] In one embodiment, the data fusion module 12 may include:

[0125] The azimuth and elevation acquisition unit is used to acquire the real-time antenna azimuth and elevation angles collected by the sensor group, and to acquire the theoretical azimuth and elevation angles calculated by the satellite orbit prediction module.

[0126] The dynamic deviation calculation unit is used to calculate the deviation between the real-time antenna azimuth and the theoretical azimuth, generating azimuth deviation data, and to calculate the deviation between the real-time antenna elevation and the theoretical elevation, generating elevation deviation data.

[0127] The disturbance feature extraction unit is used to extract disturbance features from the wind speed data collected by the sensor group and generate environmental disturbance parameters.

[0128] The multi-source fusion unit is used to perform multi-source fusion processing on azimuth deviation data, pitch deviation data, signal strength data and environmental disturbance parameters to generate a dynamic state vector.

[0129] In one embodiment, the dual-scaffold neural network module 13 may include:

[0130] The feature abstraction unit is used to input the dynamic state vector into the fully connected layer of the Actor network for feature abstraction, thereby obtaining a higher-order state representation.

[0131] The probability modeling unit is used to model the probability distribution of actions based on higher-order state representations and through a Gaussian distribution sampling layer, thereby generating a continuous action sampling space.

[0132] The variance constraint unit is used to perform variance constraint processing on the continuous motion sampling space and output the azimuth and pitch adjustment amounts as continuous motion.

[0133] The value prediction unit is used to input the dynamic state vector into the value evaluation layer of the Critic network to predict the state value and obtain the expected return estimate.

[0134] The risk analysis unit is used to calculate and process trajectory deviation based on continuous actions through the action impact analysis layer, and generate risk coefficients.

[0135] The safety decision unit is used to perform weighted fusion processing on the expected return valuation and risk coefficient, and output a safety decision result that includes the action execution priority.

[0136] In one embodiment, reinforcement learning module 14 includes:

[0137] The basic reward calculation unit is used to calculate the basic reward value by performing gain reward calculation on the state change data.

[0138] The drift direction extraction unit is used to extract the satellite drift direction vector based on the satellite velocity vector output by the satellite orbit prediction module.

[0139] The rotation direction synthesis unit is used to synthesize the azimuth and elevation angle adjustments after the antenna rotation into a vector to generate the actual rotation direction vector of the antenna.

[0140] The trajectory matching unit is used to calculate the cosine similarity between the satellite drift direction vector and the antenna's actual rotation direction vector, and generate a trajectory matching reward.

[0141] The efficiency penalty unit is used to calculate and process the antenna rotation time by using a preset time decay factor to obtain an efficiency penalty value.

[0142] The reward fusion unit is used to merge the basic reward value, trajectory matching reward, and efficiency penalty value to generate a reward value.

[0143] The data association unit is used to associate and store reward values, state change data, and corresponding dynamic state vectors to generate experience data units.

[0144] The priority management unit is used to assign priority values ​​to the experience data units, obtain priority-bearing experience data, and store the priority-bearing experience data in the experience pool.

[0145] In one embodiment, the reinforcement learning module 14 may further include:

[0146] The sample screening unit is used to prioritize and sort the experience data stored in the experience pool to generate a high-value sample set.

[0147] The gradient calculation unit is used to calculate the policy gradient based on the high-value sample set through importance sampling, and obtain the policy update gradient.

[0148] The parameter update unit is used to perform magnitude constraint processing on the policy update gradient using the pruning objective function, and to update the parameters of the Actor network and the Critic network online.

[0149] In one embodiment, the initialization module 11 includes:

[0150] The state space definition unit is used to determine the dimensions of the state space based on the acquired satellite antenna control requirements.

[0151] The action boundary setting unit is used to determine the boundary range of the continuous action space based on the constraints of the antenna's mechanical structure.

[0152] The probability initialization unit is used to initialize the continuous action space with a Gaussian distribution, generating initial values ​​for the mean and variance of the action probability distribution.

[0153] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the aforementioned PPO reinforcement learning-based automatic satellite antenna alignment method.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0155] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0156] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for automatic satellite antenna alignment based on PPO reinforcement learning, characterized in that, The method includes: Initialize the antenna controller, sensor group and satellite orbit prediction module, load the pre-trained PPO reinforcement learning model, and define the state space and continuous action space; Based on the real-time antenna angle and signal strength data acquired by the sensor group, combined with the theoretical position data calculated by the satellite orbit prediction module, data fusion processing is performed to generate a dynamic state vector. Based on the dynamic state vector, a policy reasoning process is performed through an Actor network to generate continuous actions, and a security assessment process is performed through a Critic network to obtain a security decision result. The security decision result is used to instruct the target antenna to adjust its angle. Based on the acquired state change data after the antenna rotation, a dynamic reward is calculated to obtain a reward value. The reward value and the state change data are stored in an experience pool, which is used to optimize the PPO reinforcement learning model online.

2. The method according to claim 1, characterized in that, The real-time antenna angle and signal strength data acquired based on the sensor group, combined with the theoretical position data calculated by the satellite orbit prediction module, undergo data fusion processing to generate a dynamic state vector, including: The system acquires the real-time antenna azimuth and real-time antenna elevation angles collected by the sensor group, and also acquires the theoretical azimuth and theoretical elevation angles calculated by the satellite orbit prediction module. The deviation between the real-time antenna azimuth angle and the theoretical azimuth angle is calculated to generate azimuth angle deviation data, and the deviation between the real-time antenna elevation angle and the theoretical elevation angle is calculated to generate elevation angle deviation data. Disturbance features are extracted from the wind speed data collected by the sensor group to generate environmental disturbance parameters; The azimuth deviation data, elevation deviation data, signal strength data, and environmental disturbance parameters are fused from multiple sources to generate the dynamic state vector.

3. The method according to claim 2, characterized in that, Based on the dynamic state vector, policy reasoning is performed through an Actor network to generate continuous actions, and a security assessment is performed on the continuous actions through a Critic network to obtain a security decision result, including: The dynamic state vector is input into the fully connected layer of the Actor network for feature abstraction to obtain a higher-order state representation. Based on the higher-order state representation, the action probability distribution is modeled through a Gaussian distribution sampling layer to generate a continuous action sampling space. The continuous action sampling space is subjected to variance constraint processing, and the azimuth adjustment and pitch adjustment are output as the continuous action. The dynamic state vector is input into the value evaluation layer of the Critic network to predict the state value and obtain the expected return estimate. Based on the continuous actions, the trajectory deviation is calculated and processed through the action impact analysis layer to generate a risk coefficient. The expected return valuation and the risk coefficient are weighted and fused to output a safety decision result that includes the action execution priority.

4. The method according to claim 3, characterized in that, The step of dynamically calculating a reward value based on the acquired antenna rotation state change data, and storing the reward value and the state change data in an experience pool, includes: The base reward value is obtained by performing gain reward calculation on the state change data; Based on the satellite velocity vector output by the satellite orbit prediction module, the satellite drift direction vector is obtained by direction extraction; The azimuth adjustment and elevation adjustment after antenna rotation are vector-synthesized to generate the actual antenna rotation direction vector. The cosine similarity between the satellite drift direction vector and the antenna actual rotation direction vector is calculated to generate a trajectory matching reward. The efficiency penalty value is obtained by penalizing the antenna rotation time using a preset time decay factor. The reward value is generated by combining the base reward value, the trajectory matching reward, and the efficiency penalty value. The reward value, the state change data, and the corresponding dynamic state vector are associated and stored to generate an experience data unit. The experience data units are given priority values ​​to obtain priority-bearing experience data, and the priority-bearing experience data is stored in the experience pool.

5. The method according to claim 4, characterized in that, The online optimization of the PPO reinforcement learning model includes: The experience data stored in the experience pool is prioritized and sorted to generate a high-value sample set; Based on the high-value sample set, the policy gradient is calculated through importance sampling to obtain the policy update gradient; The gradient update of the strategy is constrained by a pruning objective function, and the parameters of the Actor network and the Critic network are updated online.

6. The method according to claim 1, characterized in that, The initialization of the antenna controller, sensor group, and satellite orbit prediction module, loading the pre-trained PPO reinforcement learning model, and defining the state space and continuous action space include: Based on the acquired satellite antenna control requirements, the dimensions of the state space are determined; The boundary range of the continuous action space is determined based on the constraints of the antenna's mechanical structure. The continuous action space is initialized with a Gaussian distribution to generate initial values ​​for the mean and variance of the action probability distribution.

7. An automatic satellite antenna alignment device based on PPO reinforcement learning, characterized in that, The device includes: The initialization module is used to initialize the antenna controller, sensor group and satellite orbit prediction module, load the pre-trained PPO reinforcement learning model, and define the state space and continuous action space. The data fusion module is used to perform data fusion processing based on the real-time antenna angle and signal strength data acquired by the sensor group and the theoretical position data calculated by the satellite orbit prediction module to generate a dynamic state vector. The dual-support neural network module is used to perform policy reasoning processing through the Actor network based on the dynamic state vector, generate continuous actions, and perform security assessment processing through the Critic network to obtain a security decision result. The security decision result is used to instruct the target antenna to adjust its angle. The reinforcement learning module is used to dynamically calculate the reward value based on the acquired state change data after the antenna rotates, and store the reward value and the state change data in the experience pool. The experience pool is used to optimize the PPO reinforcement learning model online.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images