Satellite-ground integrated operation and control method for coaxial optical radar payload

Abstract

The present application relates to the technical field of directional control, and provides a satellite-ground integrated operation and control method for a coaxial optical radar load, which first collects state parameters between a satellite and a ground station through a coaxial optical radar, and performs noise reduction processing on the data to provide accurate input for subsequent state estimation; then, the current state of the satellite is predicted using Kalman filtering to generate state estimation values at the next time; according to the estimation values, control instructions are generated and decomposed, the control instructions in each dimension are processed step by step, and the control instructions are checked; subsequently, the state estimation and control instruction execution data are input into a deep reinforcement learning model to optimize the control instructions; to improve the accuracy of Kalman filtering, a space-time correlation factor is introduced to adjust the state transition matrix to more accurately reflect the dynamic relationship between the satellite and the ground station; in addition, the deep reinforcement learning model is trained through historical data and execution data to further optimize the control instructions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of directional control, more particularly, the present application relates to a coaxial optical radar payload satellite satellite-ground integrated operation and control method. BACKGROUND

[0002] Coaxial optical radar payload satellites are an advanced space measurement and control technology that integrates ranging and tracking functions on a single optical axis through coaxial optical radar, enabling high-precision, real-time space positioning and trajectory control. The satellite-ground integrated operation and control system of coaxial optical radar payload satellites combines ground control and satellite autonomous control, aiming to achieve accurate satellite positioning and efficient attitude adjustment, and can perform complex tasks such as one station multi-satellite operations under the integrated control framework. Through satellite-ground integration, the complexity of ground control can be simplified, while the reliability and real-time performance of the system can be improved.

[0003] Coaxial optical radar integrated systems have significant advantages in satellite-ground integrated control. According to document 1 (Hoge, F. (1974). Integrated Laser / Radar Satellite Ranging and Tracking System. Applied Optics , 13(12), 2356-2362.), a laser and radar integrated system can significantly improve the tracking and ranging accuracy of a satellite through multifunction multiplexing, while reducing system cost and improving control reliability. This feature is of great significance to the efficient satellite-ground integrated control of satellites, enabling ground stations to operate flexibly in various mission modes, effectively expanding the application range of the system, and showing advantages in high-precision measurement and rapid response.

[0004] Currently, satellite-ground integrated control technology has made significant progress in satellite orbit maintenance, attitude adjustment, and data transmission. Traditional satellite-ground integrated control relies on the telemetry, tracking, and control (TT&C) system, which measures, calculates, and feeds back instructions through ground stations to control the orbit and attitude of the satellite. This type of system usually shares measurement and instruction data among multiple ground stations to ensure the continuity and accuracy of control. In addition, existing ground control technologies include advanced methods such as software-defined networks (SDN) and automated task scheduling, but these methods still have certain bottlenecks in hardware resource integration, response speed, and data processing, making it difficult to fully meet the needs of multi-satellite cooperative operations.

[0005] In recent years, cloud platform-based satellite-ground integrated control architecture has gradually attracted attention. Literature 2 (Xia, Z., Li, W., & Zhang, H. (2023). Research on Satellite Operation Control System Based on Cloud Platform. Journal of Space Science&Technology , 24(5), 1234-1246.) proposed a satellite operation control system architecture based on cloud computing, which integrates the hardware and software resources of ground control systems to achieve "one station multiple satellites" parallel control and automated operation, enhancing the flexibility and real-time performance of the system. This architecture can achieve dynamic allocation of resources through cloud computing, effectively improving the scalability and automation level of the system, especially suitable for task management and coordinated operation of multiple satellites.

[0006] Although the cloud platform-based satellite-ground integrated control system has significantly improved in automation and flexibility, there are still some deficiencies in this technology. For example, the cloud platform architecture may affect control accuracy due to delays when facing real-time and high-frequency data transmission. In addition, although the multi-task processing capability of cloud computing can improve the overall response efficiency of the system, it is difficult to fully meet the requirements of high-speed computing and high-precision control when dealing with tasks involving high-precision orbit and attitude adjustment.

[0007] To solve the above problems, a technical solution is provided. SUMMARY

[0008] To overcome the above-mentioned defects of the prior art, the present application provides a coaxial optical radar payload satellite satellite-ground integrated operation and control method, which combines Kalman filtering and deep reinforcement learning model to optimize satellite orbit and attitude control; uses a space-time correlation factor to improve the state transition matrix, improving the relative state estimation accuracy between the satellite and the ground station, solving the problems of poor real-time performance and high-precision control in existing satellite satellite-ground integrated control.

[0009] To achieve the above-mentioned purposes, the present application provides the following technical solutions:

[0010] A coaxial optical radar payload satellite satellite-ground integrated operation and control method, comprising the following steps:

[0011] Step S1, collect the position parameters, attitude parameters, relative velocity parameters and acceleration parameters of the coaxial optical radar payload satellite, and perform noise reduction processing, outputting the noise-reduced data as the input for subsequent state estimation;

[0012] Step S2, use Kalman filtering to analyze the noise-reduced data, predict the satellite state at the current time according to the state of the satellite at the previous time, and output the state estimation value;

[0013] Step S3: Generate control instructions based on the state estimate, decompose the control instructions, and generate control instructions for each dimension sequentially based on the decomposed state data.

[0014] Step S4: Verify the control instructions generated for each dimension;

[0015] Step S5: Input the state estimate and the control command execution data for each dimension into the state space of the trained deep reinforcement learning model to optimize the control commands. The reward function of the deep reinforcement learning model is:

[0016] ;

[0017] In the formula, As a reward value, This represents the total number of state parameters. For the first The values ​​of the target state parameters For the first The values ​​of the actual execution status parameters, To control the total number of commands, Let j be the adjustment amount for the j-th control command. For the first The execution frequency of each control command and These are the weighting coefficients.

[0018] As a further aspect of this invention, considering the spatiotemporal variation patterns of the relative states of the satellite and the ground station, a spatiotemporal correlation factor is introduced into the state transition process of the Kalman filter to improve the update rule of the state transition matrix, including the following steps:

[0019] Step A1: Extract historical state data of relative position parameters, attitude parameters, relative velocity parameters, and acceleration parameters between the satellite and the ground station;

[0020] Step A2: Perform differential analysis on the historical state data using time series analysis methods to calculate the rate of change of state between adjacent time points, the formula of which is:

[0021] Step A3: Based on the rate of change of historical states, construct a linear regression model using regression analysis, and fit the historical data using the least squares method to solve for the influence coefficient of the rate of change of historical states and the influence coefficient of the time interval.

[0022] Step A4: Define a spatiotemporal correlation factor to reflect the spatiotemporal relationship of relative motion between the satellite and the ground station. The formula for calculating the spatiotemporal correlation factor is as follows:

[0023] ;

[0024] wherein, is a space-time correlation factor, is a state change rate at time , is a state change rate at time , is an influence coefficient of time interval, is a time difference from time to time ;

[0025] Step A5, in each time step, the state transition matrix is adjusted according to the currently calculated space-time correlation factor, and the adjustment formula is:

[0026] ;

[0027] wherein, is a state transition matrix, is a traditional state transition matrix, is a state transition matrix at the previous time, is a space-time correlation factor.

[0028] As a further scheme of the present application, in step S5, the state estimation parameters after decomposition of the state estimation value and the actual state parameters after execution of the control instructions of each dimension are combined into complete state information, and the state information is input as input data into the state space of the trained deep reinforcement learning model, to generate optimized position control instructions, attitude control instructions, speed control instructions and acceleration control instructions; the state estimation parameters include estimated position parameters, estimated attitude parameters, estimated speed parameters and estimated acceleration parameters, and the actual state parameters include actual position parameters, actual attitude parameters, actual speed parameters and actual acceleration parameters; the position control instructions include position adjustment amount, adjustment direction and execution time information; the attitude control instructions include angle adjustment amount, rotation axis and execution time information; the speed control instructions include speed adjustment amount, adjustment direction and execution time information; and the acceleration control instructions include acceleration adjustment amount, adjustment direction and execution time information.

[0029] As a further scheme of the present application, training the deep reinforcement learning model comprises the following steps:

[0030] Step B1, constructing a state space of the deep reinforcement learning model, inputting the state estimation parameters after decomposition of the state estimation value and the actual state parameters after execution of the control instructions of each dimension into the state space of the deep reinforcement learning model;

[0031] Step B2, define the action space of the deep reinforcement learning model, refine the control instruction into specific actions of pose adjustment, track position adjustment, speed adjustment and acceleration adjustment as elements of the action space;

[0032] Step B3, design a reward mechanism, set a reward function according to the control error and energy saving index, take reducing the control error as the main target, take saving energy as the secondary target, and balance the relationship between precision and energy consumption;

[0033] Step B4, train the deep reinforcement learning model by using historical data and control instruction execution data.

[0034] As a further scheme of the application, the position parameter is the relative position between the satellite and the ground station, which is obtained by real-time collection of the relative position of the satellite and the ground station in the geographic coordinate system using the coaxial optical radar, and is expressed using Cartesian coordinates; the attitude parameter is the attitude angle of the satellite, including the pitch angle, roll angle and yaw angle of the satellite, and is expressed using Euler angles; the relative velocity parameter is the relative velocity between the satellite and the ground station, including the relative horizontal velocity, relative vertical velocity and relative normal velocity; and the acceleration parameter is the acceleration of the satellite on its orbit, including the horizontal acceleration component, vertical acceleration component and normal acceleration component.

[0035] As a further scheme of the application, step S3 includes the following steps:

[0036] Step S3.1, decompose the state estimation value output by the Kalman filter into estimated position parameters, estimated attitude parameters, estimated relative velocity parameters and estimated acceleration parameters;

[0037] Step S3.2, compare the current estimated position parameters with the target position parameters, calculate the required displacement adjustment amount, decompose the total displacement adjustment amount according to the set fixed increment, calculate the required adjustment steps and the amount of each adjustment, generate a pose, and the position control instruction includes the displacement adjustment amount, adjustment direction and execution time information;

[0038] Step S3.3, compare the current estimated attitude angle with the target attitude angle, calculate the required attitude angle adjustment amount, decompose the total attitude angle adjustment amount according to the set fixed increment, calculate the required adjustment steps and the angle increment of each adjustment, generate a pose control instruction, and the control instruction includes the angle amount, rotation axis and execution time information;

[0039] Step S3.4, compare the current estimated velocity with the target velocity, calculate the required velocity adjustment amount, decompose the total velocity adjustment amount according to the set fixed increment, calculate the required adjustment steps and the increment of each adjustment, generate a velocity control instruction, and the velocity control instruction includes the velocity adjustment amount, adjustment direction and execution time information;

[0040] Step S3.5, compare the current estimated acceleration with the target acceleration, calculate the required acceleration adjustment amount, and decompose the total acceleration adjustment amount according to the set fixed increment, calculate the required adjustment step number and the increment of each adjustment, and generate an acceleration control instruction, which includes the acceleration adjustment amount, the adjustment direction, and the execution time information;

[0041] Step S3.6, schedule the decomposed control instructions in time sequence, and take the generated segmented control instruction set as the final control instruction.

[0042] As a further scheme of the application, step S4 includes the following steps:

[0043] Step S4.1, after executing the position control instruction, the current position parameters of the satellite are collected again by the coaxial optical radar, and the actual position after execution is compared with the target position to calculate the position deviation. When the position deviation is less than the preset error range, the check passes; when the position deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment.

[0044] Step S4.2, after executing the attitude control instruction, the current attitude parameters of the satellite are collected again by the coaxial optical radar, and the actual attitude after execution is compared with the target attitude to calculate the attitude angle deviation. When the attitude angle deviation is less than the preset error range, the check passes; when the attitude angle deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment.

[0045] Step S4.3, after executing the speed control instruction, the current speed parameters of the satellite are collected again by the coaxial optical radar, and the actual speed after execution is compared with the target speed to calculate the speed deviation. When the speed deviation is less than the preset error range, the check passes; when the speed deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment.

[0046] Step S4.4, after executing the acceleration control instruction, the current acceleration parameters of the satellite are collected again by the coaxial optical radar, and the actual acceleration after execution is compared with the target acceleration to calculate the acceleration deviation. When the acceleration deviation is less than the preset error range, the check passes; when the acceleration deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment.

[0047] Compared with the prior art, the coaxial optical radar payload satellite space-ground integrated operation and control method has the following advantages:

[0048] The application realizes high-precision state estimation and control optimization between satellite and ground station by combining Kalman filter and deep reinforcement learning model. Traditional satellite-ground integrated control system usually relies on the ground telemetry and control (TT&C) system, and has high requirements for real-time and high-precision control, but the existing technology often faces delay when processing high-frequency data, which affects the control accuracy. Compared with the prior art, the application improves the accuracy of satellite control by introducing deep reinforcement learning and space-time correlation factor technology, and solves the problems of real-time response and insufficient accuracy in the prior art by dynamically optimizing control instructions.

[0049] The application improves the energy efficiency of the system by optimizing the control instruction generation process. Although document 2 has great improvement in task management and automatic operation, it still faces the problem of high-precision orbit and attitude adjustment, and the delay of the cloud platform may lead to the decline of control accuracy. Compared with the prior art, the application balances the relationship between control accuracy and energy efficiency by the reward mechanism of the deep reinforcement learning model, reduces the energy consumption, and improves the response speed and efficiency of satellite control. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 It is a building drawing for AN / FPQ-6 C-band radar system and its supporting electronic equipment.

[0051] Figure 2 It is a simplified schematic diagram of a laser receiver.

[0052] Figure 3 It is a block diagram of the electronic circuit of the laser / radar ranging system.

[0053] Figure 4 It is a schematic diagram of the operation control system cloud platform in document 2.

[0054] Figure 5 It is a flowchart of a coaxial optical radar payload satellite-ground integrated operation control method of the application.

[0055] In the figure, ANODE OUTPUT: anode output; HV: high voltage; DAYLIGHT GATE: daylight shutter; PM TUBE: photomultiplier tube; FOCAL PLANE: focal plane; SPATIAL FILTER: spatial filter; TELESCOPE BODY: telescope body; RADAR TV CAMERA: radar TV camera; ADDITIONAL COMPONENTS TO FORM LASER RECEIVER: additional components to form laser receiver; SPECTRAL FILTER: spectral filter; PRIMARY OPTICAL SYSTEM: primary optical system; IN / OUT ACQUISITION AID SYSTEM FOR f / 5: input / output acquisition aid system for f / 5; PRIMARY FOCAL PLANE f / 11 OR f / 5: primary focal plane f / 11 or f / 5; LASER TRANSMIT ORDER (TO LASER): laser transmit order (to laser); RANGE RATE (FROM DRAM): range rate (from DRAM); SITE TIME OF DAY GENERATOR: site time of day generator; COUNTER: counter; STORE REGISTER: store register; RANGE DESIGNATION FROM 4101: range designation from 4101; SHIFT REGISTER: shift register; RANGE COUNTER FWD / BKD BCD: range counter fwd / bkd BCD; A / D CONV.: analog / digital converter; D / A CONV.: digital / analog converter; 2ND. 1ST. INTEGRATOR: 2nd. 1st. integrator; LASER TRANSMIT PULSE: laser transmit pulse; LASER RETURN PULSE: laser return pulse; HP5360A COMPUTING COUNTER: HP5360A computing counter; TRANSFER GATES: transfer gates; GATE REGISTER: gate register; RANGE GATE GENERATOR BKW / FWD BCD COUNTER: range gate generator bkw / fwd BCD counter; TIME INTERVAL DATA TO 4101 COMPUTER: time interval data to 4101 computer; VALIDITY TEST: validity test; DATA VALID: data valid; DECIMAL / BCD SWITCHES: decimal / BCD switches; Planning and scheduling: planning and scheduling;Simulation management; Status monitoring; Telemetry and telecontrol management; Cloud platform management; Operation control application visualization / client; Restful; BlazeDS; WebSocket; XMPP; WebService; Gateway service; Telemetry and telecontrol; Orbit calculation and prediction; Monitoring and alerting; Docker; Task planning and scheduling; Control calculation; Telemetry and telecontrol simulation; Cluster deployment; Resource management; Cluster monitoring; Task scheduling; Load control; Docker+Kubernetes; server; storage; switch; Relational Database; Simulation model Database: Simulation model database. Detailed Implementation

[0056] The technical solutions of this embodiment will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0057] Example 1

[0058] A coaxial optical radar payload satellite space-ground integrated operation and control method, comprising the following steps:

[0059] Step S1, collecting the position parameters, attitude parameters, relative velocity parameters and acceleration parameters of the coaxial optical radar payload satellite, and performing noise reduction processing, and outputting the noise-reduced data as the input of subsequent state estimation;

[0060] Step S2, using Kalman filtering to analyze the noise-reduced data, predicting the satellite state at the current time according to the satellite state at the previous time, and outputting the state estimation value;

[0061] Step S3, generating control instructions based on the state estimation value, decomposing the control instructions, and generating control instructions in each dimension in sequence according to the decomposed state data;

[0062] Step S4, checking the generated control instructions in each dimension;

[0063] Step S5, inputting the state estimation value and the control instruction execution data in each dimension into the state space of the trained deep reinforcement learning model, and optimizing the control instructions.

[0064] The position parameters in the embodiment of the application are the relative positions between the satellite and the ground station, which are obtained by real-time collection of the relative positions of the satellite and the ground station in the geographical coordinate system, and are expressed by Cartesian coordinates; the attitude parameters are the attitude angles of the satellite, including the pitch angle, roll angle and yaw angle of the satellite, and are expressed by Euler angles; the relative velocity parameters are the relative velocities between the satellite and the ground station, including the relative horizontal velocity, relative vertical velocity and relative normal velocity; and the acceleration parameters are the accelerations of the satellite on its orbit, including the horizontal acceleration component, vertical acceleration component and normal acceleration component.

[0065] As Figure 2 The simplified schematic diagram of the laser receiver is shown in the figure, which is formed by using the existing AN / FPQ-6 radar sighting telescope (f / 11) auxiliary focal plane port; the optical system from left to right in the figure includes a main optical system and a spectral filter, followed by optical elements such as a spatial filter for optimizing the spatial selectivity of the received signal; at the rear end of the telescope main body, a radar television camera is used for auxiliary imaging, and a baffle and a photomultiplier tube work together to ensure effective detection of laser signals during the day and at night; high voltage (HV) provides power support for the photomultiplier tube; through these components, the laser receiver can accurately capture and process signals from the laser source for accurate ranging and tracking of remote targets.

[0066] The embodiment of the application introduces a time-space correlation factor in the state transition process of Kalman filtering by combining the time-space variation law of the relative state of the satellite and the ground station, and improves the update rule of the state transition matrix, including the following steps:

[0067] Step A1, extracting the historical state data of the relative position parameters, attitude parameters, relative velocity parameters and acceleration parameters between the satellite and the ground station;

[0068] Step A2, performing differential analysis on the historical state data by a time series analysis method to calculate the state change rate between adjacent time points, and the formula is:

[0069] Step A3, based on the change rate of the historical state, constructing a linear regression model by a regression analysis method, and combining the least square method to fit the historical data to solve the influence coefficient of the historical state change rate and the influence coefficient of the time interval;

[0070] Step A4, defining a time-space correlation factor to reflect the time-space relationship of the relative motion between the satellite and the ground station, and the calculation formula of the time-space correlation factor is:

[0071] ;

[0072] In the formula, is the time-space correlation factor, is the state change rate at time is the state change rate at time is the influence coefficient of the time interval, is the time difference from time to time

[0073] Step A5, in each time step, adjusting the state transition matrix according to the currently calculated time-space correlation factor, and the adjustment formula is:

[0074] ;

[0075] In the formula, is the state transition matrix, is the traditional state transition matrix, is the state transition matrix at the previous time, is the time-space correlation factor.

[0076] The step S2 in the embodiment of the application includes the following steps:

[0077] ​​​Step S2.1, according to the satellite previous time position parameters, attitude parameters, relative speed parameters, acceleration parameters, state transition equation is constructed to carry out state prediction, and the satellite state at the current time is calculated; the state transition equation is:

[0078] ;

[0079] In the formula, is the predicted state vector at the current time, is the state vector at the previous time, is the control input, is the state transition matrix, is the control input matrix;

[0080] Step S2.2, the confidence of satellite state prediction is represented by calculating the prediction error covariance matrix, and the calculation of the prediction error covariance matrix is:

[0081] ;

[0082] In the formula, is the prediction error covariance matrix, is the process noise covariance matrix, is the state transition matrix, is the transpose of the state transition matrix, is the error covariance matrix at the previous time; this formula shows that when the state prediction is carried out, the error covariance matrix will also change with the state transition, and the noise introduced by the process noise covariance matrix will also have an impact on the error covariance matrix considering the uncertainty of system modeling;

[0083] Step S2.3, based on the relative measurement data between the satellite and the ground station, the observation equation is established to combine the measurement data with the predicted data, and the observation equation is:

[0084] ;

[0085] In the formula, is the real-time observation data collected by the coaxial optical radar, is the observation matrix, is the observation noise, is the predicted state vector at the current time;

[0086] Step S2.4, the Kalman gain is calculated by weighing the weights of the prediction error and the observation error, and the estimated value of the satellite state is adjusted, wherein the formula for calculating the Kalman gain is:

[0087] ;

[0088] In the formula, is a Kalman gain matrix, is a prediction error covariance matrix, is a transpose of an observation matrix, is an observation noise covariance, is an observation matrix;

[0089] Step S2.5, according to the calculated Kalman gain, the updated state estimation value is obtained by adjusting the predicted value, and the state update formula is:

[0090] ;

[0091] In the formula, is an updated current time prediction state vector, is a current time prediction state vector, is a Kalman gain matrix, is an observation data vector, is an observation matrix, is a theoretical observation value calculated based on the predicted state

[0092] Step S2.6, after the state is updated, the uncertainty of the system to the state estimation is updated by adjusting the error covariance matrix, and the updated error covariance matrix represents the confidence degree of the system to the satellite state estimation, and the update formula is:

[0093] ;

[0094] In the formula, is an updated error covariance matrix, is an identity matrix, is a Kalman gain matrix, is an observation matrix, is a prediction error covariance matrix.

[0095] The step S3 in the embodiment of the application comprises the following steps:

[0096] Step S3.1, the state estimation value output by the Kalman filter is decomposed into an estimated position parameter, an estimated attitude parameter, an estimated relative velocity parameter and an estimated acceleration parameter;

[0097] Step S3.2, the current estimated position parameter is compared with the target position parameter, the required displacement adjustment amount is calculated, and the total displacement adjustment amount is decomposed according to the set fixed increment, the required adjustment step number and the amount of each adjustment are calculated, the attitude is generated, and the position control instruction includes the position adjustment amount, the adjustment direction and the execution time information;

[0098] Step S3.3, comparing the current estimated attitude angle with the target attitude angle, calculating the required attitude angle adjustment amount, and decomposing the total attitude angle adjustment amount according to the set fixed increment, calculating the required adjustment steps and the angle increment of each adjustment, generating the attitude control instruction, the attitude control instruction including the angle adjustment amount, the rotation axis and the execution time information;

[0099] Step S3.4, comparing the current estimated speed with the target speed, calculating the required speed adjustment amount, and decomposing the total speed adjustment amount according to the set fixed increment, calculating the required adjustment steps and the increment of each adjustment, generating the speed control instruction, the speed control instruction including the speed adjustment amount, the adjustment direction and the execution time information;

[0100] Step S3.5, comparing the current estimated acceleration with the target acceleration, calculating the required acceleration adjustment amount, and decomposing the total acceleration adjustment amount according to the set fixed increment, calculating the required adjustment steps and the increment of each adjustment, generating the acceleration control instruction, the acceleration control instruction including the acceleration adjustment amount, the adjustment direction and the execution time information;

[0101] Step S3.6, scheduling the decomposed control instructions in time sequence, and taking the generated segmented control instruction set as the final control instruction, preparing for subsequent multi-layer review and sending execution.

[0102] The adjustment direction of the position control instruction in the embodiment of the application is determined according to the positive and negative of the displacement adjustment amount, and the execution time is allocated in sequence.

[0103] The rotation axis of the attitude control instruction in the embodiment of the application is respectively the pitch axis, the roll axis and the yaw axis, and the execution time is allocated in sequence.

[0104] The adjustment direction of the speed control instruction in the embodiment of the application is determined according to the positive and negative of the speed adjustment amount, and the execution time is allocated in sequence.

[0105] The adjustment direction of the acceleration control instruction in the embodiment of the application is determined according to the positive and negative of the acceleration adjustment amount, and the execution time is allocated in sequence.

[0106] The step S4 in the embodiment of the application includes the following steps:

[0107] Step S4.1, after executing the position control instruction, the current position parameters of the satellite are collected again by the coaxial optical radar, and the actual position after execution is compared with the target position to calculate the position deviation, when the position deviation is less than the preset error range, the review is passed; when the position deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment;

[0108] Step S4.2, after executing the attitude control instruction, the current attitude parameters of the satellite are collected again by the coaxial optical radar, and the actual attitude after execution is compared with the target attitude, the attitude angle deviation is calculated, when the attitude angle deviation is less than the preset error range, the check passes; when the attitude angle deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment;

[0109] Step S4.3, after executing the speed control instruction, the current speed parameters of the satellite are collected again by the coaxial optical radar, and the actual speed after execution is compared with the target speed, the speed deviation is calculated, when the speed deviation is less than the preset error range, the check passes; when the speed deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment;

[0110] Step S4.4, after executing the acceleration control instruction, the current acceleration parameters of the satellite are collected again by the coaxial optical radar, and the actual acceleration after execution is compared with the target acceleration, the acceleration deviation is calculated, when the acceleration deviation is less than the preset error range, the check passes; when the acceleration deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment.

[0111] In step S5 in the embodiment of the application, the state estimation parameters after decomposition of the state estimation value and the actual state parameters after execution of the control instruction of each dimension are combined into complete state information, and the state information is input as input data into the state space of the trained deep reinforcement learning model to generate optimized position control instruction, attitude control instruction, speed control instruction and acceleration control instruction; the state estimation parameters include estimated position parameters, estimated attitude parameters, estimated speed parameters and estimated acceleration parameters, and the actual state parameters include actual position parameters, actual attitude parameters, actual speed parameters and actual acceleration parameters; the position control instruction includes position adjustment amount, adjustment direction and execution time information; the attitude control instruction includes angle adjustment amount, rotation axis and execution time information; the speed control instruction includes speed adjustment amount, adjustment direction and execution time information; and the acceleration control instruction includes acceleration adjustment amount, adjustment direction and execution time information.

[0112] The deep reinforcement learning model in the embodiment of the application includes the following steps:

[0113] Step B1, a state space of the deep reinforcement learning model is constructed, and the state estimation parameters after decomposition of the state estimation value and the actual state parameters after execution of the control instruction of each dimension are input into the state space of the deep reinforcement learning model;

[0114] Step B2, defining the action space of the deep reinforcement learning model, refining the control instructions into specific actions of attitude adjustment, orbit position adjustment, speed adjustment and acceleration adjustment as elements of the action space;

[0115] Step B3, designing a reward mechanism, setting a reward function according to the control error and energy saving index, taking reducing the control error as the main target and saving energy as the secondary target, balancing the relationship between precision and energy consumption, wherein the reward function of the deep reinforcement learning model is:

[0116] ;

[0117] In the formula, is the reward value, is the total number of state parameters, is the value of the i-th target state parameter, is the value of the i-th actual executed state parameter, is the total number of control instructions, is the adjustment amount of the j-th control instruction, is the execution frequency of the i-th control instruction, and are weight coefficients; Step B4, training the deep reinforcement learning model using historical data and control instruction execution data.

[0118] Embodiment 2

[0119] In the training process of the deep reinforcement learning model, first, the state space of the model is constructed, and the decomposed state estimation parameters and the actual state parameters after the execution of each dimension control instruction are taken as inputs and filled into the state space of the deep reinforcement learning model. Then, the action space of the model is defined, and the control instructions are refined into specific actions of attitude adjustment, orbit position adjustment, speed adjustment and acceleration adjustment as basic elements of the action space. Subsequently, a reward mechanism is designed, aiming to reduce the control error through the reward function setting, while taking into account the energy saving index, so as to balance the relationship between control precision and energy consumption. Finally, the deep reinforcement learning model is trained through historical data and control instruction execution data to improve its adaptability and control effect under different conditions.

[0120] In the training process of the deep reinforcement learning model, first, the state space of the model is constructed, and the decomposed state estimation parameters and the actual state parameters after the execution of each dimension control instruction are taken as inputs and filled into the state space of the deep reinforcement learning model. Then, the action space of the model is defined, and the control instructions are refined into specific actions of attitude adjustment, orbit position adjustment, speed adjustment and acceleration adjustment as basic elements of the action space. Subsequently, a reward mechanism is designed, aiming to reduce the control error through the reward function setting, while taking into account the energy saving index, so as to balance the relationship between control precision and energy consumption. Finally, the deep reinforcement learning model is trained through historical data and control instruction execution data to improve its adaptability and control effect under different conditions.

[0121] ​​The deep reinforcement learning algorithm is used to train the model in the embodiments of the present application. During the training process, the model learns how to make optimal control decisions in different states by interacting with the virtual environment. In each training iteration, the model selects an action based on the current state, and then obtains the next state and the corresponding reward value through the simulation of the control result. Specifically, the training process includes the following steps:

[0122] Step C1, setting the initial parameters of the deep reinforcement learning model, including learning rate, discount factor, exploration rate, etc.

[0123] Step C2, the model interacts with the virtual environment in each training iteration, and selects an action from the action space based on the current state. The selection of action is realized through the greedy strategy to ensure that the model can both explore the new state space and utilize the known optimal strategy.

[0124] Step C3, executing the selected action in the virtual environment and recording the feedback of the environment, including the next state and the reward value.

[0125] Step C4, storing the current state, selected action, obtained reward, and next state into the experience replay buffer.

[0126] Step C5, randomly drawing training samples from the experience replay buffer for batch update. Experience replay helps to break the time correlation between samples and improves the stability of training.

[0127] Step C6, updating the policy network of the model using the extracted samples to improve the model's policy by minimizing the temporal difference error.

[0128] Step C7, periodically copying the weights of the policy network to the target network to stabilize the training process.

[0129] Step C8, gradually reducing the exploration rate as the training progresses to reduce the selection of random actions and better utilize the learned policy.

[0130] Step C9, evaluating the performance of the model after each training iteration, and further adjusting the model parameters if the reward value does not reach the expected value.

[0131] The following is a Python code example for training a deep reinforcement learning model to output control instructions. Please note that this example is only a starting point, and actual applications may need to be adjusted according to actual situations and device interfaces.

[0132] ```python

[0133] import numpy as np

[0134] import tensorflow as tf

[0135] from tensorflow.keras import layers

[0136] import random

[0137] class DDPGAgent:

[0138] def __init__(self, state_size, action_size):

[0139] self.state_size = state_size

[0140] self.action_size = action_size

[0141] self.memory = []

[0142] self.gamma = 0.99

[0143] self.epsilon = 1.0

[0144] self.epsilon_min = 0.01

[0145] self.epsilon_decay = 0.995

[0146] self.learning_rate = 0.001

[0147] self.batch_size = 64

[0148] self.actor_model = self._build_actor_model()

[0149] self.critic_model = self._build_critic_model()

[0150] self.target_actor_model = self._build_actor_model()

[0151] self.target_critic_model = self._build_critic_model()

[0152] self.update_target_models()

[0153] def _build_actor_model(self):

[0154] model = tf.keras.Sequential([

[0155] layers.Dense(400, input_dim=self.state_size, activation='relu'),

[0156] layers.Dense(300, activation='relu'),

[0157] layers.Dense(self.action_size, activation='tanh') ])

[0159] model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=self.learning_rate))

[0160] return model

[0161] def _build_critic_model(self):

[0162] state_input = layers.Input(shape=(self.state_size,))

[0163] action_input = layers.Input(shape=(self.action_size,))

[0164] concat = layers.Concatenate()([layers.Dense(32, activation='relu')(state_input),

[0165] layers.Dense(32, activation='relu')(action_input)])

[0166] out = layers.Dense(256, activation='relu')(concat)

[0167] out = layers.Dense(1)(out)

[0168] model = tf.keras.Model([state_input, action_input], out)

[0169] model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=self.learning_rate), loss='mse')

[0170] return model

[0171] def update_target_models(self):

[0172] self.target_actor_model.set_weights(self.actor_model.get_weights())

[0173] self.target_critic_model.set_weights(self.critic_model.get_weights())

[0174] def remember(self, state, action, reward, next_state, done):

[0175] self.memory.append((state, action, reward, next_state, done))

[0176] def act(self, state):

[0177] if np.random.rand()<= self.epsilon:

[0178] return np.random.uniform(-1, 1, self.action_size)

[0179] return self.actor_model.predict(state)[0]

[0180] def replay(self):

[0181] if len(self.memory)<self.batch_size:

[0182] return

[0183] minibatch = random.sample(self.memory, self.batch_size)

[0184] for state, action, reward, next_state, done in minibatch:

[0185] target = reward + self.gamma * (1 - done) * self.target_critic_model.predict([next_state, self.target_actor_model.predict(next_state)])[0]

[0186] self.critic_model.train_on_batch([state, action], target)

[0187] with tf.GradientTape() as tape:

[0188] predicted_action = self.actor_model(state)

[0189] actor_loss = -tf.reduce_mean(self.critic_model([state, predicted_action]))

[0190] actor_grads = tape.gradient(actor_loss, self.actor_model.trainable_variables)

[0191] tf.keras.optimizers.Adam(learning_rate=self.learning_rate).apply_gradients(zip(actor_grads, self.actor_model.trainable_variables))

[0192] if self.epsilon > self.epsilon_min:

[0193] self.epsilon *= self.epsilon_decay

[0194] # Example usage

[0195] state_size = 4

[0196] action_size = 4

[0197] agent = DDPGAgent(state_size, action_size)

[0198] # Training loop example

[0199] for e in range(1000):

[0200] state = np.reshape(np.random.rand(state_size), [1, state_size])

[0201] for time in range(500):

[0202] action = agent.act(state)

[0203] next_state = np.reshape(np.random.rand(state_size), [1, state_size])

[0204] reward = random.random()

[0205] done = time == 499

[0206] agent.remember(state, action, reward, next_state, done)

[0207] state = next_state

[0208] if done:

[0209] agent.update_target_models()

[0210] print(f"Episode {e+1} / {1000} - Time: {time}, Epsilon:{agent.epsilon:.2}")

[0211] break

[0212] agent.replay()

[0213] This code is just an example, and in actual applications, it needs to be modified and adjusted according to the specific situation.

[0214] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0215] Finally: The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A coaxial optical radar payload satellite space-ground integrated operation and control method, characterized in that, The method comprises the following steps: Step S1, collecting position parameters, attitude parameters, relative velocity parameters and acceleration parameters of a coaxial optical radar payload satellite, and performing noise reduction processing to output noise-reduced data; Step S2, using Kalman filtering to analyze the noise-reduced data, predicting the satellite state at the current time according to the satellite state at the previous time, and outputting a state estimation value; Step S3, generating a control instruction based on the state estimation value, decomposing the control instruction, and generating a control instruction in each dimension in sequence according to the decomposed state data; Step S4, checking the generated control instruction in each dimension; Step S5, inputting the state estimation value and the control instruction execution data in each dimension into the state space of the trained deep reinforcement learning model, and optimizing the control instruction, wherein the reward function of the deep reinforcement learning model is: ； In the formula, As a reward value, This represents the total number of state parameters. For the first The values ​​of the target state parameters For the first The values ​​of the actual execution status parameters, To control the total number of commands, Let j be the adjustment amount for the j-th control command. For the first The execution frequency of each control command and These are the weighting coefficients; In combination with the spatiotemporal variation law of the relative state of the satellite and the ground station, a spatiotemporal correlation factor is introduced in the state transition process of Kalman filtering, and the update rule of the state transition matrix is improved, comprising the following steps: Step A1, extracting historical state data of relative position parameters, attitude parameters, relative velocity parameters and acceleration parameters between the satellite and the ground station; Step A2, performing differential analysis on the historical state data through a time series analysis method to calculate the state change rate between adjacent time points; Step A3, based on the change rate of the historical state, a linear regression model is constructed using a regression analysis method, and the historical data is fitted using a least squares method to solve the influence coefficient of the historical state change rate and the influence coefficient of the time interval; Step A4, defining a spatiotemporal correlation factor to reflect the spatiotemporal relationship of the relative motion between the satellite and the ground station, and the calculation formula of the spatiotemporal correlation factor is: ； In the formula, is a space-time correlation factor, is a time is a rate of change of state, is a time is a rate of change of state, is an influence coefficient of time interval, is a time difference from a time to a time . Step A5, in each time step, the state transition matrix is adjusted according to the currently calculated spatiotemporal correlation factor, and the adjustment formula is: ； wherein is the state transition matrix, is the conventional state transition matrix, is the state transition matrix of the previous time instant, is the space-time correlation factor.

2. The method according to claim 1, wherein the method is characterized by, In step S5, the state estimation parameters after decomposition of the state estimation value and the actual state parameters after execution of the control instruction in each dimension are combined into complete state information, and the state information is input as input data into the state space of the trained deep reinforcement learning model to generate optimized position control instructions, attitude control instructions, velocity control instructions and acceleration control instructions; the state estimation parameters include estimated position parameters, estimated attitude parameters, estimated velocity parameters and estimated acceleration parameters, and the actual state parameters include actual position parameters, actual attitude parameters, actual velocity parameters and actual acceleration parameters; the position control instruction includes a position adjustment amount, an adjustment direction and execution time information; the attitude control instruction includes an angle adjustment amount, a rotation axis and execution time information; the velocity control instruction includes a velocity adjustment amount, an adjustment direction and execution time information; and the acceleration control instruction includes an acceleration adjustment amount, an adjustment direction and execution time information.

3. The integrated satellite-ground operation and control method for a coaxial optical radar payload according to claim 1 or 2, characterized in that, Training the deep reinforcement learning model comprises the following steps: Step B1, constructing a state space of the deep reinforcement learning model, inputting the state estimation parameters after the state estimation value decomposition and the actual state parameters after the execution of each dimension control instruction into the state space of the deep reinforcement learning model; Step B2, defining an action space of the deep reinforcement learning model, refining the control instruction into specific actions of attitude adjustment, orbit position adjustment, speed adjustment and acceleration adjustment as elements of the action space; Step B3, designing a reward mechanism, setting a reward function according to the control error and energy saving index, taking reducing the control error as the main target and saving energy as the secondary target, balancing the relationship between precision and energy consumption; Step B4, training the deep reinforcement learning model by using historical data and control instruction execution data.

4. The method of claim 1, wherein the method further comprises: The position parameter is the relative position between the satellite and the ground station, which is obtained by real-time collection of the relative position of the satellite and the ground station in the geographic coordinate system using coaxial optical radar, and is expressed using Cartesian coordinates; the attitude parameter is the attitude angle of the satellite, including the pitch angle, roll angle and yaw angle of the satellite, and is expressed using Euler angles; the relative velocity parameter is the relative velocity between the satellite and the ground station, including the relative horizontal velocity, relative vertical velocity and relative normal velocity; and the acceleration parameter is the acceleration of the satellite on its orbit, including the horizontal acceleration component, vertical acceleration component and normal acceleration component.

5. The method of claim 1, wherein the method further comprises: Step S3 includes the following steps: Step S3.1, decomposing the state estimation value output by the Kalman filter into estimated position parameters, estimated attitude parameters, estimated relative velocity parameters and estimated acceleration parameters; Step S3.2, comparing the current estimated position parameters with the target position parameters, calculating the required displacement adjustment amount, and decomposing the total displacement adjustment amount according to the set fixed increment, calculating the required adjustment steps and the amount of each adjustment, generating a position control instruction, the position control instruction including the position adjustment amount, adjustment direction and execution time information; Step S3.3, comparing the current estimated attitude angle with the target attitude angle, calculating the required attitude angle adjustment amount, and decomposing the total attitude angle adjustment amount according to the set fixed increment, calculating the required adjustment steps and the angle increment of each adjustment, generating an attitude control instruction, the attitude control instruction containing the angle adjustment amount, rotation axis and execution time information; Step S3.4, comparing the current estimated velocity with the target velocity, calculating the required velocity adjustment amount, and decomposing the total velocity adjustment amount according to the set fixed increment, calculating the required adjustment steps and the increment of each adjustment, generating a speed control instruction, the speed control instruction including the speed adjustment amount, adjustment direction and execution time information; Step S3.5, comparing the current estimated acceleration with the target acceleration, calculating the required acceleration adjustment amount, and decomposing the total acceleration adjustment amount according to the set fixed increment, calculating the required adjustment steps and the increment of each adjustment, generating an acceleration control instruction, the acceleration control instruction including the acceleration adjustment amount, adjustment direction and execution time information; Step S3.6, scheduling the decomposed control instructions according to time sequence, and taking the generated segmented control instruction set as the final control instruction.

6. The method of claim 1, wherein the method further comprises: Step S4 includes the following steps: Step S4.1, after executing the position control instruction, the current position parameters of the satellite are collected again by the coaxial optical radar, and the actual position after execution is compared with the target position to calculate the position deviation. When the position deviation is less than the preset error range, the check passes; when the position deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment; Step S4.2, after executing the attitude control instruction, the current attitude parameters of the satellite are collected again by the coaxial optical radar, and the actual attitude after execution is compared with the target attitude to calculate the attitude angle deviation. When the attitude angle deviation is less than the preset error range, the check passes; when the attitude angle deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment; Step S4.3, after executing the speed control instruction, the current speed parameters of the satellite are collected again by the coaxial optical radar, and the actual speed after execution is compared with the target speed to calculate the speed deviation. When the speed deviation is less than the preset error range, the check passes; when the speed deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment; Step S4.4, after executing the acceleration control instruction, the current acceleration parameters of the satellite are collected again by the coaxial optical radar, and the actual acceleration after execution is compared with the target acceleration to calculate the acceleration deviation. When the acceleration deviation is less than the preset error range, the check passes; when the acceleration deviation is greater than or equal to the preset error range, the error is recorded and marked for re-adjustment.

Images