Double-depth Q network ultralow-temperature vacuum two-dimensional rotary table heat preservation active-disturbance-rejection control method

Abstract

The invention discloses an ultralow-temperature vacuum two-dimensional rotary table heat preservation active-disturbance-rejection control method based on a double-depth Q network, and the method comprises the steps: carrying out the identification modeling of a two-dimensional rotary table temperature control system, and carrying out the real-time estimation and compensation of unmodeled dynamics and external disturbance through an ESO (extended state observer). A reinforcement learning algorithm is adopted to establish a Markov decision process model of observer and controller parameters, and the Markov decision process model comprises a state space, an action space and a reward function. A dual-depth Q network is designed, a current network selects an action, a target network calculates a Q value to carry out action evaluation, a Bellman equation is adopted in the training process to carry out Q value updating, and the training target is maximized accumulated rewards. Finally, the trained DDQN-ADRC network is applied to an ultralow-temperature two-dimensional turntable control example, and compared with simulation of a PID controller and a fixed-parameter ADRC controller under two working conditions of changing a set value and externally applying disturbance, the algorithm shows better tracking precision, anti-interference capability and robust performance.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent control and extreme environment applications, and in particular relates to a dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation self-disturbance rejection control method. Background Art

[0002] The 2D turntable serves as the core motion actuator for achieving wide-field remote sensing imaging in ground-based space camera simulations. It drives the space camera in both horizontal and vertical degrees of freedom, thereby acquiring remote sensing images from various orientations. Given that the on-orbit operating temperature of the space camera is below 100K, thermal control design of key components within the 2D turntable is crucial to ensure the proper performance of key components such as the grating, motor, and bearings.

[0003] Reinforcement learning (RL) is a learning method based on reward and punishment mechanisms. It does not rely on precise mathematical models. Instead, it uses the interaction between an agent and its environment to self-learn how to take actions to maximize long-term rewards. Deep reinforcement learning (DRL), in particular, can effectively process large-scale, high-dimensional state space data, enabling more accurate and flexible learning and decision-making.

[0004] As a classic control strategy, the PID controller is widely used in industrial control systems due to its simple structure and ease of implementation. However, PID is sensitive to system changes, struggles with nonlinear and time-varying environments, has poor interference immunity, and is difficult to adjust parameters. Its response speed and control accuracy are particularly limited in specialized environments such as ultra-low temperatures and vacuum conditions.

[0005] Active disturbance rejection control (ADRC) technology does not require a precise mathematical model of the controlled object. Instead, it dynamically estimates disturbances and uncertainties through an extended state observer, effectively suppressing the impact of external interference on system performance. However, under the complex and changing on-orbit conditions of spacecraft, fixed-parameter ADRC controllers struggle to guarantee ideal tracking performance. Summary of the Invention

[0006] Purpose of the Invention: This invention provides a dual-deep Q-network-based active disturbance rejection control method for a cryogenic vacuum two-dimensional turntable. The two-dimensional turntable temperature control system is identified and modeled. An extended state observer (ESO) is used to estimate and compensate for unmodeled dynamics and external disturbances in real time. A dual-deep Q-network (DDQN) is designed and trained to select and evaluate controller parameters. Ultimately, this method is applied to a cryogenic two-dimensional turntable control example, improving tracking accuracy and robustness.

[0007] Technical solution: The present invention provides a dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method, comprising the following steps:

[0008] Step 1: Construct a two-dimensional turntable thermal control model ADRC in an ultra-low temperature vacuum environment and perform bias relay identification on the thermal control model;

[0009] Step 2: Based on the ADRC model of the two-dimensional turntable thermal control in ultra-low temperature vacuum environment, the extended state observer (ESO) is used to estimate and compensate for the unmodeled dynamics and external disturbances in real time;

[0010] Step 3: Use reinforcement learning algorithm to establish the Markov decision process model MDP of observer and controller parameters, including state space, action space and reward function;

[0011] Step 4. Design a dual deep Q network (DDQN) consisting of a current Q network and a target Q network. The current Q network selects actions, and the target Q network calculates Q values for action evaluation. The Bellman equation is used to update Q values during training, and the training goal is to maximize the cumulative reward.

[0012] Step 5: Apply the trained DDQN-ADRC network to ultra-low temperature two-dimensional turntable control.

[0013] Furthermore, step 1 is specifically as follows: the thermal control model uses a polyimide film heater for electrical heating, and uses the temperature of each component of the two-dimensional turntable as control feedback to actively adjust the heating power and control the temperature of the two-dimensional turntable; the bias relay identification is specifically as follows: a first-order inertia link with pure hysteresis is used to represent:

[0014]

[0015] Through the relay link, the phase lag of the controlled object at high frequency is made greater than -π, thereby obtaining the amplitude and oscillation frequency of the model at the critical point, and then the calculation formulas of various identification parameters are obtained as follows:

[0016]

[0017] Among them, K u is the critical gain of the process model, W u is the oscillation frequency of the process model, T u is the oscillation period of the process model, a is the amplitude of the process output, d is the amplitude of the relay link, K is the gain of the model, T is the time constant of the model, and L is the delay of the model.

[0018] Furthermore, step 2 is as follows: Since the controlled object has a time delay, the control action has a transmission delay, so the delay is modeled into the extended state observer ESO. The ESO expression is as follows:

[0019]

[0020] The ADRC control system is controlled by adjusting the controller parameters b0, β1, β2, k p , so that z1 tracks the system output y, z2 tracks the total disturbance f; b0 is extracted through the system transfer function, β1, β2, k p By configuring the closed-loop system poles, the parameter tuning complexity is reduced. u represents the control input, t represents the time variable, and L represents the time delay.

[0021] The feedback control law uses z1 as negative feedback and is expressed as follows:

[0022]

[0023] Where r represents the set signal value to be tracked;

[0024] The expected dynamic characteristics of the closed-loop system are:

[0025]

[0026] Among them, y d represents the expected response value, s represents the complex frequency domain variable in Laplace transform;

[0027] The controller transfer function is:

[0028]

[0029] For the characteristic equation of the state observer ESO, the closed-loop pole is configured at s = -ω o , we can get the observer parameter β1=2ω o ,β2=ω o 2 ,ω o Observer bandwidth; for the characteristic equation of the ADRC controller, the closed-loop pole is configured at s = -ω c , we can get the controller parameter k p =ω c ,ω c is the controller bandwidth;

[0030] The ADRC controller with fixed parameters and the classic PID controller were used as control groups. The formula of the classic PID controller is as follows:

[0031]

[0032] Among them, K P , K I , K D , K N They are proportional, integral, differential and filter coefficients respectively.

[0033] Furthermore, in step 3, the state space is specifically:

[0034] S=[e i ,st i ] T (i=1,2...n) (8)

[0035] Where i represents a time step in the simulation process, n is the number of time steps required for the agent to complete a complete interaction sequence Episode with the environment; e i is the difference between the mean value of the input signal in the next time step and the last output value of the system simulation in the current time step, that is, the error between the expected value and the current output value; st i The mean of the input signal for the next time step is the future target.

[0036] Furthermore, in step 3, the action space is specifically:

[0037] A=[ω o ,ω c ] T (9)

[0038] Among them, ω o is the bandwidth of the extended state observer, ω c To satisfy the tracking response requirements, different actions need to be designed for different input signals during the training process.

[0039] Furthermore, in step 3, the reward function is specifically:

[0040] R t =-k1(IAE-IAE d )-k2|M S -1.4| 2 +k3G m +k4P m (10)

[0041] Among them, P m , G m are the amplitude margin and phase margin of the system respectively. k1, k2, k3, and k4 are weight coefficients that can be flexibly adjusted according to the actual training situation. IAE is the total deviation between the output value y and the set value r. The calculation formula is as follows:

[0042]

[0043] Where T is the total time length of each time step, and t is the set signal value to be tracked;

[0044] IAE d is the expected output value y d The total deviation from the set value r is calculated as follows:

[0045]

[0046] M S is the maximum sensitivity function of the system, and the calculation formula is as follows:

[0047]

[0048] Among them G op (s) is the system open-loop transfer function:

[0049] G op (s)=G c (s)G p (s) (14)

[0050] The reward function is a real-time reward function within each time step, and the cumulative reward function within one round is:

[0051]

[0052] R t Indicates the cumulative reward value in one round.

[0053] Furthermore, step 4 is as follows: the dual-depth Q network is divided into the current Q network and the target Q network; the current Q network generates experience data through interactive learning with the environment during the training process, the current state s, action a, reward R t , the next state s′ is stored in the experience replay pool; the target Q network calculates the target Q value corresponding to the action based on the optimal action corresponding to the maximum Q value found by the current Q network, and evaluates the selected action;

[0054] The Bellman equation is used to update the Q value. The calculation formula of the Q value is as follows:

[0055]

[0056] Among them, γ is the discount rate, which is used to measure the current reward and future long-term reward, θ, θ - are the parameters of the current Q network and the target Q network respectively, The optimal action selected by the current Q network based on the maximum Q value The next state value function, i.e. the expected cumulative reward;

[0057] The action selection strategy adopts the ε-greedy strategy, which uses the probability ε to make a trade-off between exploration and exploitation. The probability ε changes according to the training rounds, so that the intelligent agent can conduct an extensive search of the environment in the early stage of training to obtain sufficient training samples, and make full use of the optimal strategy to accelerate convergence in the later stage of training.

[0058] Furthermore, in step 5, the trained DDQN-ADRC network is subjected to system robustness analysis using the Monte-Carlo random method.

[0059] The present invention further discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0060] The present invention further discloses a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of the method of the present invention when the computer program / instruction is executed by a processor.

[0061] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0062] (1) This paper trains a dual-depth Q network and applies it to a two-dimensional turntable in an ultra-low-temperature vacuum environment, achieving timely adjustment and optimization of ADRC control parameters. Compared to classic PID controllers and fixed-parameter ADRC controllers, the DDQN-ADRC can more accurately and rapidly track setpoint temperature changes, better adapt to ultra-low-temperature environments, and reduce the adjustment frequency of the actuator, thereby reducing the energy consumption of electric heating.

[0063] (2) The present invention identifies and models the temperature control system of a two-dimensional turntable, uses the extended state observer (ESO) to estimate and compensate for unmodeled dynamics and external interference in real time, designs and trains a dual deep Q network (DDQN) to select and evaluate controller parameters, and finally applies it to an ultra-low temperature two-dimensional turntable control example, improving tracking accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a two-dimensional turntable thermal insulation auto-disturbance rejection control method in an ultra-low temperature vacuum environment based on a dual-depth Q network;

[0065] Figure 2 A schematic diagram of a two-dimensional turntable structure and a heating plate installation position according to a specific embodiment of the present invention;

[0066] Figure 3 Schematic diagram of a simulation platform for an active disturbance rejection control system based on DDQN according to a specific embodiment of the present invention;

[0067] Figure 4 A schematic diagram of the structure of an ADRC feedback controller according to a specific embodiment of the present invention;

[0068] Figure 5 A schematic diagram of the structure of an ESO extended state observer according to a specific embodiment of the present invention;

[0069] Figure 6Schematic diagram of the neural network structure of the current and target networks according to a specific embodiment of the present invention.

[0070] Figure 7 This is a flow chart of the DDQN algorithm according to a specific embodiment of the present invention.

[0071] Figure 8 This is a training curve diagram of the DDQN-ADRC algorithm according to a specific embodiment of the present invention.

[0072] Figure 9 This is an offline simulation response tracking curve diagram of a specific embodiment of the present invention.

[0073] Figure 10 This is a graph showing the tracking performance of an extended state observer (ESO) in offline simulation according to a specific embodiment of the present invention.

[0074] Figure 11 This is an offline simulation of a specific embodiment of the present invention. o With ω c Dynamic change graph.

[0075] Figure 12 This is an offline simulation of a specific embodiment of the present invention. S 、P m With G m Dynamic change graph.

[0076] Figure 13 This is a simulation tracking result of changing the set value of a specific embodiment of the present invention.

[0077] Figure 14 The change setting value ω of a specific embodiment of the present invention is o With ω c Dynamic change graph.

[0078] Figure 15 This is a simulation tracking result of external interference according to a specific embodiment of the present invention.

[0079] Figure 16 The external interference ω of a specific embodiment of the present invention is o With ω c Dynamic change graph.

[0080] Figure 17 This is a Monte-Carlo random perturbation test diagram of a specific embodiment of the present invention. (a) is the IAE-adjustment time T s Distribution diagram, (b) is the system overshoot σ-adjustment time T s Distribution map. DETAILED DESCRIPTION

[0081] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0082] like Figure 1 This example describes a method for thermal insulation auto-disturbance rejection control of a two-dimensional turntable in an ultra-low temperature vacuum environment based on a dual-depth Q network. The process is as follows: Figure 1 The method comprises the following steps:

[0083] Step 1: The present invention conducts control design research based on a two-dimensional turntable thermal control model in an ultra-low temperature vacuum environment. The system transfer function of the thermal control system is obtained through a bias relay identification experiment.

[0084] Step 2: Based on the proposed two-dimensional turntable thermal control model under ultra-low temperature vacuum environment, a DDQN-based thermal insulation anti-disturbance control system is established.

[0085] In step 3, for the two parameters to be adjusted in step 2, a reinforcement learning algorithm is used to establish a Markov decision process (MDP) model for the observer and controller parameters, including a state space to describe uncertainty, an action space for adjusting device parameters, and a real-time feedback reward function.

[0086] Step 4: Establish a dual-depth Q network for training. The current Q network selects actions, the target Q network calculates Q values, and the Bellman equation is used to update Q values.

[0087] Step 5: Save the trained DDQN-ADRC network and load it into an embodiment of the present invention for real-time parameter adjustment.

[0088] Step 6: For one embodiment of the present invention, DDQN-ADRC is used to perform real-time parameter adjustment control by changing the set value and adding external disturbance, and simulation comparison is performed with the classic PID controller and the fixed-parameter ADRC controller.

[0089] Step 7: For one embodiment of the present invention, a Monte-Carlo random method is used to perform system robustness analysis.

[0090] like Figure 3 Figure 4 Figure 5 The DDQN-ADRC control method of the embodiment of the present invention includes: an ESO extended state observer, which estimates the state variables of the two-dimensional turntable in real time and designs the ADRC controller based on negative feedback of the output temperature. The DDQN algorithm adjusts the ADRC controller parameters according to the difference between the output response and the set signal input so that the system output reaches the set temperature.

[0091] According to one embodiment of the present invention, the parameters set during DDQN-ADRC network training are shown in Table 1:

[0092] Table 1 Reinforcement learning training parameters

[0093]

[0094]

[0095] The training curve of the DDQN-ADRC algorithm is as follows Figure 8 As shown in the figure, the average curve is the smoothed value of the training reward every 5 steps. As can be seen from the curve, the training convergence effect is good.

[0096] DDQN-ADRC is applied to offline simulation to simulate the temperature control of the two-dimensional turntable on-orbit operation scenario. The response tracking curve is shown in the figure below. Figure 9 The IAE of the ADRC control system with dynamic parameter adjustment by DDQN is 173.2, while the IAE of the ADRC controller with the median of the action interval as a fixed parameter is 194.9. This shows that the DDQN algorithm can greatly improve the response tracking of the ADRC control system to the input value.

[0097] Depend on Figure 10 It can be seen that the real-time estimation deviations of ESO for the total disturbance f and the output value y are close to 0, and both have good tracking effects.

[0098] Depend on Figure 11 Figure 12 It can be seen that the DDQN-ADRC controller shows moderate sensitivity and a certain response speed during the dynamic change of parameters. It is not overly sensitive to noise and has a phase margin P that meets engineering applications. m and amplitude margin G m .

[0099] Furthermore, the thermal control model operating conditions are changed for comparative simulation.

[0100] Condition 1: Since the optimal operating temperature for the components of the 2D turntable is around 10°C, the set temperature is changed, and the set value is given as a step signal to observe the controller performance. The comparison results between DDQN-ADRC and fixed parameter ADRC are as follows:

[0101] Table 2 Simulation comparison of changing set values

[0102]

[0103] Where T sThe settling time is defined as the time it takes for the system to enter and remain within ±2% of its stable value. Table 2 shows that under setpoint jump conditions, the DDQN-ADRC controller exhibits superior dynamic performance, with shorter settling time and smaller IAE. Furthermore, its robust performance meets control requirements, demonstrating that the parameter tuning network can achieve real-time parameter tuning under setpoint tracking conditions.

[0104] Working condition 2: When the 2D turntable gradually enters a specific orbit, the ambient temperature will gradually drop below 100K. At this time, active electric heating is required to ensure that the 2D turntable remains at a normal working temperature. At the same time, due to the complex and changeable space environment, many external factors interfere with the controlled object. Therefore, if Figure 3 As shown in Figure 1, a ramp signal and a sine signal are added to the system to simulate the ambient temperature that continues to drop in the fluctuation. Figure 15 As shown, the green curve has the largest fluctuation and slow response, indicating that the PID controller has poor anti-disturbance ability. The comparison results between DDQN-ADRC and fixed parameter ADRC are shown in Table 3;

[0105] Table 3 Comparison of external disturbance tracking performance

[0106]

[0107] It can be seen that when periodic external disturbance occurs, the DDQN-ADRC control system achieves a shorter response time, and the maximum amplitude is controlled at about ±1.2°C. Therefore, its total deviation IAE is significantly smaller than that of the fixed parameter ADRC control system. After verification, the phase margin P m , amplitude margin G m All are within the scope allowed by the project.

[0108] Furthermore, in one embodiment of the present invention, a Monte-Carlo random method is used to perform system robustness analysis. Assuming that the system gain K, time constant T, and delay time L of the controlled object are perturbed by ±5% around the nominal value, and the controller parameters remain unchanged at the nominal value, 600 random experiments are performed and the system overshoot σ-adjustment time T is plotted. s , IAE-adjustment time T s ,like Figure 17 As shown in Figure 2, the DDQN-ADRC controller has good robustness when system parameters change randomly, especially compared with the PID controller. In addition, compared with the other two controllers, the DDQN-ADRC controller has the smallest overshoot σ and adjustment time T. s and IAE, showing the good comprehensive performance of the DDQN-ADRC controller.

[0109] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. "Multiple" means two or more, unless otherwise specifically defined.

[0110] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0111] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0112] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0113] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0114] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0115] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0116] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0117] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0118] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A dual-depth Q network ultra-low temperature vacuum two-dimensional turntable insulation auto-disturbance rejection control method, characterized in that: The steps include: Step 1: Construct a two-dimensional turntable thermal control model ADRC in an ultra-low temperature vacuum environment and perform bias relay identification on the thermal control model; Step 2: Based on the ADRC model of the two-dimensional turntable thermal control in ultra-low temperature vacuum environment, the extended state observer (ESO) is used to estimate and compensate for the unmodeled dynamics and external disturbances in real time; Step 3: Use reinforcement learning algorithm to establish the Markov decision process model MDP of observer and controller parameters, including state space, action space and reward function; Step 4. Design a dual deep Q network (DDQN) consisting of a current Q network and a target Q network. The current Q network selects actions, and the target Q network calculates Q values for action evaluation. The Bellman equation is used to update Q values during training, and the training goal is to maximize the cumulative reward. Step 5: Apply the trained DDQN-ADRC network to ultra-low temperature two-dimensional turntable control.

2. The dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method according to claim 1 is characterized in that: Step 1 specifically includes: the thermal control model uses a polyimide film heater for electrical heating, and uses the temperature of each component of the two-dimensional turntable as control feedback to actively adjust the heating power and control the temperature of the two-dimensional turntable; the bias relay identification specifically includes: using a first-order inertia link with pure hysteresis to represent: Through the relay link, the phase lag of the controlled object at high frequency is made greater than -π, thereby obtaining the amplitude and oscillation frequency of the model at the critical point, and then the calculation formulas of various identification parameters are obtained as follows: Among them, K u is the critical gain of the process model, W u is the oscillation frequency of the process model, T u is the oscillation period of the process model, a is the amplitude of the process output, d is the amplitude of the relay link, K is the gain of the model, T is the time constant of the model, and L is the delay of the model.

3. The dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method according to claim 1 is characterized in that: Step 2 is as follows: Since the controlled object has a time delay, the control action has a transmission delay, so the delay is modeled into the extended state observer ESO. The ESO expression is as follows: The ADRC control system is controlled by adjusting the controller parameters b0, β1, β2, k p , so that z1 tracks the system output y, z2 tracks the total disturbance f; b0 is extracted through the system transfer function, β1, β2, k p By configuring the closed-loop system poles, the parameter tuning complexity is reduced. u represents the control input, t represents the time variable, and L represents the time delay. The feedback control law uses z1 as negative feedback and is expressed as follows: Where r represents the set signal value to be tracked; The expected dynamic characteristics of the closed-loop system are: Among them, y d represents the expected response value, s represents the complex frequency domain variable in Laplace transform; The controller transfer function is: For the characteristic equation of the state observer ESO, the closed-loop pole is configured at s = -ω o , we can get the observer parameter β1=2ω o ,β2=ω o 2 ,ω o Observer bandwidth; for the characteristic equation of the ADRC controller, the closed-loop pole is configured at s = -ω c , we can get the controller parameter k p =ω c ,ω c is the controller bandwidth; The ADRC controller with fixed parameters and the classic PID controller were used as control groups. The formula of the classic PID controller is as follows: Among them, K P , K I , K D , K N They are proportional, integral, differential and filter coefficients respectively.

4. The dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method according to claim 1, characterized in that: In step 3, the state space is specifically: S=[e i ,st i ] T (i=1,2...n) (8) Where i represents a time step in the simulation process, n is the number of time steps required for the agent to complete a complete interaction sequence Episode with the environment; e i is the difference between the mean value of the input signal in the next time step and the last output value of the system simulation in the current time step, that is, the error between the expected value and the current output value; st i The mean of the input signal for the next time step is the future target.

5. The dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method according to claim 1, characterized in that: In step 3, the action space is specifically: A=[ω o ,oh c ] T (9) Among them, ω o is the bandwidth of the extended state observer, ω c To satisfy the tracking response requirements, different actions need to be designed for different input signals during the training process.

6. The dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method according to claim 1, characterized in that: In step 3, the reward function is specifically: R t =-k1(IAE-IAE d )-k2|M S -1.4| 2 +k3G m +k4P m (10) Among them, P m , G m are the amplitude margin and phase margin of the system respectively. k1, k2, k3, and k4 are weight coefficients that can be flexibly adjusted according to the actual training situation. IAE is the total deviation between the output value y and the set value r. The calculation formula is as follows: Where T is the total time length of each time step, and t is the set value to be tracked; IAE d is the expected output value y d The total deviation from the set value r is calculated as follows: M S is the maximum sensitivity function of the system, and the calculation formula is as follows: Among them G op (s) is the system open-loop transfer function: G op (s)=G c (s)G p (s) (14) The reward function is a real-time reward function within each time step, and the cumulative reward function within one round is: R t Indicates the cumulative reward value in one round.

7. The dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method according to claim 1, characterized in that: Step 4 is specifically as follows: the dual-depth Q network is divided into the current Q network and the target Q network; The current Q network generates experience data through interactive learning with the environment during the training process, the current state s, action a, reward R t , the next state s′, and stored in the experience replay pool; The target Q network calculates the target Q value corresponding to the optimal action corresponding to the maximum Q value found by the current Q network and evaluates the selected action; The Bellman equation is used to update the Q value. The calculation formula of the Q value is as follows: Among them, γ is the discount rate, which is used to measure the current reward and future long-term reward, θ, θ - are the parameters of the current Q network and the target Q network respectively, The optimal action selected by the current Q network based on the maximum Q value The next state value function, i.e. the expected cumulative reward; The action selection strategy adopts the ε-greedy strategy, which uses the probability ε to make a trade-off between exploration and exploitation. The probability ε changes according to the training rounds, so that the intelligent agent can conduct an extensive search of the environment in the early stage of training to obtain sufficient training samples, and make full use of the optimal strategy to accelerate convergence in the later stage of training.

8. The dual-depth Q network ultra-low temperature vacuum two-dimensional turntable thermal insulation auto-disturbance rejection control method according to claim 1, characterized in that: In step 5, the trained DDQN-ADRC network is subjected to system robustness analysis using the Monte-Carlo random method.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to claim 1.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

Images