Model Predictive Control Method Based on Sequence Generation Model
By using a sequence generation model to fit the control sequence and reinforcement learning methods, this approach addresses the shortcomings of existing model predictive control methods in terms of control strategy optimization and system constraints, achieving more efficient control sequence prediction, particularly demonstrating effectiveness in spacecraft trajectory optimization problems.
Patent Information
- Application Number
- CN202411285438.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-13
AI Technical Summary
Existing model predictive control methods have shortcomings in control strategy optimization and system constraint satisfaction, making it difficult to effectively combine local and global performance, resulting in inaccurate control sequence predictions.
A sequence generation model is used to fit the control sequence. Combined with reinforcement learning, the value function of the current state is fitted by the sequence generation model, which improves the accuracy of the assessment of the future state and reduces the sampling sample size to improve the efficiency of model prediction and control.
By improving the accuracy of control sequence prediction and reducing the sample size, the efficiency and accuracy of model predictive control are enhanced, especially demonstrating effectiveness in spacecraft trajectory optimization problems.
Smart Images

Figure CN118778460B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of motion control, and in particular relates to a model predictive control method based on a sequence generation model and related products. Background Art
[0002] Model predictive control methods usually optimize the performance of control strategies based on a full description of the system model, while ensuring that the control process can meet system constraints. For example, Chinese invention patent application CN117492366A proposes a parallel puncture robot trajectory control method based on event-triggered model predictive control. Chinese invention patent application CN108897217A proposes a trajectory tracking control method for an underactuated surface ship based on model predictive control.
[0003] The present invention proposes a novel model predictive control method based on a sequence generation model. Summary of the invention
[0004] The present invention provides a model predictive control method based on a sequence generation model and related products.
[0005] The technical solution of the present invention is as follows: A model predictive control method based on a sequence generation model, comprising:
[0006] Step 1: Provide initial value for input value ;
[0007] Step 2: Loop through the following prediction steps: Input the current value of the input value into the sequence generation model to get the output value, where the input value is recorded as input. , the output value is counted as output, , is the state of the controlled system, and its subscript represents time. is the control input of the controlled system, and its subscript represents time. To predict the number of steps, is the value function, whose subscript represents time, and then according to and And the dynamic model update of the controlled system , will update The output value after the prediction is used as the input value to perform the prediction step again, and the prediction control input sequence is obtained after the cycle ends;
[0008] Step 3: Calculate the loss function based on the predictive control input sequence and the target control input sequence, and update the sequence generation model based on the loss function;
[0009] Step 4: loop through steps 2 and 3 to obtain a trained sequence generation model;
[0010] Step 5, execute the predictive control steps in a loop. In each predictive control step, obtain the current value of the input value, and input it into the trained sequence generation model to obtain the output value. Use the control input sequence in the output value as the mean of the Gaussian distribution sequence, sample the Gaussian distribution sequence to obtain multiple alternative control input sequences, select an alternative control input sequence from the alternative control input sequence according to the value function corresponding to each alternative control input sequence, replace the control input sequence in the output value with the selected alternative control input sequence, and update the value function in the output value according to the selected alternative control input sequence. Apply the control input corresponding to the current moment in the input value to the controlled system to obtain the state value at the next moment, replace the state value at the next moment predicted in the output value with the state value at the next moment obtained after the controlled system is actually operated, and use the updated output value as the input value to execute the next predictive control step.
[0011] The technical solution of the present invention is as follows: A model training device includes a memory and a processor, the memory stores a program, and the processor runs the program to execute:
[0012] Step 1: Provide initial value for input value ;
[0013] Step 2: Loop through the following prediction steps: Input the current value of the input value into the sequence generation model to get the output value, where the input value is recorded as input. , the output value is counted as output, , is the state of the controlled system, and its subscript represents time. is the control input of the controlled system, and its subscript represents time. To predict the number of steps, is the value function, whose subscript represents time, and then according to and And the dynamic model update of the controlled system , will update The output value after the prediction is used as the input value to perform the prediction step again, and the prediction control input sequence is obtained after the cycle ends;
[0014] Step 3: Calculate the loss function based on the predictive control input sequence and the target control input sequence, and update the sequence generation model based on the loss function;
[0015] Step 4: Loop through steps 2 and 3 to obtain the trained sequence generation model.
[0016] The technical solution of the present invention is as follows: A controller includes a memory and a processor, the memory stores a program, and the processor runs the program to execute:
[0017] The predictive control steps are executed cyclically. In each predictive control step, the current value of the input value is obtained and input into the trained sequence generation model to obtain the output value. The control input sequence in the output value is used as the mean of the Gaussian distribution sequence. The Gaussian distribution sequence is sampled to obtain multiple candidate control input sequences. An alternative control input sequence is selected from the alternative control input sequence according to the value function corresponding to each alternative control input sequence. The control input sequence in the output value is replaced with the selected alternative control input sequence, and the value function in the output value is updated according to the selected alternative control input sequence. The control input corresponding to the current moment in the input value is applied to the controlled system to obtain the state value at the next moment. The state value at the next moment predicted in the output value is replaced with the state value at the next moment obtained after the controlled system is actually operated. The updated output value is used as the input value to execute the next predictive control step, wherein the input value is recorded as input. , the output value is counted as output, , is the state of the controlled system, and its subscript represents time. is the control input of the controlled system, and its subscript represents time. To predict the number of steps, is the value function, and its subscript represents time.
[0018] The technical solution of the present invention is as follows: a moving body comprises the above-mentioned controller.
[0019] The powerful characterization ability of the sequence generation model is used to fit the control sequence to measure the local performance of the control strategy; in addition, based on the reinforcement learning method, the value of the current state is simultaneously fitted to measure the global performance of the control strategy, thereby enhancing the accuracy of the sampling-based model predictive control method in assessing future states. Compared with the existing sampling-based model predictive control method, the present invention proposes to use the sequence generation model to fit the control sequence (i.e., predictive control input sequence) generation strategy, which improves the accuracy of the control sequence prediction and reduces the sampling sample size of the control sequence in the model predictive control process, thereby improving the efficiency of sequence generation in the sampling-based model predictive control. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is the architecture of the sequence generation model of the embodiment of the present invention.
[0021] Figure 2 This is the motion trajectory corresponding to the predictive control input sequence of a test example of the present invention. The starting positions of different black lines are different, that is, multiple simulations are performed. DETAILED DESCRIPTION
[0022] The present invention is further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto.
[0023] The present invention provides a model predictive control method based on a sequence generation model, comprising the following steps.
[0024] Step 1: Provide initial value for input value .
[0025] Step 2: Loop through the following prediction steps: Input the current value of the input value into the sequence generation model to get the output value, where the input value is recorded as input. , the output value is counted as output, , is the state of the controlled system, and its subscript represents time. is the control input of the controlled system, and its subscript represents time. To predict the number of steps, is the value function, whose subscript represents time, and then according to and And the dynamic model update of the controlled system , will update The output value after that is used as the input value to execute the prediction step again, and the prediction control input sequence is obtained after the cycle ends.
[0026] The state of the controlled system is, for example, position, velocity, angular velocity, etc. The prediction step is greater than or equal to 2. The dynamic model of the controlled system is known and can be determined according to the prior art. The dynamic model of the controlled system describes the influence of the current state and the current control input on the state at the next moment.
[0027] Step 3: Calculate the loss function based on the predicted control input sequence and the target control input sequence, and update the sequence generation model based on the loss function.
[0028] The target control input sequence is known and can be obtained by any known method. For example, the target control input sequence is pre-generated using a sampling-based model predictive control method. The target control input sequence can also be a record of the real control input of a well-performing controlled system during actual operation.
[0029] Optionally, the loss function is written as , , Input sequence for target control, Enter the sequence for predictive control.
[0030] Step 4: Loop through steps 2 and 3 to obtain the trained sequence generation model.
[0031] refer to Figure 1 , the sequence generation model includes an encoder and a decoder. The sequence generation model first encodes the input value input, then predicts it, and then decodes the prediction result to get the output value output.
[0032] Optionally, the sequence generation model is a generative pre-trained model (GPT). The present invention does not limit the encoding and decoding methods, which can be designed according to the prior art. The present invention does not limit the structure of the generative pre-trained model, which can be designed according to the prior art.
[0033] Step 5, execute the predictive control steps in a loop. In each predictive control step, obtain the current value of the input value, and input it into the trained sequence generation model to obtain the output value. Use the control input sequence in the output value as the mean of the Gaussian distribution sequence, sample the Gaussian distribution sequence to obtain multiple alternative control input sequences, select an alternative control input sequence from the alternative control input sequence according to the value function corresponding to each alternative control input sequence, replace the control input sequence in the output value with the selected alternative control input sequence, and update the value function in the output value according to the selected alternative control input sequence. Apply the control input corresponding to the current moment in the input value to the controlled system to obtain the state value at the next moment, replace the state value at the next moment predicted in the output value with the state value at the next moment obtained after the controlled system is actually operated, and use the updated output value as the input value to execute the next predictive control step.
[0034] Initial value of input value For example, it can be generated according to the existing sampling-based model predictive control method, and then the sequence generation model is cyclically calculated to obtain the output value.
[0035] Each item in the Gaussian distribution sequence is a random variable that follows a Gaussian distribution. The variance of each random variable can be a set value.
[0036] According to the alternative control input sequence, the next state value predicted by the controlled system, and the dynamic model of the controlled system, the state value corresponding to each moment in the alternative control input sequence can be determined, and then the value function corresponding to the alternative control input series can be calculated. The alternative control input sequence with the highest value function is selected to update the output value.
[0037] The control input is used to control the motion parameters of the controlled system, and the motion parameters include at least one of velocity, acceleration, angular velocity, and angular acceleration.
[0038] Optionally, the value function is calculated as follows:
[0039] , r(.) is the immediate reward function, V final (.) is the terminal value function. The value function is an overall evaluation of the process and the terminal state.
[0040] The above is only an example. The present invention does not limit how to evaluate the value function of a control input sequence, and it can be designed in any conventional way.
[0041] Finally, in order to verify the effectiveness of the model predictive control method based on the sequence generation model proposed in this paper, a simulation verification is carried out using the spacecraft trajectory optimization problem as an example. In this scenario, the main spacecraft is operating in a near-Earth circular orbit with an altitude of 1000 kilometers, and the sub-spacecraft moves near it. The path needs to be designed so that it forms a spacecraft formation with the main spacecraft. Without loss of generality, the upper limit of the small thrust amplitude is set to 7.309×10 -4 m / s 2 The main spacecraft is located at the origin of the LVLH coordinate system; the initial coordinates of the sub-spacecraft in the LVLH coordinate system are set to , the initial velocity is set to , the target point coordinates are set to , the target point speed is set to The above method is used to conduct a simulation experiment of spacecraft formation on circular orbit to show the objective and real-time performance of the proposed method in solving the problem of spacecraft trajectory planning. The simulation results are shown in Figure 2 As shown, the black dot represents the target formation position of the sub-spacecraft, the origin represents the position of the master spacecraft, and the black solid line represents the motion trajectory of the sub-spacecraft when forming a formation under the given strategy. It can be seen that the relative position and relative speed of the master and slave spacecraft will converge to zero within a finite time, which shows the effectiveness of the training strategy according to the method proposed in the present invention.
[0042] Based on the same inventive concept, the present invention also proposes a model training device, including a memory and a processor, wherein the memory stores a program, and the processor runs the program to execute:
[0043] Step 1: Provide initial value for input value ;
[0044] Step 2: Loop through the following prediction steps: Input the current value of the input value into the sequence generation model to get the output value, where the input value is recorded as input. , the output value is counted as output, , is the state of the controlled system, and its subscript represents time. is the control input of the controlled system, and its subscript represents time. To predict the number of steps, is the value function, whose subscript represents time, and then according to and And the dynamic model update of the controlled system , will update The output value after the prediction is used as the input value to perform the prediction step again, and the prediction control input sequence is obtained after the cycle ends;
[0045] Step 3: Calculate the loss function based on the predictive control input sequence and the target control input sequence, and update the sequence generation model based on the loss function;
[0046] Step 4: Loop through steps 2 and 3 to obtain the trained sequence generation model.
[0047] Based on the same inventive concept, the present invention also proposes a controller, including a memory and a processor, wherein the memory stores a program, and the processor runs the program to execute:
[0048] The predictive control steps are executed cyclically. In each predictive control step, the current value of the input value is obtained and input into the trained sequence generation model to obtain the output value. The control input sequence in the output value is used as the mean of the Gaussian distribution sequence. The Gaussian distribution sequence is sampled to obtain multiple candidate control input sequences. An alternative control input sequence is selected from the alternative control input sequence according to the value function corresponding to each alternative control input sequence. The control input sequence in the output value is replaced with the selected alternative control input sequence, and the value function in the output value is updated according to the selected alternative control input sequence. The control input corresponding to the current moment in the input value is applied to the controlled system to obtain the state value at the next moment. The state value at the next moment predicted in the output value is replaced with the state value at the next moment obtained after the controlled system is actually operated. The updated output value is used as the input value to execute the next predictive control step, wherein the input value is recorded as input. , the output value is counted as output, , is the state of the controlled system, and its subscript represents time. is the control input of the controlled system, and its subscript represents time. To predict the number of steps, is the value function, and its subscript represents time.
[0049] Based on the same inventive concept, the present invention also provides a moving body, comprising the above-mentioned controller.
[0050] The above memories include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.
[0051] The above processors are, for example, a central processing unit (CPU), a graphics processing unit (GPU), etc. The number of processors can be one or more, and they can be deployed on the same computing device or on different computing devices.
[0052] The moving body is, for example, a spacecraft, which also includes a motion control mechanism. The motion control mechanism is, for example, a propulsion device. The control input determines the thrust direction and thrust amplitude of the motion control mechanism.
[0053] The various embodiments of the present invention are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments.
[0054] The protection scope of the present invention is not limited to the above-mentioned embodiments. Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the scope and spirit of the present invention. If these changes and modifications fall within the scope of the claims of the present invention and their equivalents, the present invention is intended to include these changes and modifications.
Claims
1. A model predictive control method based on a sequence generation model for a spacecraft, characterized in that: include: Step 1: Provide initial value for input value ; Step 2: Loop through the following prediction steps: Input the current value of the input value into the sequence generation model to obtain the output value, where the input value is recorded as input. , the output value is counted as output, , is the state of the spacecraft, is the control input of the spacecraft, Set a value for the number of prediction steps, is the value function, state , control input and value function The subscript of the three t All represent time, and then according to and and updates to the spacecraft's dynamics model , will update The output value after the prediction is used as the input value to perform the prediction step again, and the prediction control input sequence is obtained after the cycle ends; Step 3: Calculate the loss function based on the predictive control input sequence and the target control input sequence, and update the sequence generation model based on the loss function; Step 4: loop through steps 2 and 3 to obtain a trained sequence generation model; Step 5, loop through the predictive control steps. In each predictive control step, obtain the current value of the input value, and input it into the trained sequence generation model to obtain the output value. Use the control input sequence in the output value as the mean of the Gaussian distribution sequence, sample the Gaussian distribution sequence to obtain multiple candidate control input sequences, select an alternative control input sequence from the alternative control input sequence according to the value function corresponding to each alternative control input sequence, replace the control input sequence in the output value with the selected alternative control input sequence, and update the value function in the output value according to the selected alternative control input sequence. Apply the control input corresponding to the current moment in the input value to the spacecraft to obtain the state value at the next moment. Replace the state value at the next moment predicted in the output value with the state value at the next moment obtained after the spacecraft actually operates. Use the updated output value as the input value to execute the next predictive control step. The value function is calculated according to the following formula: , r (.) is the immediate reward function, V final (.) is the terminal value function, h is the number of prediction steps.
2. The method according to claim 1, characterized in that The sequence generation model is a generative pre-trained model.
3. The method according to claim 1, characterized in that: The loss function is recorded as , , Input sequence for target control, Enter the sequence for predictive control.
4. The method according to claim 1, characterized in that: The control input is used to control the motion parameters of the spacecraft, and the motion parameters include at least one of velocity, acceleration, angular velocity, and angular acceleration.
5. The method according to claim 1, characterized in that The target control input sequence is generated in advance using a sampling-based model predictive control approach.
6. A model training device for a spacecraft, characterized in that: The device comprises a memory and a processor, wherein the memory stores a program and the processor runs the program to execute: Step 1: Provide initial value for input value ; Step 2: Loop through the following prediction steps: Input the current value of the input value into the sequence generation model to obtain the output value, where the input value is recorded as input. , the output value is counted as output, , is the state of the spacecraft, is the control input of the spacecraft, Set a value for the number of prediction steps, is the value function, state , control input and value function The subscript of the three t All represent time, and then according to and and updates to the spacecraft's dynamics model , will update The output value after the prediction is used as the input value to perform the prediction step again, and the prediction control input sequence is obtained after the cycle ends; Step 3: Calculate the loss function based on the predictive control input sequence and the target control input sequence, and update the sequence generation model based on the loss function; Step 4: loop through steps 2 and 3 to obtain a trained sequence generation model; The value function is calculated according to the following formula: , r (.) is the immediate reward function, V final (.) is the terminal value function, h is the number of prediction steps.
7. A spacecraft controller, characterized in that: The device comprises a memory and a processor, wherein the memory stores a program and the processor runs the program to execute: The predictive control steps are executed cyclically. In each predictive control step, the current value of the input value is obtained and input into the trained sequence generation model to obtain the output value. The control input sequence in the output value is used as the mean of the Gaussian distribution sequence. The Gaussian distribution sequence is sampled to obtain multiple candidate control input sequences. An alternative control input sequence is selected from the alternative control input sequence according to the value function corresponding to each alternative control input sequence. The control input sequence in the output value is replaced with the selected alternative control input sequence, and the value function in the output value is updated according to the selected alternative control input sequence. The control input corresponding to the current moment in the input value is applied to the spacecraft to obtain the state value at the next moment. The state value at the next moment predicted in the output value is replaced with the state value at the next moment obtained after the spacecraft is actually operated. The updated output value is used as the input value to execute the next predictive control step, wherein the input value is recorded as input. , the output value is counted as output, , is the state of the spacecraft, is the control input of the spacecraft, Set a value for the number of prediction steps, is the value function state , control input and value function The subscript of the three t Both indicate time; The value function is calculated according to the following formula: , r (.) is the immediate reward function, V final (.) is the terminal value function, h is the number of prediction steps.
8. A spacecraft, characterized in that: Comprising the controller as claimed in claim 7.
