Inverted pendulum system and control method thereof

Abstract

The present invention provides an inverted pendulum system and a control method thereof. The control method comprises providing an angle loop and a position loop, wherein the angle loop provides positive feedback, and the position loop provides negative feedback. The closed-loop feedback outputs the current PWM value to the actuator motor, which rotates according to the different PWM values. The angle loop is established to measure the current angle value of the pendulum and, based on the current angle value of the pendulum, controls the relative motion of the rotating arm to keep the pendulum upright. The position loop is established to set a desired position for the rotating arm, obtain the current position of the rotating arm via an encoder, and compensate for any deviation between the current position and the desired position. The PWM value fed back by the angle loop is balanced to keep the pendulum dynamically upright. The PWM value fed back by the position loop is balanced to keep the pendulum dynamically upright. The present invention significantly improves adjustment time and overshoot, is efficient, and has high precision, enabling stable control of the inverted pendulum.

Description

Technical Field

[0001] The invention relates to an inverted pendulum system and a control method thereof, and belongs to the technical field of mechanical stability control. Background Art

[0002] The inverted pendulum is a typical nonlinear unstable system. With numerous advantages, such as low cost and easily adjustable physical parameters, it is an ideal experimental platform for research in control theory, including variable structure control, adaptive control, and neural network control. Compared to a linear inverted pendulum system, a rotary inverted pendulum offers greater degrees of freedom, overcoming travel limitations, making it a more convenient experimental device for conducting various theoretical studies.

[0003] In view of this, it is necessary to propose an inverted pendulum system and a control method thereof to solve the above problems. Summary of the Invention

[0004] The object of the present invention is to provide an inverted pendulum system and a control method thereof, which can stably control a rotating inverted pendulum.

[0005] To achieve the above object, the present invention provides a control method for an inverted pendulum system, which mainly includes the following steps:

[0006] Step 1. Set the angle loop and position loop. The angle loop is positive feedback, and the position loop is negative feedback. The closed-loop feedback outputs the current PWM value to the actuator motor, and the motor rotates according to different PWM values.

[0007] Step 2: Establish an angle loop, measure the current angle value of the pendulum, and control the rotating arm to make relative motion according to the current angle value of the pendulum so that the pendulum remains upright;

[0008] Step 3: Establish a position loop, set the desired position for the rotating arm, obtain the current position of the rotating arm through the encoder, and compensate for the deviation between the current position of the rotating arm and the desired position;

[0009] Step 4: Balance the PWM value fed back by the angle loop to keep the pendulum upright dynamically;

[0010] Step 5: Balance the PWM value of the position loop feedback to keep the pendulum upright dynamically.

[0011] As a further improvement of the present invention, in step 1, the angle loop is updated once every 5 ms, the position loop is updated once every 25 ms, and the motor updates and feeds back a PWM value once every ms.

[0012] As a further improvement of the present invention, step 4 specifically includes the following steps:

[0013] Step 41: Set the D value of the angle ring to 0, and adjust the P value of the angle ring separately;

[0014] Step 42: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is given by the formula Get, where the response time is set to 5ms;

[0015] Step 43: Set the action space A. The action space A is a discrete space, where 0 represents a P value of -1 and 1 represents a P value of +1.

[0016] Step 44: Set the reward function R.

[0017] Step 45: Set 84,000 initial states (i) , 84,000 initial states s (i) All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0018]

[0019] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0020] Step 46: Build a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0021]

[0022] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns

[0023] Step 47: Each of The trajectory of

[0024]

[0025] Among them, β is searched by calling the Bayesian optimization method;

[0026] Step 48: In the dataset use Constructing a dataset For state s (i) For each executable action a, get the state s (i) -The value function of action a pair, that is Building a dataset

[0027]

[0028] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000;

[0029] Step 49: Through the dataset Using the multi-classification regression method, update the strategy π k .

[0030] As a further improvement of the present invention, in step 44, the reward function R is

[0031]

[0032] Here, x represents the angle change measured every 5 ms.

[0033] As a further improvement of the present invention, in step 46, each trajectory returns for

[0034]

[0035] in, is the evaluation weight θ k-1 Calculated, that is

[0036] As a further improvement of the present invention, in step 49, the strategy π k for

[0037]

[0038] Among them, penalty is the STEW regularization function.

[0039] As a further improvement of the present invention, step 4 further includes:

[0040] Step 410: Set the P value of the angle ring to 0 and adjust the D value of the angle ring separately;

[0041] Step 411: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is obtained by the formula Get, where the response time is set to 5ms;

[0042] Step 412: Set the action space A. The action space A is a discrete space, where 0 represents a D value of -1 and 1 represents a p value of +1.

[0043] Step 413: Set the reward function R.

[0044] Step 414: Reset 84,000 initial states s (i) , 84,000 initial states s (i) All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0045]

[0046] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0047] Step 415: Reconstruct a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0048]

[0049] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns

[0050] Step 416: Re-align Each of The trajectory of

[0051]

[0052] Among them, β is searched by calling the Bayesian optimization method;

[0053] Step 417: Re-add the dataset use Constructing a dataset For state s (i) For each executable action a, get the state s (i) -The value function of action a pair, that is Building a dataset

[0054]

[0055] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000,

[0056] Step 418: Through the data set Using the multi-classification regression method, update the strategy w k .

[0057] As a further improvement of the present invention, the step 5 specifically includes the following steps:

[0058] Step 51: Set the D value of the position loop to 0 and adjust the P value of the position loop separately;

[0059] Step 52: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is given by the formula Get, where the response time is set to 5ms;

[0060] Step 53: Set the action space A. The action space A is a discrete space, where 0 represents a P value of -1 and 1 represents a P value of +1.

[0061] Step 54: Set the reward function R.

[0062] Step 55: Set 84,000 initial states (i) , 84,000 initial states s (i) All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0063]

[0064] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0065] Step 56: Build a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0066]

[0067] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns

[0068] Step 57: Each of The trajectory of

[0069]

[0070] Among them, β is searched by calling the Bayesian optimization method;

[0071] Step 58: In the dataset use Constructing a dataset For state s (i) For each executable action a, get the state s (i) -The value function of action a pair, that is Building a dataset

[0072]

[0073] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000;

[0074] Step 59: Through the dataset Using the multi-classification regression method, update the strategy π k ;

[0075] Step 510: Set the P value of the position loop to 0 and adjust the D value of the position loop separately;

[0076] Step 511: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is obtained by the formula Get, where the response time is set to 5ms;

[0077] Step 512: Set the action space A. The action space A is a discrete space, where 0 represents a D value of -1 and 1 represents a p value of +1.

[0078] Step 513: Set the reward function R;

[0079] Step 514: Reset 84,000 initial states s (i) , 84,000 initial states s (i)All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0080]

[0081] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0082] Step 515: Reconstruct a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0083]

[0084] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns

[0085] Step 516: Re-align Each of The trajectory of

[0086]

[0087] Among them, β is searched by calling the Bayesian optimization method;

[0088] Step 517: Re-in the dataset use Constructing a dataset For state s (i) For each executable action a, get the state s (i) -The value function of action a pair, that is Building a dataset

[0089]

[0090] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000,

[0091] Step 518: Through the data set Using the multi-classification regression method, update the strategy w k .

[0092] To achieve the above object, the present invention further provides an inverted pendulum system, which is controlled by the control method of the inverted pendulum system described above.

[0093] As a further improvement of the present invention, it comprises a bracket, in which a motor is fixedly provided, and an output end of the motor is provided with a rotating arm and a rocker, and the rotating arm and the rocker are rotatably connected.

[0094] The beneficial effects of the present invention are: the present invention has great improvements in adjustment time, overshoot, etc., is efficient and has high precision, and can stably control the inverted pendulum. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 Schematic diagram of the structure of the inverted pendulum system of the present invention;

[0096] Figure 2 PID principle diagram of the rotating arm and the pendulum rod in the inverted pendulum system of the present invention;

[0097] Figure 3 Schematic diagram of the process of the control method of the inverted pendulum system in the present invention;

[0098] Figure 4 Schematic diagram of the process of the control method of the inverted pendulum system in the present invention;

[0099] Figure 5 This is an organizational diagram of the control method for the inverted pendulum system of the present invention. DETAILED DESCRIPTION

[0100] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0101] It should be noted here that in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions of the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0102] In addition, it should be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0103] like Figures 1 to 5 As shown, the present invention discloses an inverted pendulum system and a control method thereof, wherein the inverted pendulum system includes a bracket in which a motor is fixed, and an output end of the motor is provided with a rotating arm and a rocker rod, and the rotating arm and the rocker rod are rotatably connected.

[0104] The control method of the inverted pendulum system mainly comprises the following steps:

[0105] Step 1. Set the angle loop and position loop. The angle loop is positive feedback, and the position loop is negative feedback. The closed-loop feedback outputs the current PWM value to the actuator motor, and the motor rotates according to different PWM values.

[0106] Step 2: Establish an angle loop, measure the current angle value of the pendulum, and control the rotating arm to make relative motion according to the current angle value of the pendulum so that the pendulum remains upright;

[0107] Step 3: Establish a position loop, set the desired position for the rotating arm, obtain the current position of the rotating arm through the encoder, and compensate for the deviation between the current position of the rotating arm and the desired position;

[0108] Step 4: Balance the PWM value fed back by the angle loop to keep the pendulum upright dynamically;

[0109] Step 5: Balance the PWM value of the position loop feedback to keep the pendulum upright dynamically.

[0110] Steps 1-5 are described in detail below.

[0111] In step 1, the rotary inverted pendulum is controlled by a cascade of angle and position loops. The angle loop is positive feedback and is updated every 5 ms, while the position loop is negative feedback and is updated every 25 ms. The two loops are summed and the closed-loop feedback outputs the current PWM value. The PWM value is output to the actuator motor, which rotates according to different PWM values ​​and updates the PWM value every ms.

[0112] In step 2, the angle loop is established to keep the pendulum upright. First, the angle sensor measures the current angle value. Assume that the pendulum is in the mechanical center when it is upright, which is 0 degrees. A deviation to the left is a negative angle, and a deviation to the right is a negative angle. Our expected value is 0 degrees. When the pendulum tilts to the right, without changing the pendulum's current position, the rotating arm is moved relative to it. The rotating arm rotates to the right to compensate for the deviation and keep the pendulum upright.

[0113] In step 3, the position loop is established. Using only the PID of the angle loop can make the pendulum stand upside down, but the entire pendulum will rotate clockwise or counterclockwise as a whole, and the rotating arm will not be stable. Therefore, we need to set the rotating arm to maintain a desired position. We set it to 10000. When the motor rotates, it will drive the encoder to rotate. Through the encoder, we can know the current position of the rotating arm. To compensate for the deviation between the current position and the desired position, the motor rotates to make the rotating arm rotate to the desired position.

[0114] In step 4, in order to make the PWM values ​​of the two closed-loop PID feedbacks suitable, the pendulum is kept dynamically upright; step 4 specifically includes the following steps:

[0115] Step 41: Set the D value of the angle ring to 0, and adjust the P value of the angle ring separately;

[0116] Step 42: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is given by the formula Get, where the response time is set to 5ms;

[0117] Step 43: Set the action space A. The action space A is a discrete space, where 0 represents a P value of -1 and 1 represents a P value of +1.

[0118] Step 44: Set the reward function R.

[0119]

[0120] Where x represents the angle change measured every 5ms;

[0121] Step 45: Set 84,000 initial states (i) , 84,000 initial states s (i) All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0122]

[0123] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0124] Step 46: Build a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0125]

[0126] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns Calculated

[0127]

[0128] in, is the evaluation weight θ k-1 Calculated, that is

[0129] Step 47: Each of The trajectory of

[0130]

[0131] Among them, β is searched by calling the Bayesian optimization method;

[0132] Step 48: In the dataset use Constructing a dataset For state s (i)Each executable action a of the approximate state s (i) -The value function of action a pair, that is Thus, we can get the state s (i) The optimal action is So, build the dataset

[0133]

[0134] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000;

[0135] Step 49: Through the dataset Using the multi-classification regression method, update the strategy π k ,

[0136]

[0137] Penalty is the STEW regularization function. When the P value is adjusted correctly, the pendulum will stabilize at the median for a short period of time, then swing slightly left and right, move in one direction, and finally fall.

[0138] Step 410: Set the P value of the angle ring to 0 and adjust the D value of the angle ring separately;

[0139] Step 411: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is obtained by the formula Get, where the response time is set to 5ms;

[0140] Step 412: Set the action space A. The action space A is a discrete space, where 0 represents a D value of -1 and 1 represents a p value of +1.

[0141] Step 413: Set the reward function R.

[0142]

[0143] Where x represents the angle change measured every 5ms;

[0144] Step 414: Reset 84,000 initial states s (i) , 84,000 initial states s (i)All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0145]

[0146] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0147] Step 415: Reconstruct a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0148]

[0149] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns Calculated

[0150]

[0151] in, is the evaluation weight θ k-1 Calculated, that is

[0152] Step 416: Re-align Each of The trajectory of

[0153]

[0154] Among them, β is searched by calling the Bayesian optimization method;

[0155] Step 417: Re-add the dataset use Constructing a dataset For state s (i) Each executable action a of the approximate state s (i) -The value function of action a pair, that is Thus, we can get the state s (i) The optimal action is So, build the dataset

[0156]

[0157] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000,

[0158] Step 418: Through the data set Using the multi-classification regression method, update the strategy π k ,

[0159]

[0160] Among them, penalty is the STEW regularization function.

[0161] In step 5, the position loop is adjusted. Similar to the angle loop adjustment, the D value is first set to 0, and the P value is adjusted first. After the P value is adjusted, the D value is adjusted. Step 5 specifically includes the following steps:

[0162] Step 51: Set the D value of the position loop to 0 and adjust the P value of the position loop separately;

[0163] Step 52: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is given by the formula Get, where the response time is set to 5ms;

[0164] Step 53: Set the action space A. The action space A is a discrete space, where 0 represents a P value of -1 and 1 represents a P value of +1.

[0165] Step 54: Set the reward function R.

[0166]

[0167] Where x represents the angle change measured every 5ms;

[0168] Step 55: Set 84,000 initial states (i) , 84,000 initial states s (i) All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0169]

[0170] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0171] Step 56: Build a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0172]

[0173] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns Calculated

[0174]

[0175] in, is the evaluation weight θ k-1 Calculated, that is

[0176] Step 57: Each of The trajectory of

[0177]

[0178] Among them, β is searched by calling the Bayesian optimization method;

[0179] Step 58: In the dataset use Constructing a dataset For state s (i) Each executable action a of the approximate state s (i) -The value function of action a pair, that is Thus, we can get the state s (i) The optimal action is So, build the dataset

[0180]

[0181] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000;

[0182] Step 59: Through the dataset Using the multi-classification regression method, update the strategy w k ,

[0183]

[0184] Among them, penalty is the STEW regularization function;

[0185] Step 510: After the P value is adjusted, the swing arm can stabilize at the middle value for a short period of time, then it will swing slightly left and right, move in one direction, and finally fall;

[0186] Step 511: Set the P value of the position loop to 0, and adjust the D value of the position loop separately;

[0187] Step 512: Set the state space S, perform feature processing on the observation value s of S to obtain φ(s), where φ(s) is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the angle change rate. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the angle change rate is given by the formula Get, where the response time is set to 5ms;

[0188] Step 513: Set the action space A. The action space A is a discrete space, where 0 represents a D value of -1 and 1 represents a p value of +1.

[0189] Step 514: Set the reward function R.

[0190]

[0191] Where x represents the angle change measured every 5ms;

[0192] Step 515: Reset 84,000 initial states s (i) , 84,000 initial states s (i) All follow a distribution, and s (i) As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories:

[0193]

[0194] in, Is based on strategy π k-1 Obtained, that is yes take The new state after transfer, yes take The rewards received after

[0195] Step 516: Reconstruct a dataset containing 5000 samples right Each of the s (i) and s (i) For each executable action a, 12 steps of trajectory sampling are performed and repeated 7 times. One of the trajectories is of the form:

[0196]

[0197] Among them, when t≥1, Is based on strategy π k-1 Obtained, that is yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns Calculated

[0198]

[0199] in, is the evaluation weight θ k-1 Calculated, that is

[0200] Step 517: Re-align Each of The trajectory of

[0201]

[0202] Among them, β is searched by calling the Bayesian optimization method;

[0203] Step 518: Re-add the dataset use Constructing a dataset For state s (i) Each executable action a of the approximate state s (i) -The value function of action a pair, that is Thus, we can get the state s (i) The optimal action is So, build the dataset

[0204]

[0205] in, As the classification label, φ(s (i) ,a1),…,φ(s (i) ,a {A(s)} ) as feature input, construct 5000,

[0206] Step 519: Through the data set Using the multi-classification regression method, update the strategy w k ,

[0207]

[0208] Among them, penalty is the STEW regularization function.

[0209] In summary, the present invention has a wide range of uses in the fields of military industry, aerospace, robotics, and general industrial processes, such as balance control during robot walking, verticality control during rocket launch, and attitude control during satellite flight. It has great improvements in adjustment time, overshoot, etc., is efficient and has high precision, and can stably control an inverted pendulum.

[0210] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A control method for an inverted pendulum system, characterized in that: The main steps include: Step 1. Set the angle loop and position loop. The angle loop is positive feedback, and the position loop is negative feedback. The closed-loop feedback outputs the current PWM value to the actuator motor, and the motor rotates according to different PWM values. Step 2: Establish an angle loop, measure the current angle value of the pendulum, and control the rotating arm to make relative motion according to the current angle value of the pendulum so that the pendulum remains upright; Step 3: Establish a position loop, set the desired position for the rotating arm, obtain the current position of the rotating arm through the encoder, and compensate for the deviation between the current position of the rotating arm and the desired position; Step 4: Balance the PWM value fed back by the angle loop to keep the pendulum upright dynamically; Step 5: Balance the PWM value fed back by the position loop so that the pendulum remains dynamically upright. The step 4 specifically includes the following steps: Step 41: Set the D value of the angle ring to 0, and adjust the P value of the angle ring separately; Step 42: Set the state space S and perform feature processing on the observation value s of S to obtain ,in, It is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the rate of change of the angle. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the rate of change of the angle is obtained by the formula Get, where the response time is set to 5ms; Step 43: Set the action space A. The action space A is a discrete space, where 0 represents a P value of -1 and 1 represents a P value of +1. Step 44: Set the reward function R. Step 45: Set 84,000 initial states , 84,000 initial states All follow a distribution, As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories: ， in, It is based on strategy Obtained, that is , yes take The new state after transfer, yes take The rewards received after Step 46: Build a dataset containing 5000 samples ;right Each of and Every executable action , all samples were taken for 12 steps and repeated 7 times. One of the trajectories is of the form: ， ， in, It is based on strategy Obtained, that is , yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns ; Step 47: Each of The trajectory of ， in, Search by calling the Bayesian optimization method; Step 48: In the dataset use Constructing a dataset , for the status Every executable action , get the status -action The value function of , , build a dataset ， in, As a classification label, As feature input, construct 5000; Step 49: Through the dataset , using the multi-classification regression method, update the strategy k ; The step 5 specifically includes the following steps: Step 51: Set the D value of the position loop to 0 and adjust the P value of the position loop separately; Step 52: Set the state space S and perform feature processing on the observation value s of S to obtain ,in, It is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the rate of change of the angle. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the rate of change of the angle is obtained by the formula Get, where the response time is set to 5ms; Step 53: Set the action space A. The action space A is a discrete space, where 0 represents a P value of -1 and 1 represents a P value of +1. Step 54: Set the reward function R. Step 55: Set 84,000 initial states , 84,000 initial states All follow a distribution, As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories: ， in, It is based on strategy Obtained, that is , yes take The new state after transfer, yes take The rewards received after Step 56: Build a dataset containing 5000 samples ;right Each of and Every executable action , all samples were taken for 12 steps and repeated 7 times. One of the trajectories is of the form: ， ， in, It is based on strategy Obtained, that is , yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns ; Step 57: Each of The trajectory of ， in, Search by calling the Bayesian optimization method; Step 58: In the dataset use Constructing a dataset , for the status Every executable action , get the status -action The value function of , , build a dataset ， in, As a classification label, As feature input, construct 5000; Step 59: Through the dataset , using the multi-classification regression method, update the strategy k ; Step 510: Set the P value of the position loop to 0 and adjust the D value of the position loop separately; Step 511: Set the state space S and perform feature processing on the observation value s of S to obtain ,in, It is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the rate of change of the angle. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the rate of change of the angle is obtained by the formula Get, where the response time is set to 5ms; Step 512: Set the action space A. The action space A is a discrete space, where 0 represents a D value of -1 and 1 represents a p value of +1. Step 513: Set the reward function R; Step 514: Reset 84,000 initial states , 84,000 initial states All follow a distribution, As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories: ， in, It is based on strategy Obtained, that is , yes take The new state after transfer, yes take The rewards received after Step 515: Reconstruct a dataset containing 5000 samples ;right Each of and Every executable action , all samples were taken for 12 steps and repeated 7 times. One of the trajectories is of the form: ， ， in, It is based on strategy Obtained, that is , yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns ; Step 516: Re-align Each of The trajectory of ， in, Search by calling the Bayesian optimization method; Step 517: Re-in the dataset use Constructing a dataset , for the status Every executable action , get the status -action The value function of , , build a dataset ， in, As a classification label, As feature input, construct 5000, Step 518: Through the data set , using the multi-classification regression method, update the strategy k .

2. The control method of the inverted pendulum system according to claim 1, wherein: In step 1, the angle loop is updated every 5 ms, the position loop is updated every 25 ms, and the motor updates and feeds back the PWM value every ms.

3. The control method of the inverted pendulum system according to claim 1, wherein: In step 44, the reward function R is ， Here, x represents the angle change measured every 5 ms.

4. The control method of the inverted pendulum system according to claim 3, wherein: In step 46, each trajectory returns for ， in, The evaluation weight Calculated, that is .

5. The control method of the inverted pendulum system according to claim 4, characterized in that: In step 49, the strategy k for ， in, is the STEW regularization function.

6. The control method of the inverted pendulum system according to claim 1, characterized in that: The step 4 further comprises: Step 410: Set the P value of the angle ring to 0 and adjust the D value of the angle ring separately; Step 411: Set the state space S and perform feature processing on the observation value s of S to obtain ,in, It is a 3D feature vector, including the relative position of the disk, the angle of the vertical rod, and the rate of change of the angle. The relative position of the disk is measured by the encoder, the angle of the vertical rod is measured by the angle sensor, and the rate of change of the angle is obtained by the formula Get, where the response time is set to 5ms; Step 412: Set the action space A. The action space A is a discrete space, where 0 represents a D value of -1 and 1 represents a p value of +1. Step 413: Set the reward function R. Step 414: Reset 84,000 initial states , 84,000 initial states All follow a distribution, As the starting point, 7 steps of trajectory sampling are performed, resulting in 84,000 trajectories: ， in, It is based on strategy Obtained, that is , yes take The new state after transfer, yes take The rewards received after Step 415: Reconstruct a dataset containing 5000 samples ;right Each of and Every executable action , all samples were taken for 12 steps and repeated 7 times. One of the trajectories is of the form: ， ， in, It is based on strategy Obtained, that is , yes take The state to which it is transferred later, yes take The reward obtained after each trajectory returns ; Step 416: Re-align Each of The trajectory of ， in, Search by calling the Bayesian optimization method; Step 417: Re-add the dataset use Constructing a dataset , for the status Every executable action , get the status -action The value function of , , build a dataset ， in, As a classification label, As feature input, construct 5000, Step 418: Through the data set , using the multi-classification regression method, update the strategy k .

7. An inverted pendulum system, characterized in that: The inverted pendulum system is controlled by applying the control method of any one of claims 1 to 6.

8. The inverted pendulum system according to claim 7, wherein: The invention comprises a bracket, wherein a motor is fixedly arranged in the bracket, a rotating arm and a rocking rod are arranged at the output end of the motor, and the rotating arm and the rocking rod are rotatably connected.

Images