Multi-mode attitude control method for two-wheeled mobile platform

Abstract

The invention discloses a multi-mode attitude control method for a two-wheeled mobile platform, and the method comprises the steps: driving a real vehicle through a prior controller, collecting data, and constructing a real vehicle dynamics data set; fitting platform dynamics by adopting a radial basis function neural network, and constructing a data driving model; building a reinforcement learning agent proxy, playing back a control track, minimizing a behavior difference between a proxy strategy and a prior controller, and completing behavior simulation; constructing a simulation environment by using a data-driven model, training an agent proxy by adopting a near-end strategy optimization algorithm, and searching an optimal control strategy through environment interaction iteration; and accessing the real vehicle to the optimal control strategy, operating the agent model and collecting data, and dynamically updating the data driving model to complete physical environment alignment. According to the method, a platform dynamics model capable of being dynamically updated is constructed in a data driving mode, multi-mode attitude control is completed by applying a reinforcement learning PPO algorithm and combining priori knowledge and high-dimensional features, and self-adaptive control can be rapidly and stably achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of robot control, and particularly relates to a two-wheel motor platform multi-mode posture control method. BACKGROUND

[0002] The two-wheel motor platform, also known as a two-wheel balance car, is initially widely used in short-distance transportation. The core control mechanism thereof depends on the fusion sensing of an electronic gyroscope and an accelerometer, acquires vehicle posture information in real time, adjusts the rotation speed of the two wheels by combining a dynamic control algorithm, so as to realize longitudinal dynamic balance control. At the same time, the rotation speed difference of the two wheels is adjusted by a differential driving strategy, so as to realize accurate control of the heading angle. Compared with a traditional multi-wheel mobile platform, the configuration has excellent maneuverability and spatial adaptability in a narrow space due to the compact mechanical structure and high degree of freedom of motion.

[0003] However, the configuration has several inherent limitations. First, the motion stability thereof is highly dependent on the road conditions, and the adaptability to complex terrain is weak, so that the balance is easily lost due to ground friction force fluctuation or local deformation. Second, as a typical static unstable system, the nonlinear dynamic characteristics thereof determine that when the external disturbance exceeds the stable threshold of the controller, the system will inevitably lose stability. This characteristic puts higher requirements on the design of the controller, especially in real-time performance, robustness and anti-interference ability, so as to ensure the stability and reliability of the system in a dynamic environment.

[0004] The most commonly used controllers in the balance control of the two-wheel motor platform include PID, fuzzy PID, LQR, MPC and the like. The existing control method based on a model method needs to model the controlled system as accurately as possible, and it is difficult to apply when the controlled object structure is complex or contains unknown links. If a neural network is used to model the dynamics of the system, there is a problem of difficulty in guaranteeing ergodicity. In addition, the data-driven algorithm represented by PID does not require a system model, but engineers need to perform parameter tuning, especially when multiple PID controllers are combined, there are too many corresponding parameters to determine the optimal value, and the precision is insufficient in nonlinear system control. The reinforcement learning algorithm based on data-driven needs the agent to interact with the environment to obtain data and rewards, so as to gradually iterate an excellent control result. However, directly allowing the agent to learn in the actual physical environment brings high cost and safety risk, and training in a simulation environment inevitably needs to understand the real model of the system. Compared with supervised learning, reinforcement learning has the problems of training difficulty and difficulty in convergence, which undoubtedly affects the landing difficulty of the algorithm. SUMMARY

[0005] To solve the above technical problems, the application provides a two-wheel mobile platform multi-mode attitude control method, for the problem that the system model is difficult to accurately model or the system model is time-varying, a neural network is used for fitting, and a dynamic updating mechanism is realized to ensure that the model continuously and accurately tracks the physical characteristics of the real vehicle. In view of the slow convergence and stability problem of the reinforcement learning algorithm, the method of first behavior imitation and then autonomous learning and providing multiple high-dimensional features is adopted to integrate prior knowledge, accelerate system training and enhance stability. For different working modes, the SVM method is used for identification, which further enhances the stability of the system. The application does not need to model the mechanism of the mobile platform, adopts a data-driven method to build a dynamically updated platform dynamics model, and uses the reinforcement learning PPO algorithm, combined with prior knowledge and high-dimensional features, to complete multi-mode attitude control, which can quickly and stably realize adaptive control.

[0006] To achieve the above purpose, the application provides a two-wheel mobile platform multi-mode attitude control method, comprising: The real vehicle is driven by the prior controller, and data is collected to build a real vehicle dynamics dataset; based on the real vehicle dynamics dataset, a radial basis function neural network is used to fit the platform dynamics to build a data-driven model; An intelligent agent proxy of reinforcement learning is built, a control trajectory is played back, the difference between the agent strategy and the behavior of the prior controller is minimized, and behavior imitation is completed; An analog environment is built using the data-driven model, the intelligent agent proxy is trained using the proximal policy optimization algorithm, and the optimal control strategy is found through environment interaction iteration; The real vehicle is connected to the optimal control strategy, the agent model is run and data is collected, and the data-driven model is dynamically updated, so that the dynamics of the data-driven model and the real vehicle are gradually aligned, and the reinforcement control algorithm can be applied to real vehicle control.

[0007] Optionally, building a real vehicle dynamics dataset comprises: The system state space is divided into dimensions and combined into clusters, and a queue with the maximum length is set in each cluster to save data in chronological order; When new data is added, the state information of the new data is indexed to the corresponding cluster and pressed into the queue, and if the queue is full, the oldest data is popped out.

[0008] Optionally, the process of behavior imitation comprises: The trajectory given by the prior controller is detected from the real vehicle dynamics dataset; the KL divergence between the output strategy of the agent and the prior strategy is used as a loss function for training; and when the loss value is less than a set value, the imitation is stopped.

[0009] Optionally, building a data-driven model comprises: The input data is standardized, the number of center points and the width of the radial basis function network are determined using an empirical formula, a mean square error loss function including a regularization term is used to train the network, and a data-driven model is constructed.

[0010] Optionally, the dynamic updating process of the data-driven model comprises: calculating a weighted relative increment of the newly collected data and the existing data cluster; if the weighted relative increment does not exceed a set threshold, updating the corresponding data cluster with the newly collected data and fine-tuning the model; if the weighted relative increment exceeds the threshold, ignoring the newly collected data.

[0011] Optionally, the observation space of the reinforcement learning intelligent agent agent includes: current working point dynamics information provided by the data-driven model, current vehicle state, control target fixed-length sequence, two-side tire slip ratio and current working mode.

[0012] Optionally, the acquisition process of the current working mode comprises: The feature vector is composed of the current value and the differential value of the vehicle body attitude and the control amount, and is input to a support vector machine classifier using a linear kernel function and a soft interval for pattern recognition; The classification result is filtered, the last high confidence result is kept when the confidence is low, and the final output is one-hot encoded mode information.

[0013] Optionally, the process of training using the proximal policy optimization algorithm comprises: The advantage function is calculated using the generalized advantage estimation; based on the probability ratio and the advantage value of the new policy and the old policy, the policy update range is limited using the clipping mechanism, the Actor network is optimized through gradient descent; the value function estimate of the Critic network is optimized by minimizing the mean square error loss.

[0014] The application discloses a two-wheeled motor platform multi-mode attitude control method, working modes of the two-wheeled motor platform are distinguished, so that a controller makes differential optimization according to system dynamics characteristics and control targets in different stages. Through the combination of data-driven modeling and reinforcement learning, the adaptability and stability of the two-wheeled balance vehicle attitude control are significantly improved. A function fitter based on a RBF neural network is adopted, a platform dynamics model is dynamically updated through real vehicle data-driven modeling. Compared with a traditional method relying on explicit mechanism modeling, the scheme does not need to accurately model nonlinear characteristics of a complex system. When the system faces slow time-varying characteristics, the neural network can dynamically correct model errors through continuous data collection and parameter adjustment. This mechanism solves the problem of accuracy reduction caused by parameter drift of the traditional model. In addition, the application uses real vehicle trajectory data of a prior controller as initial training samples of a reinforcement learning agent through behavior strategy imitation technology, and effectively solves the problem of slow convergence speed of the traditional reinforcement learning algorithm. According to the difference of dynamic characteristics of the two-wheeled vehicle in different working modes, the application introduces a support vector machine (SVM) for mode recognition, as high-dimensional feature embedding observation space, to improve the system's qualitative. In summary, through the cooperative optimization of data-driven modeling and reinforcement learning, the two-wheeled balance vehicle attitude control realizes the self-adaptive, high-precision and low-power consumption targets, and provides a general solution for dynamic system control in the fields of automatic driving, robots and the like. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application and are incorporated herein for a purpose of explanations and are not intended as an improper limitation to the present application. In the drawings: Figure 1 It is a working mode observer schematic diagram of the embodiment of the application. Figure 2 It is a PPO reinforcement learning process schematic diagram of the embodiment of the application. Figure 3 It is a process schematic diagram of a two-wheeled motor platform multi-mode attitude control method of the embodiment of the application. DETAILED DESCRIPTION

[0016] It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0017] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0018] AsFigure 3 As shown, the embodiment provides a two-wheeled motor platform multi-mode attitude control method, comprising: The prior controller drives the real vehicle and collects data to construct a real vehicle dynamics dataset; based on the real vehicle dynamics dataset, a radial basis function neural network is used to fit the platform dynamics to construct a data-driven model; An intelligent agent proxy of reinforcement learning is built, the control trajectory is played back, the difference between the proxy strategy and the prior controller behavior is minimized, and behavior imitation is completed; An analog environment is constructed using the data-driven model, the intelligent agent proxy is trained using a proximal policy optimization algorithm, and the optimal control strategy is iteratively found through environment interaction; The real vehicle is connected to the optimal control strategy, the proxy model is run and data is collected, and the data-driven model is dynamically updated, so that the dynamics and physical characteristics of the data-driven model and the real vehicle are gradually aligned, and the reinforcement control algorithm can be applied to real vehicle control.

[0019] Further, constructing a real vehicle dynamics dataset comprises: The state space of the system is divided into dimensions and combined into clusters, and a queue with the maximum length is set in each cluster to save data in chronological order; When new data is added, the state information of the new data is indexed to the corresponding cluster and pushed into the queue, and if the queue is full, the oldest data is popped out.

[0020] Constructing a data-driven model comprises: The input data is standardized; the number of center points and the width of the radial basis function network are determined using empirical formulas; a mean square error loss function containing a regularization term is used to train the network to construct the data-driven model.

[0021] The dynamic updating process of the data-driven model comprises: calculating the weighted relative increment of the newly collected data and the existing data cluster; if the weighted relative increment does not exceed a set threshold, updating the corresponding data cluster with the newly collected data and fine-tuning the model; if the weighted relative increment exceeds the threshold, the newly collected data is ignored.

[0022] Specifically, an implementation of the embodiment is: S1 data-driven model: S1.1 data-driven model based on neural network fitting: The two-wheeled motor platform controls the attitude of the vehicle body by controlling the torque and speed of the wheels on both sides. Since the motor operating characteristics are basically stable and the vehicle body structure does not change much over time, the next attitude of the system is only related to the previous attitude and control, basically satisfying the Markov property and having a local linearization condition, so its dynamics model can be written as follows: ; where x is the system state vector, u is the system control variable, and θ is the function parameter, which is only related to the dynamic characteristics of the system and does not change dramatically over time. The data-driven model in this embodiment uses x, which includes chassis speed, two-wheel speed, three-axis angle, three-axis angular velocity, and three-axis acceleration; and u, which is the current size of the two-side drive motor: ; ; where, is the velocity vector in the body coordinate system; and are the measured values of the left and right wheel speed meters; is the three-axis angle in the body coordinate system; is the three-axis angular velocity in the body coordinate system; is the three-axis acceleration in the body coordinate system; and are the control currents on both sides.

[0023] The radial basis function network (RBF) has significant advantages over the back propagation neural network (BP) in system dynamics modeling: its local approximation feature can accurately capture nonlinear dynamic behavior, and the design of the preset hidden layer parameters and the linear output layer greatly improves the training speed and convergence efficiency; at the same time, RBF has global approximation ability, which can model complex system characteristics with high precision by reasonably configuring the center point and width parameters, and has stronger robustness to noise and disturbance. Its parameter separation learning mechanism and low computational complexity make it more suitable for real-time control and adaptive scenarios (such as motor speed regulation and satellite attitude control), while the BP network has weaknesses in dynamic system modeling due to its global response characteristics, dependence on iterative optimization, and tendency to fall into local minimum. For the above reasons, RBF is used as the actual modeling method, and the Gaussian function is used as the radial function, ; where, is the mean; is the variance; is the actual state variable; is the Gaussian probability distribution.

[0024] First, the prior controller drives the real vehicle and synchronously collects data to form a real vehicle dynamics dataset. Then, the input data is standardized to eliminate dimensional differences and speed up the training. The number of center points and width is determined using empirical formulas. Finally, the mean squared error is used to evaluate the best fit, and a regularization term is added to avoid overfitting, and the final loss function is as follows: ; where, is the integrated system dynamics change for the state vector, control vector and system dynamics within m time steps; is the number of time steps considered for fitting; is the network parameter; is the state vector; is the control vector; is the state vector change prediction given by the data-driven model; is the actual state vector change; is the L2 regularization term weight; is the number of parameters; is the jth model parameter.

[0025] S1.2 Data distribution requirement: The data-driven model is a supervised learning process, and its fitting effect is closely related to the data quality. The data used for training has the characteristics of independent and identical distribution and ergodicity. In this problem, it is described that the distribution of the data should be as uniform as possible at different working states of the system; at the same time, as much sample data as possible should be supported near each working point.

[0026] To achieve the above two requirements, the storage structure of the data set needs to be designed. The data range is divided in each dimension of the system state space, and then combined. Each combination forms a cluster, and a queue with the maximum length is saved in the cluster. The internal elements are arranged in chronological order. When new data is added, it is indexed to the cluster according to its state information, and then pushed into the queue. If the queue is full, the old information is popped out.

[0027] Each data cluster actually describes the working state of the system in a unit cell in the state space. By limiting the queue length, the distribution of the data is adjusted; and based on the first-in first-out, the data is added with time effectiveness.

[0028] S1.3 Dynamic tracking of data-driven model: Although the structure of the two-wheel maneuvering platform will not change dramatically, the system characteristics will still gradually deviate from the pre-set model over a long period of time. To solve this problem, new collected data can be updated to the model gradually during operation. Through the specially designed data set storage structure, the old data in the data set is replaced with new running data, and the fitting model is fine-tuned to realize the dynamic tracking of the data-driven model. To avoid the excessive influence of abnormal noise on the model, the newly collected running data must be updated when there is a certain similarity between the existing data and the data near the data cluster. In order to describe the size of this similarity, the weighted relative increment is proposed, which is mathematically expressed as follows: ; Wherein, the superscript t represents the current value, t+1 represents the new observation value, The state vector change prediction given by the data-driven model, the Q matrix is the importance weight matrix of different components of the state and control vector, and describes the importance of each dimension to the similarity description. When If it exceeds a certain threshold, it is considered abnormal noise, and the data collected this time is ignored, and the data update is skipped.

[0029] Further, the observation space of the reinforcement learning intelligent agent agent includes: the current working point dynamics information provided by the data-driven model, the current state of the vehicle, the control target fixed-length sequence, the slip rate of the two sides of the tire and the current working mode.

[0030] The acquisition process of the current working mode includes: The current values and differential values of the vehicle body posture and control quantity form a feature vector, which is input to a support vector machine classifier using a linear kernel function and a soft interval for pattern recognition; The classification result is filtered, and when the confidence is low, the last high confidence result is maintained, and the final output is one-hot encoded mode information.

[0031] The process of behavior imitation includes: The trajectory given by the prior controller is detected from the real vehicle dynamics data set; the KL divergence between the intelligent agent output strategy and the prior strategy is used as the loss function for training; when the loss value is less than the set value, the imitation is stopped.

[0032] Specifically, an implementation process of the embodiment is: S2 behavior strategy imitation: S2.1 intelligent agent model: Its observation space includes the dynamics characteristic information of the current working point given by the data-driven model, the state of the current vehicle, the fixed-length sequence of the control target, the slip rate of the two sides of the tire and the current working mode. Its action space is the direct control quantity of the two sides of the motor. By artificially introducing high-dimensional features, the convergence of the intelligent agent can be accelerated. Thus the appearance of the intelligent agent agent can be abstracted as: ; Wherein, the input parameters of the actor are the current state quantity, the predicted system dynamic response, the reference state, the tire slip rate and the current working mode. U is the recommended control quantity given by the agent.

[0033] Since LSTM is better than multilayer perceptron in capturing the time sequence information of the system, it is easier to make correct sequential decisions, and the embodiment adopts LSTM as the implementation method of the agent.

[0034] S2.1.1 Working point dynamics information: The pseudo-gradient of the vehicle body attitude with respect to the control variable at the current working point is calculated by difference calculation to provide the controller with a reference about the dynamics information. The calculation is based on a data-driven model, and the sampling points within the control dimension domain of the current working point are obtained by sampling, and the corresponding state change is estimated by the model to obtain: ; In the formula, the length of the control variable is taken as an example, represents a small amount, is the state vector change prediction given by the dynamics model.

[0035] S2.1.2 Reference sequence: The two-wheel mobile platform has under-actuated characteristics, and some state points cannot be reached, and some state points can only be reached according to a certain trajectory. Therefore, unlike traditional controllers, the control reference is a single value of the target state, and the control reference in this embodiment is a fixed-length sequence. The current expected target state and the target state in the subsequent multiple time steps are spliced into a control sequence and input into the controller, so that it is possible for the controller to consider the current optimal and long-term optimal when planning and decision-making, which is more conducive to the under-actuated system to reach the target state; Reference sequence: ; Among them, represents the reference state quantity at the corresponding time.

[0036] S2.1.3 Working mode: There are multiple working modes in the operation of the mobile platform, such as the start-up state from static to start-up, the self-stable state to maintain the attitude of the platform, the contact state when the platform contacts with the environment or other platforms, and the stop state from balance to dump shutdown. Different working modes naturally require the controller to adopt different strategies. Therefore, by embedding an observation value of a working mode into the observation space of the controller, it will be beneficial for the controller to make correct decisions.

[0037] In this embodiment, SVM support vector machine is used to complete the recognition of the working mode. According to prior knowledge, it is considered that the distinction in some stages is not only related to the current attitude of the vehicle body, but also related to the trend of attitude change. Therefore, the feature vector is selected as the current value and the differential value of the vehicle body attitude and the control variable, that is: ; A linear kernel function is used for dimensionality transformation, and a soft margin is employed to improve the classifier's tolerance to noise signals. A One vs. Reset mode is used to train different classifiers for different modes, enabling multi-class classification under various operating conditions. To avoid excessively frequent changes in modal information and introduce too many unstable factors to the controller, the modal information output by the classifier needs to be filtered. When the confidence level of the result given by the classifier is low, i.e., falls within the soft margin, the modal information is not updated but retains the most recent and most reliable result. Figure 1 The workflow of the classifier system shown is as follows: the final output working mode information is one-hot encoded to form a feature vector.

[0038] S2.1.4 Tire slip ratio observation: Tire slip ratio is an indicator that measures the degree of relative slippage between the tire and the road surface during braking or acceleration. Experience shows that when the slip ratio is around 20%, the longitudinal adhesion coefficient between the tire and the road surface is the highest, resulting in the best braking effect. If the slip ratio is too high, the wheels will lock up, the braking distance will increase, and the vehicle will be prone to loss of control; if the slip ratio is too low, although directional stability is good, the braking force is insufficient, and the braking distance will also be longer.

[0039] Similarly, tire slip ratio describes a similar behavior of a vehicle during acceleration. For ease of subsequent analysis, we will no longer distinguish between slip and rotation, and will denote its absolute value as the slip ratio s: ; in, v is the wheel speed, and v is the vehicle speed.

[0040] When determining the strategy for the controller, the slip ratio should be fully considered to avoid tire slippage or idling. This is beneficial to the stability of the control system and also helps reduce system energy consumption. Therefore, it should be observed for subsequent high-dimensional feature embedding or the establishment of an evaluation system.

[0041] S2.2 mimics the behavior of the prior controller: The trajectory given by the prior controller is retrieved from the dataset, and the convergence speed of the agent is accelerated by mimicking behavioral patterns. In this process, the loss function is the difference between the controller's output policy and the agent's estimated policy, described using KL divergence, and its mathematical expression is: ; in, The action taken when t is... The system state at time t. Let be the probability of taking action a in state s. This represents the number of entries recorded in the database under state 's'.

[0042] Since the prior controller is not the optimal controller, the agent does not need to closely simulate the behavior of the prior controller. The behavior simulation can be stopped when the loss value is less than a certain value.

[0043] Furthermore, the training process using the proximal strategy optimization algorithm includes: The advantage function is calculated using generalized advantage estimation; based on the probability ratio of the new policy to the old policy and the advantage value, a pruning mechanism is used to limit the policy update magnitude, and the Actor network is optimized through gradient descent; the value function estimation of the Critic network is optimized by minimizing the mean squared error loss.

[0044] Specifically, one implementation process of this embodiment is as follows: like Figure 2 The S3 simulation space optimization is shown below: Once the agent has initially learned the control policy of the prior controller, it can begin to explore better solutions autonomously using reinforcement learning algorithms. Policy Proximity Optimization (PPO) is an efficient and stable reinforcement learning algorithm. It combines the ideas of policy gradient methods and trust region optimization, ensuring stability and sample efficiency during training by limiting the magnitude of policy updates. PPO performs exceptionally well in complex tasks involving continuous action spaces and high-dimensional state spaces, and is widely used in robot control, game AI, and autonomous driving.

[0045] State is generated through interaction with the current strategy and the environment. -action -award -Next state The trajectory is recorded, and the action probability under the old strategy is also recorded. The advantage function at each time step is calculated using generalized advantage estimation (GAE). This measures the value of an action relative to the average policy. Based on the new policy... Compared to the old strategy The probability ratios and advantage values ​​are used to limit the policy update magnitude through a pruning mechanism, and the loss is reduced through gradient descent. The value function is optimized by minimizing the mean squared error loss. Estimate. Repeat the above process to gradually converge to a stable policy: The temporal difference (TD) error represents the difference between the current state value estimate and the reward and state value at the next time step. Its mathematical expression is: ; ; in, Discount factor; These are GAE parameters that adjust the dynamic balance between bias and variance.

[0046] The GAE is used for advantage estimation, and the calculation formula is: ; wherein, is the discount factor; is the GAE parameter, is the TD error.

[0047] The loss function of the actor network is as follows: ; wherein, is the value function, and clip is the clipping function, is an adjustable value, is the advantage value calculated by the generalized advantage estimation at each time step.

[0048] The optimization goal of the critic network is: ; wherein, is the mathematical expectation when s follows the policy ; is the state; is the value function with as the parameter; is the reward function.

[0049] The definition of the reward function mainly reflects the accurate tracking of the current reference by the vehicle body posture and the proximity to the subsequent reference; lower slip rate, maximum utilization of tire grip; lower energy consumption; more smooth control: ; wherein, is the reference prediction step number; is the reference state; is the actual state; , , are three weight adjustment parameters; is the slip rate; is the control amount square term; is the absolute value of the control amount change.

[0050] Further, the S4 physical environment is aligned: The real vehicle is connected to the system, the agent model trained in S3 is run, the controller output is applied to the real vehicle, and data is collected synchronously. According to the dynamic tracking method of the data-driven model agreed in S1.3, synchronous data updating iteration and model optimization are carried out. On this basis, the training cycle of reinforcement learning is continued to be executed, so that the agent can learn the difference between the data set and the actual physical system, complete the landing of sim2real, and continuously track the dynamic performance changes of the system.

[0051] The application discloses a two-wheeled motor platform multi-mode attitude control method, which distinguishes the working modes of the two-wheeled motor platform, so that the controller makes differential optimization according to the system dynamics characteristics and control targets in different stages. Through the combination of data-driven modeling and reinforcement learning, the adaptability and stability of the two-wheeled balance vehicle attitude control are significantly improved. A function fitter based on RBF neural network is adopted to dynamically update the platform dynamics model through real vehicle data-driven modeling. Compared with the traditional method relying on explicit mechanism modeling, the scheme does not need to accurately model the nonlinear characteristics of the complex system. When the system faces slow time-varying characteristics, the neural network can dynamically correct the model error through continuous data collection and parameter adjustment. This mechanism solves the problem of accuracy reduction caused by parameter drift of the traditional model. In addition, the application uses the real vehicle trajectory data of the prior controller as the initial training sample of the reinforcement learning agent through the behavior strategy imitation technology, effectively solving the slow convergence problem of the traditional reinforcement learning algorithm. According to the dynamic characteristic difference of the two-wheeled vehicle in different working modes, the application introduces support vector machine SVM for mode recognition, as high-dimensional feature embedding observation space, to improve the system's qualitative. In summary, through the cooperative optimization of data-driven modeling and reinforcement learning, the application realizes the adaptive, high-precision and low-power consumption of the two-wheeled balance vehicle attitude control, and provides a general solution for dynamic system control in the fields of automatic driving, robots and the like.

[0052] The above is only the preferred specific embodiment of the application, but the protection scope of the application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A multi-mode attitude control method for a two-wheeled motorized platform, characterized in that, The method comprises the following steps: Driving the real vehicle through a prior controller and collecting data to build a real vehicle dynamics dataset; Based on the real vehicle dynamics dataset, fitting platform dynamics by using a radial basis function neural network to build a data-driven model; Building a reinforcement learning agent proxy, replaying the control trajectory, minimizing the difference between the agent strategy and the prior controller behavior, and completing behavior imitation; Using the data-driven model to build a simulation environment, training the agent proxy by using a proximal policy optimization algorithm, and finding the optimal control strategy through environment interaction iteration; Connecting the real vehicle to the optimal control strategy, running the agent model and collecting data, and dynamically updating the data-driven model to gradually align the dynamics and physical characteristics of the data-driven model and the real vehicle, so that the reinforcement control algorithm can be applied to real vehicle control.

2. The multi-mode attitude control method of the two-wheeled motor platform according to claim 1, wherein building a real vehicle dynamics dataset comprises: Dividing the system state space into clusters by dimension and combining them, and setting a queue with the maximum length in each cluster to save data in chronological order; When new data is added, indexing the new data to the corresponding cluster according to the state information of the new data and pressing it into the queue, and if the queue is full, popping out the oldest data.

3. The multi-mode attitude control method of the two-wheeled motor platform according to claim 1, wherein the process of behavior imitation comprises: Retrieving the trajectory given by the prior controller from the real vehicle dynamics dataset; using the KL divergence between the agent output strategy and the prior strategy as the loss function for training; and stopping imitation when the loss value is less than a set value.

4. The multi-mode attitude control method of the two-wheeled motor platform according to claim 1, wherein building a data-driven model comprises: Standardizing the input data; using empirical formulas to determine the number of center points and the width of the radial basis function network; and using a mean square error loss function with a regularization term to train the network and build the data-driven model.

5. The multi-mode attitude control method of the two-wheeled motor platform according to claim 1, wherein the dynamic updating process of the data-driven model comprises: calculating the weighted relative increment of the newly collected data and the existing data cluster; if the weighted relative increment does not exceed a set threshold, updating the corresponding data cluster with the newly collected data and fine-tuning the model; and if the weighted relative increment exceeds the threshold, ignoring the newly collected data.

6. The multi-mode attitude control method of the two-wheeled motor platform according to claim 1, wherein the observation space of the reinforcement learning agent proxy comprises: the current working point dynamics information provided by the data-driven model, the current state of the vehicle, the control target fixed-length sequence, the slip rate of the two tires, and the current working mode.

7. The multi-mode attitude control method of the two-wheeled motor platform according to claim 6, wherein the process of obtaining the current working mode comprises: Using the current values and differential values of the vehicle attitude and control quantities to form a feature vector, which is input to a support vector machine classifier using a linear kernel function and soft interval for pattern recognition; Filtering the classification results, keeping the last high-confidence result when the confidence is low, and finally outputting the one-hot encoded mode information. ​ ​ ​ ​ ​ ​ 8. The multi-mode attitude control method of a two-wheeled motor platform according to claim 1, characterized in that, The process of training using the proximal policy optimization algorithm comprises: The advantage function is calculated using the generalized advantage estimation; based on the probability ratio and the advantage value of the new policy and the old policy, the policy update range is limited using the clipping mechanism, and the Actor network is optimized through gradient descent; the value function estimation of the Critic network is optimized by minimizing the mean square error loss.

Images