Method for constructing a robot controller and robot
By collecting data while the wheel-leg robot is in a equilibrium state and calculating the linear equilibrium parameter matrix using numerical iterative methods, a dynamic characteristic controller is constructed, which solves the motion control problem under unknown dynamic characteristics and achieves better balance control Effects and system stability.
Patent Information
- Application Number
- CN202111063979.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-10
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-09-10
AI Technical Summary
The prior art is difficult to achieve flexible motion control under unknown robot dynamic characteristics, especially wheel-leg robots have challenges in balancing control.
By collecting motion state data and control data when the robot is in a equilibrium state, numerical iterative method is used to calculate the linear equilibrium parameter matrix, and a corresponding dynamic characteristic controller is constructed to achieve better motion control effect.
In the case of unknown dynamic models, the robot can be effectively controlled, so that it can be closer to the balance point during movement, improve the control effect and system stability, and simplify the controller design process.
Smart Images

Figure CN115781658B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence and robots, and more particularly to a method for constructing a robot controller, a robot, a non-transitory computer-readable medium, and a method for controlling a robot. Background Art
[0002] With the widespread application of artificial intelligence and robotics technology in civil and commercial fields, robots based on artificial intelligence and robotics technology are playing an increasingly important role in smart transportation, smart home and other fields, and are also facing higher requirements.
[0003] At present, when controlling the motion of robots, especially underactuated robots, it is usually necessary to design a dynamic model that accurately corresponds to the mechanical structure of the robot, and then determine the control force at each joint of the robot based on the changes in the dynamic model during the movement process to ensure the balance of the robot during the movement process. However, due to the complex mechanical structure of the robot, especially some wheel-legged robots, it is difficult to derive an accurate dynamic model even if the mechanical structure of the robot is known. In addition, even if the dynamic model is known, it is difficult to accurately identify the parameters in the dynamic model in some cases. If the dynamic model is known but the parameters are inaccurate, the controller effect of the robot will also be unsatisfactory. For this reason, it is necessary to propose a solution that can flexibly control the robot under the unknown dynamic characteristics of the robot. Summary of the invention
[0004] In view of the above problems, the present disclosure provides a method for constructing a robot controller, a robot, a non-transitory computer-readable medium, and a method for controlling a robot. The method for constructing a robot controller provided by the present disclosure can enable the robot to be flexibly controlled without an accurate dynamic model.
[0005] According to one aspect of the present disclosure, a method for constructing a controller for a robot is proposed, the method comprising: controlling a robot with unknown dynamic characteristics to move in a quasi-equilibrium state to obtain motion state data and the control data during the movement, wherein the robot in the quasi-equilibrium state is near a balance point during the movement; calculating a linear balance parameter matrix using a numerical iteration method based on the motion state data and the control data; and constructing a controller corresponding to the dynamic characteristics of the robot based on the linear balance parameter matrix.
[0006] Optionally, the quasi-balanced motion state of the robot is controlled by a first controller, and the robot under the control of the controller corresponding to the dynamic characteristics of the robot has a better control effect during the motion process than the robot under the control of the first controller.
[0007] According to one aspect of the present disclosure, a robot is proposed, comprising: a data acquisition device, configured to: when controlling a robot with unknown dynamic characteristics to move in a quasi-equilibrium state, obtain motion state data of the robot in a quasi-equilibrium state, wherein the robot in the quasi-equilibrium state is near a balance point; a data processing device, configured to: obtain control data corresponding to the motion state data; calculate a linear equilibrium parameter matrix by numerical iteration according to the motion state data of the robot in the quasi-equilibrium state and the control data; and construct a controller corresponding to the dynamic characteristics of the robot based on the linear equilibrium parameter matrix.
[0008] According to one aspect of an embodiment of the present disclosure, there is provided a wheel-legged robot, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the above method.
[0009] According to one aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above method.
[0010] Optionally, an embodiment of the present disclosure also correspondingly discloses a method for controlling a robot using a controller. The robot includes a wheel leg portion and a base portion connected to the wheel leg portion, and the wheel leg portion includes a driving wheel and at least one joint. Specifically, the method includes: receiving a motion instruction, the motion instruction indicating the motion trajectory of the robot; according to the motion instruction, using a controller to control the driving force of the driving wheel so that the robot moves smoothly along the target trajectory. The robot under the control of the controller is closer to the balance point during the movement process than the robot under the control of the first controller.
[0011] Optionally, an embodiment of the present disclosure also correspondingly discloses a method for controlling a robot. The robot is driven by a driving wheel, and the dynamic characteristics of the robot are unknown, and the method comprises: receiving a motion instruction, the motion instruction indicating the motion trajectory of the robot; according to the motion instruction, controlling the driving force of the driving wheel so that the robot moves in a quasi-equilibrium state and obtaining motion state data and control data during the motion process, wherein the robot in the quasi-equilibrium state is near a balance point during the motion process; based on the motion state data and the control data, constructing a controller corresponding to the dynamic characteristics of the robot using a numerical iteration method; and using the controller to control the driving force of the driving wheel so that the robot moves smoothly.
[0012] The present invention provides a method for constructing a robot controller, a robot, a computer-readable storage medium, and a method for controlling a robot including driving wheels, which can enable the robot to be flexibly controlled even when the precise dynamic model of the robot is unknown. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The following drawings are not deliberately scaled according to the actual size, and the focus is on illustrating the main purpose of the present invention.
[0014] Figure 1 A schematic structural diagram of a robot having a left wheel leg portion and a right wheel leg portion of a single-wheel leg configuration according to an embodiment of the present disclosure is shown.
[0015] Figure 2 An exemplary flow chart of a method for constructing a controller of a robot according to an embodiment of the present disclosure is shown.
[0016] Figure 3 A schematic diagram of a simplified dynamic model corresponding to a robot according to a disclosed embodiment is shown.
[0017] Figure 4 A flowchart of a numerical iteration scheme according to an embodiment of the present disclosure is shown.
[0018] Figure 5 Another structural view of the robot according to the embodiment of the present disclosure is shown.
[0019] Fig. 6A The motion state data and control data of the robot in the process of using the first controller to control the robot's motion in a quasi-balanced state are shown.
[0020] Figure 6B The process of constructing a controller corresponding to the dynamic characteristics of the robot is shown in K(t f -kh) and H(t f -kh) convergence process, where the base height of the robot is 0.65 meters.
[0021] Figure 6C The process of constructing a controller corresponding to the dynamic characteristics of the robot is shown in K(t f -kh) and H(t f -kh) convergence process, where the base height of the robot is 0.38 meters.
[0022] Fig.6D The motion state data and control data of the robot using the constructed controller to control the robot to walk in a straight line when the base height is 0.38 meters.
[0023] Fig. 6E The motion state data and control data of the robot using the constructed controller to control the robot to yaw when the base height is 0.65 meters.
[0024] Fig. 6F The motion state data of the robot using the constructed controller to control the robot to move along the "S" path is shown.
[0025] Figure 6G The data shows the motion state of the robot on the speed bump controlled by the constructed controller.
[0026] Figure 6H The motion state data of the robot is shown using the constructed controller to control the robot to resist external forces. DETAILED DESCRIPTION
[0027] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present invention.
[0028] As shown in the present disclosure and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
[0029] Although the present disclosure makes various references to certain modules in the system according to embodiments of the present disclosure, any number of different modules can be used and run on a user terminal and / or server. The modules are illustrative only, and different aspects of the systems and methods can use different modules.
[0030] Flowcharts are used in this disclosure to illustrate the operations performed by the system according to the embodiments of the present disclosure. It should be understood that the preceding or following operations are not necessarily performed precisely in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.
[0031] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0032] The disclosed technical solution mainly relates to the robot technology in artificial intelligence technology, mainly to the intelligent control of robots. Robots are mechanical and electronic devices that can imitate certain human skills by combining mechanical transmission and modern microelectronics technology. Robots are developed on the basis of electronic, mechanical and information technology. Robots do not necessarily have to look like humans. As long as they can independently complete the tasks and commands assigned to them by humans, they belong to the large family of robots. A robot is an automated machine that has some intelligent abilities similar to those of humans or organisms, such as perception, planning, movement and coordination. It is an automated machine with high flexibility. With the development of computer technology and artificial intelligence technology, robots have been greatly improved in terms of function and technical level. Mobile robots and robot vision and touch technologies are typical representatives.
[0033] The present disclosure relates to an application of artificial intelligence in robot control. Specifically, the present disclosure proposes a method for constructing a robot controller based on artificial intelligence, wherein the solution adds the use of an arbitrary controller to be optimized to construct a controller corresponding to the dynamic characteristics of the robot. The robot under the control of the controller corresponding to the dynamic characteristics of the robot has a motion state closer to a balance point than the robot under the control of the arbitrary controller to be optimized.
[0034] The robot involved in the present disclosure may be an underactuated robot, for example, Figure 1 The wheel-legged robot shown. Specifically, Figure 1 A schematic structural diagram of a robot having a left wheel leg portion and a right wheel leg portion of a single-wheel leg configuration according to an embodiment of the present disclosure is shown.
[0035] like Figure 1 As shown, it exemplarily shows a structural schematic diagram of a robot 100. The wheel-legged robot 100 may include: a base portion 101, a wheel-leg portion 103, and a driving wheel 104 (also called a foot) in the wheel-leg portion 103. The base portion 101 refers to the main part of the robot, for example, it may be the torso of the robot, and the base portion may be, for example, a flat plate-shaped component or a rectangular parallelepiped component connected to the wheel-leg portion of the robot. As an example, one end of the wheel-leg portion 103 is connected to the base portion 101, and the other end is connected to the driving wheel 104. A power output device (for example, a motor) is mounted on the base portion 101, which can be used to drive the joints of the wheel-leg portion 103 to provide electricity. It should be understood that, according to actual needs, the robot may also include, for example, a base portion connected to the wheel-leg portion or an additional component arranged on the base portion. It should be understood that the above only gives an example of the structure of a robot, and the embodiments of the present disclosure are not limited by the specific components of the robot and their connection methods.
[0036] The active wheel 104 in the wheel-legged part 103 can enable the wheel-legged robot 100 to both walk and perform wheeled movements. Optionally, the wheel-legged robot 100 may also include a controllable additional component (e.g., a tail), which can be used to balance the wheel-legged robot or to assist the wheel-legged robot in moving. For example, the tail can assist the wheel-legged robot in maintaining balance during movement. Optionally, the wheel-legged robot may also include a controllable mechanical arm, which can be used to perform operational tasks such as carrying and picking. The wheel-legged robot may include a bipedal wheel-legged robot, a quadrupedal wheel-legged robot, etc., which is not limited in the embodiments of the present disclosure.
[0037] For example, the wheel-legged part 103 is a parallel structure leg (the two legs of the bipedal wheel-legged robot 100 are balanced). Figure 1, the wheel leg part 102 of the robot 100 includes a left wheel leg part and a right wheel leg part, each of which includes a driving wheel and two parallel legs connected to the central axis of the driving wheel and used to realize the motion control of the driving wheel. For example, the left wheel leg part includes a left driving wheel, and a first left wheel leg part and a second left wheel leg part connected in parallel; and the right wheel leg part 112 includes a right driving wheel, and a first right wheel leg part and a second right wheel leg part connected in parallel. Figure 1 As shown, the left wheel leg and the right wheel leg have mirror-symmetrical structures.
[0038] For example, the driving wheel can be a single wheel, two wheels, four wheels or other number of driving wheel configurations, and each driving wheel can be controlled by two legs in parallel or multiple legs in series. It should be understood that the embodiments of the present disclosure are not limited by the specific composition types of the left and right wheel legs and the number of driving wheels. In some embodiments, the left wheel leg and the right wheel leg are both single-wheel leg configurations. The single-wheel leg configuration means that the wheel leg only includes a single driving wheel.
[0039] For example, the left wheel leg and the right wheel leg may include the same number of joints and have the same joint configuration, or, according to actual needs, the left wheel leg and the right wheel leg may have different numbers of joints and different joint configurations. The embodiments of the present disclosure are not limited by the specific number of joints and joint configurations of the left wheel leg and the right wheel leg. Figure 1 Taking the example shown as an example, the left and right wheel legs each contain 5 joints and have a total of 2 rotational degrees of freedom. The change in the center of mass height of the wheel leg / base and the inclination angle of the base can be adjusted by adjusting the joints of the wheel leg 103.
[0040] Compared with the serial structural legs, the parallel structural legs can have stronger rigidity and can withstand the impact that may be caused by complex movements. When in contact with the ground, the active wheel 104 can provide the wheel-legged robot 100 with the ability to slide. Optionally, the bipedal wheel-legged robot 100 can also include an additional component 102, which is connected to the base part 101. A passive wheel 105 can be installed on the additional component 102. The additional component 102 includes 1 degree of freedom of rotation. The movement of the additional component 102 will also affect the changes of the base part 101 and the wheel-leg part 103, for example, driving the base part to have a certain rotation speed. Therefore, the balance and posture of the robot 100 can be adjusted by adjusting the position of the additional component 102.
[0041] The wheel-legged robot 100 has both the flexibility of a wheeled robot and the flexibility of a legged robot, so it can move quickly on flat ground and cross rough roads. Figure 1The wheel-legged robot 100 shown in the figure has only two contact points with the ground. And the wheel-legged robot 100 is a non-minimum phase system, so it is still difficult to balance control the wheel-legged robot 100 in practical applications. In addition, due to the complex mechanical structure of the (wheel-legged) robot, it is difficult to determine the dynamic characteristics of the (wheel-legged) robot. Since the traditional balance control method needs to know the dynamic characteristics of the robot, therefore, in the case of unknown dynamic characteristics, the traditional balance control method is difficult to balance control such a robot.
[0042] The method for constructing a robot controller provided by an embodiment of the present disclosure involves designing a controller that enables the robot to maintain balance during motion using a data-driven numerical iteration (VI) scheme when the robot's dynamic characteristics are unknown. Specifically, the embodiment of the present disclosure proposes a numerical iteration method based on adaptive dynamic programming (ADP) based on optimal control and adaptive dynamic programming (ADP) technology. The numerical iteration method based on adaptive dynamic programming can calculate and converge to a controller corresponding to the dynamic characteristics of the robot when the robot's dynamic characteristics are unknown. The controller corresponding to the precise dynamic characteristics of the robot is also the controller corresponding to the optimal solution of the linear quadratic regulation problem, which can enable the robot to move along the target trajectory with the optimal control effect in a balanced state.
[0043] The embodiments of the present disclosure are based on artificial intelligence technologies, such as reinforcement learning and ADP technologies, and utilize numerical iteration schemes to solve the optimal balance control problem of the robot when the dynamic characteristics of the robot are unknown. The process of constructing a controller in the embodiments of the present disclosure only requires the wheel-legged robot to move for a period of time / a trajectory under the control of a non-optimal controller / an arbitrary controller, and collects motion state data and control data corresponding to the time period / trajectory as training data. Therefore, the amount of training data in the embodiments of the present disclosure is much smaller than the amount of data required by traditional reinforcement learning algorithms. Furthermore, the trained controller of the embodiments of the present disclosure gradually converges to a controller corresponding to the optimal solution of the linear quadratic regulation problem as the learning step size increases, thereby ensuring the stability of the closed-loop system, and the training process is greatly simplified, and no additional restrictions are required on the training data, thereby simplifying the design process of the controller of the wheel-legged robot.
[0044] To facilitate further description of the present disclosure, the meanings of various operators and sets that may be used in the following text are briefly explained here.
[0045] In this disclosure, represents the set of real numbers. |·| represents the Euclidean norm of a vector. stands for Kronecker product. Bold letters represent vectors or matrices. Italic letters represent scalars.
[0046] For any matrix A=[a1,…,a n ], For any symmetric matrix S = [s i,j ],vecs(S)=[s 1,1 ,2s 1,2 ,…,2s 1,n ,s 2,2 ,2s 2,3 ,…,2s n-1,n ,s n,n ] T For any vector
[0047] Figure 2 An exemplary flow chart of a method 200 for constructing a robot controller according to an embodiment of the present disclosure is shown. Figure 2 The method 200 is described in more detail. For example, the robot may be Figure 1 The robot 100 shown in FIG.
[0048] Reference Figure 2 In step S201, a robot with unknown dynamic characteristics is controlled to move in a quasi-equilibrium state to obtain motion state data and control data during the motion process, and the robot in the quasi-equilibrium state is near the equilibrium point during the motion process. For example, a first controller can be used to control a robot with unknown dynamic characteristics to move in a quasi-equilibrium state. For example, under the control of some first controllers, the robot will swing left and right at the equilibrium point with a certain amplitude. For example, if a robot with unknown dynamic characteristics is controlled to move in a quasi-equilibrium state under the control of a first controller, the output of the first controller can be used as the control data. If the controller is not set to control the robot to move in a quasi-equilibrium state, but the robot is controlled randomly, the control data can be obtained by collecting the output of the controller on a real machine (for example, detecting the driving force of the active wheel, etc.). The present disclosure does not limit the method of obtaining control data.
[0049] In step S202, a linear balance parameter matrix is calculated using numerical iteration according to the motion state data and the control data. In step S203, a controller corresponding to the dynamic characteristics of the robot is constructed based on the linear balance parameter matrix. For example, the robot under the control of the controller corresponding to the dynamic characteristics of the robot can have a better control effect during the movement process than the robot under the control of the first controller. For example, the swing amplitude of the robot under the controller corresponding to the dynamic characteristics of the robot around the balance point can be smaller than that under the first controller. For another example, the robot under the control of the controller can converge to the vicinity of the balance point faster, or the robot has smaller oscillation, or the control speed is faster, or the overshoot is smaller, or the steady-state error is smaller, etc. during the movement process compared to the robot under the control of the first controller. Or, in some cases, any controller and controller have the same control effect, but the control input of the controller is smaller. This disclosure is not limited to this.
[0050] Take the moving robot 100 as an example for explanation. The robot 100 in a balanced state may be in a stable balanced state in the linear motion dimension and the rotational motion dimension. For example, the robot 100 in a balanced state can maintain a state that is the same as / very similar to the state defined by the balance point during the movement, or can recover to the state defined by the balance point at the fastest speed / minimum energy consumption during the movement. The state defined by the balance point may be a state in which the robot 100 is in a state in which the pitch angle is zero, the angular velocity corresponding to the pitch angle is zero, and the linear speed is at the target speed. For example, at this time, the posture of the robot 100 is in a vertically upward state, and at this time, the robot 100 does not have a speed in the rotational motion dimension but only has a target speed in the linear motion dimension.
[0051] The robot 100 in a quasi-equilibrium state is in a state defined near the equilibrium point during the movement. For example, it may be in an intermediate state transitioning from a stable equilibrium state to an unstable equilibrium state in the linear motion dimension and the rotational motion dimension. The robot 100 in a quasi-equilibrium state may need the driving wheel to provide a large force and torque to ensure that it does not fall during the movement. For example, at this time, the robot 100 can tilt left and right, and at this time the robot 100 not only has the speed in the linear motion dimension but also has the speed in the rotational motion dimension. It is worth noting that the robot 100 in a quasi-equilibrium state herein may also be in a state close to an unstable equilibrium state in the linear motion dimension or the rotational motion dimension at certain moments during the movement, as long as it can be restored to a state where it can move normally through the driving force of the driving wheel 104.
[0052] As an example, if the robot 100 moves in a straight line only under the control of the driving wheel 104, the robot 100 in a balanced state can always maintain a vertically upward posture and move in a straight line at a uniform speed, that is, the central axis of the base of the robot 100 in an unstable balanced state can always be perpendicular to the horizontal line and has no speed / acceleration in the rotational motion dimension. The base of the robot 100 in a quasi-balanced state may have an inclination angle (pitch angle) and a speed / acceleration in the rotational motion dimension.
[0053] The embodiments of the present disclosure first enable the robot to move in a quasi-equilibrium state for a period of time / a trajectory, and collect motion state data and control data corresponding to the period of time / trajectory as training data. Since the dynamic characteristics of the robot 100 are unknown, the training data corresponds to a non-optimal controller (or, in some cases, to a first controller). A non-optimal controller is a controller that can only make the robot 100 move along the target trajectory in a stumbling manner. For example, the first controller that is a non-optimal controller can be a controller corresponding to simplified dynamic characteristics. For example, the precise dynamic model corresponding to a complex wheel-legged robot can be simplified into an equivalent dynamic model consisting only of active wheels and a base part, and so on. Reference will be made later Figure 3 Describe how to further design the first controller based on the simplified dynamic model. No further details are given here. For another example, the first controller may also be a PID controller, etc. In some cases, the parameters in the first controller may even be arbitrary values. Even more, the robot may be directly and randomly controlled with arbitrary control data. The present disclosure does not limit the specific design of the first controller, as long as it can control the robot to move in a quasi-equilibrium state.
[0054] Then, the embodiments of the present disclosure utilize a data-driven numerical iteration (VI) scheme to calculate the linear equilibrium parameter matrix and then construct a controller. The control effect of the controller is better than that of the non-optimal controller (or, in some cases, corresponding to the first controller).
[0055] Optionally, the control force provided by the controller at a moment is negatively correlated to the product of the linear balance parameter matrix and the motion state data of the robot at the moment.
[0056] For example, the controller may have the form of u(t)=-Kx(t), where K is the linear balance parameter matrix of the robot corresponding to the unknown dynamic characteristics, u(t) is the control force corresponding to the motor output controlled by the controller at time t, and x(t) is the motion state data corresponding to the robot at time t. The constructed controller can converge to the controller corresponding to the optimal solution of the linear quadratic regulation problem. The controller corresponding to the optimal solution of the linear quadratic regulation problem is also the controller corresponding to the precise dynamic characteristics of the robot. And the controller corresponding to the optimal solution of the linear quadratic regulation problem can minimize the cost functional of the robot during the movement process, so that the robot moves along the target trajectory with the optimal control effect in a balanced state. The numerical iteration scheme and the calculation scheme of the linear balance parameter matrix will be further described below, so they will not be repeated here.
[0057] Therefore, the amount of training data of the embodiment of the present disclosure is much smaller than the amount of data required by the traditional reinforcement learning algorithm. Furthermore, the trained controller of the embodiment of the present disclosure gradually converges to the controller corresponding to the optimal solution of the linear quadratic regulation problem as the learning step size increases, thereby ensuring the stability of the closed-loop system, and the training process is greatly simplified, and no additional restrictions are required on the training data, thereby simplifying the design process of the robot controller. In addition, the embodiment of the present disclosure can directly collect data on the real machine, and the trained controller is directly applied to the real machine. The embodiment of the present disclosure does not need to collect data in the simulation machine, and also eliminates some problems caused by the migration of data from the virtual world to the real world.
[0058] Optionally, an embodiment of the present disclosure also correspondingly discloses a method for controlling a robot using a controller corresponding to the dynamic characteristics of the robot. The robot includes a wheel leg portion and a base portion connected to the wheel leg portion, and the wheel leg portion includes a driving wheel and at least one joint. Specifically, the method includes: receiving a motion instruction, the motion instruction indicating the motion trajectory of the robot; according to the motion instruction, using a controller corresponding to the dynamic characteristics of the robot to control the driving force of the driving wheel so that the robot moves smoothly along the target trajectory. The robot under the control of the controller corresponding to the dynamic characteristics of the robot is closer to the balance point during the movement than the robot under the control of the first controller.
[0059] Optionally, an embodiment of the present disclosure also correspondingly discloses a method for controlling a robot. The dynamic characteristics of the robot are unknown. Specifically, the method includes: receiving a motion instruction, the motion instruction indicating the motion trajectory of the robot; according to the motion instruction, controlling the driving force of the driving wheel so that the robot moves in a quasi-equilibrium state and obtaining motion state data and control data during the motion process, the robot in the quasi-equilibrium state is near the balance point during the motion process; based on the motion state data and the control data, using a numerical iteration method to construct a controller corresponding to the dynamic characteristics of the robot, and using the controller to control the driving force of the driving wheel so that the robot moves smoothly. The robot under the control of the controller has a better control process during the motion process than the robot under the control of any other controller, for example, closer to the balance point.
[0060] Therefore, the method for controlling a robot according to an embodiment of the present disclosure enables a robot with unknown dynamic characteristics to learn data during movement, and gradually improve / generate a controller corresponding to the dynamic characteristics of the robot, and finally achieve smooth movement. Since any control input can be used to control the movement of the robot for a period of time to obtain training data, in such a case, the embodiment of the present disclosure realizes the generation of a controller corresponding to the dynamic characteristics of the robot from scratch. That is, the embodiment of the present disclosure can enable the robot to be flexibly controlled without an accurate dynamic model.
[0061] Next reference Figure 3 An example of the first controller is further described. Figure 3 A schematic diagram of a simplified dynamic model corresponding to a robot according to a disclosed embodiment is shown.
[0062] like Figure 3 As shown, due to the complex mechanical structure of the robot 100, it is difficult to obtain the accurate dynamic characteristics corresponding to the robot 100. In order to enable the robot to travel, the dynamic model of the robot can be simplified to obtain an inaccurate dynamic model. Figure 1 The complex robot 100 shown in FIG. 1 is simplified to Figure 3 The inverted pendulum model shown. For example, when the driving wheel 104 is driven, the angles of the joints in the wheel leg 103 will change accordingly, and then the angle θ between the line connecting the center of mass of the base 101 and the center of mass of the driving wheel 104 and the vertical line will change. θ is also called the tilt angle (pitch angle) of the robot. To simplify the calculation, when designing the first controller (the first controller is only used to control the driving force u of the driving wheel 104), the weight of the parallel leg is not considered, but only the mass m of the base 101 is considered. b and the mass m of the driving wheel 104c Therefore, a rough dynamic model can be constructed as shown in the following equation (1).
[0063]
[0064] Where m 11 =m b +m c , m 12 =m b l, m 22 =m b l 2 , m 22 =m b l 2 , G = m b Specifically, x represents the position of the center of the driving wheel 104. represents the forward speed of the robot 100. u represents the driving force provided by the driving wheel 104. w is the radius of the driving wheel 104. l is the height of the base 101. is the second-order derivative of x, representing the forward acceleration of the robot 100 . is the first-order derivative of the angle θ, and represents the rotational angular velocity of the base 101 . is the second-order derivative of the angle θ, and represents the rotational angular acceleration of the base 101 .
[0065] definition and at the balance point If we linearize formula (1), then formula (1) can be rewritten as formula (2). Where, Indicates the target forward speed of the robot 100.
[0066]
[0067] in,
[0068]
[0069] Therefore, the driving force u of the driving wheel 104 can be obtained according to formula (2). That is, the first controller can be obtained based on x, A and B.
[0070] As an example, based on the above derivation, the first controller can determine an initial control instruction according to the current motion state of the robot, and the initial control instruction is used to control the robot to move in a quasi-equilibrium state and obtain control data of the first controller; and according to the control data of the first controller, control the robot to move in a quasi-equilibrium state to obtain motion state data during the movement process.
[0071] Among them, the driving force u(t) provided by the first controller at time t is a function of x corresponding to time t, which is abbreviated as u(t)=u0(x). The above model is only an imprecise dynamic model, which does not take into account the possible influence of other components in the robot on the movement, and the component parameters may also have errors. Therefore, when the robot is controlled to move by the first controller based on formula (2), although the robot can roughly move along the target trajectory, there will be a large number of moments near the equilibrium point during the movement, and even moments that are largely near the equilibrium point. Therefore, using the first controller to control the movement of the robot often puts the robot in a quasi-equilibrium state. In this way, the motion state data of the robot in a quasi-equilibrium state can be collected.
[0072] Those skilled in the art should understand that Figure 3 It is only an example of a simplified dynamic model (inverted pendulum model of a cart) corresponding to the robot according to the disclosed embodiment. The first controller can also be obtained based on other simplified dynamic models such as an inverted pendulum model of a pulley. For example, the equivalent model can be designed by further considering the mass of each joint of the wheel leg and / or the linkage of each joint. Alternatively, it can be further considered that during the march, the distance between the center of mass of the base 101 and the center of mass of the driving wheel 104 will shorten and lengthen with the movement, and the joint part of the base 101 and the wheel leg 103 can be equivalent to a variable length telescopic cuboid, and so on. Different equivalent models will only lead to differences in specific parameters in the two matrices A and B, and under the technical solution in the embodiment of the present application, it will not affect the subsequent training results of the controller corresponding to the dynamic characteristics of the robot. That is, the difference in the equivalent model corresponding to the first controller will only lead to the difference in the acquired motion state data and the difference in the control data of the first controller, and will not lead to the difference in the constructed controller corresponding to the dynamic characteristics of the robot.
[0073] For another example, if the equivalent model considers more variables or the equivalent model is closer to the actual configuration of the robot, the robot controlled by the first controller will be less likely to be near the equilibrium point during the movement, or the deviation from the equilibrium point will be smaller. Even for some possible models, the robot controlled by the first controller can move in a near-equilibrium state. That is, the robot controlled by the first controller can move around the equilibrium point during the movement.
[0074] As another example, based on the above derivation, the first controller can also determine the initial control instruction according to the current motion state of the robot, and the initial control instruction is used to control the robot to move in a balanced state or in a quasi-balanced state; apply a disturbance to the control data indicated by the initial control instruction to obtain the control data of the first controller; and control the robot to move in a quasi-balanced state according to the control data of the first controller to obtain the motion state data during the motion process. For example, noise β(t) can be added to u0 to apply a disturbance to the above control data. In this case, the driving force u(t) provided by the first controller at time t is a linear superposition of the noise β(t) and the function of x corresponding to time t, which is abbreviated as u(t)=u0(x)+β(t). As a result, the system can collect enough motion data in a quasi-balanced motion state.
[0075] When the first controller is known, the motion state data during the motion process and the control data of the first controller can be obtained. For example, in the case of a preset target trajectory, the motion state data of the robot during the motion process can be collected by various sensors on the robot, and the value corresponding to the first controller can be calculated in real time based on the motion state data as control data, and then the motion state of the robot at the next moment can be further controlled based on the control data.
[0076] For example, for Figure 1 The motion state data of the robot 100 described above may be data of continuous moments sampled during the process of the robot 100 moving in a quasi-balanced state for a predetermined period of time under the control of the first controller, including the pitch angle of the base 101, the pitch angular velocity of the base 101, the linear velocity of the driving wheel 104 and the difference between the target linear velocity corresponding to these continuous moments. The control data is the driving force and / or torque of the driving wheel 104 corresponding to these continuous moments.
[0077] Next reference Figure 4 An example of obtaining a controller corresponding to the dynamic characteristics of the robot is further described. Figure 4 A flowchart of a numerical iteration scheme according to an embodiment of the present disclosure is shown.
[0078] First, the motion process of the robot 100 can be mathematically regarded as a continuous-time linear system. It is assumed that there is a controller corresponding to the optimal solution of the linear quadratic regulation problem for the robot 100, which can minimize the cost functional corresponding to the robot's motion process. For example, the controller corresponding to the optimal solution of the linear quadratic regulation problem can minimize the cost of the robot near the equilibrium point and can move along the target trajectory with minimum energy consumption. As an example, the linear quadratic regulation problem is defined by formula (3), which indicates that In the case of , solve the controller that can minimize the cost functional J of the continuous-time linear system.
[0079]
[0080] Where J is the cost functional of the continuous-time linear system, Q is a real symmetric and positive semidefinite matrix, is observable, and r>0. If the precise dynamic characteristics of the robot 100 are known, then the matrices A and B in formula (3) are known. When the matrices A and B in formula (3) are known, the following Algebraic Riccati equation (formula (4)) can be used as a solution to the following linear quadratic regulation (LQR) problem defined by formula (3).
[0081]
[0082] Among them, u * (t) is the controller corresponding to the optimal solution of the linear quadratic regulation problem, P * To satisfy According to the LQR related theory, for formula (3) and formula (4), if (A, B) is stable and is observable, then for any S ≥ 0, lim S→-∞ P(s)=P * , where P(s) is the solution of the following differential Riccati equation (Formula (5)), and P * is the solution of formula (4).
[0083]
[0084] That is, when s approaches negative infinity, P(s) will converge to P * That is, when the matrices A and B are known and the initial and final states are known, the solution of formula (5) can be deduced step by step in time to solve P * (s). However, since the precise dynamic characteristics of the robot 100 are unknown, the matrices A and B are unknown, and the above formula (4) cannot be solved directly.
[0085] To this end, the present disclosure proposes a solution using a data-driven numerical iteration solution to solve formula (5), which can solve the problem of convergence to P when matrices A and B are unknown. * P(s). Formula (5) can be solved by numerical iteration.
[0086] For example, for Figure 2The step S202 in the embodiment may further include: step S2021, respectively performing integral operations on the motion state data and the control data corresponding to the multiple time intervals to construct an iterative relationship function; and step S2022, performing numerical iteration on the iterative target item according to the iterative relationship function to approximate the linear balance parameter matrix corresponding to the dynamic characteristics of the robot. Optionally, the linear balance parameter matrix K is Here, s tends to negative infinity.
[0087] Next, Figures 1 to 3 The examples described in the above are used to illustrate step S2021 and step S2022 respectively. Figures 1 to 3 The description of can obtain the motion state data of the robot in the quasi-balanced state and the control data of the first controller. That is, at least in this motion process, based on formula (2), the following formula (6) is established.
[0088]
[0089] Where H(s) = A T P(s)+P(s)A and
[0090] Furthermore, the motion state data is collected by the sensor at certain time intervals over a period of time, which respectively corresponds to the motion state of the robot at each discrete moment in the period of time. Therefore, the motion state data and the control data of the first controller can correspond to [t1,t l ] in multiple time intervals. Any time interval t in the multiple time intervals i to i+1 can be written as [t i ,t i+1 ], the duration of which can be determined based on the data collection time interval that the robot's sensors can achieve.
[0091] refer to Figure 4 , the motion state data and the control data corresponding to the multiple time intervals can be integrated respectively to construct an iterative relationship function. The motion state data and the control data after the integration operation will be used as training data to participate in the process of numerically iterating the iterative target item in step S2022 to approximate the linear balance parameter matrix corresponding to the dynamic characteristics of the robot. It is worth noting that the following description is only an exemplary integration operation, and the present disclosure is not limited to this.
[0092] For example, for any [t i ,t i+1], based on formula (6), we can get formula (7).
[0093]
[0094] According to formula (7), any two adjacent time points t i and t i+1 The time integral of the motion state data between time t and time t may be related to at least one of the following: i The quadratic term of the motion state data, time t i+1 The quadratic term of the motion state data, time t i The motion state data and time t i+1 The product of the motion state data, time t i The control data and time t i The product of the motion state data, time t i+1 The control data and time t i+1 The product of motion state data, and so on.
[0095] To further describe the process of numerical iteration, the following three data sets are defined by formula (8): xx , the second data set I xx and the third data set I xu Each data set corresponds to a nonlinear combination of the motion state data and the control data in the plurality of time intervals, for example, involving integral operations and product calculations, etc. Optionally, the second data set I xx The third data set 2rI after linear change xu can be concatenated into a full-rank matrix M. The present disclosure is not limited thereto.
[0096]
[0097] In formula (8), t0 <t1<…<t i <… <t l+1 , [t i ,t i+1 ] is the data collection interval. For example, for Figure 1 The robot 100 in the first data set D xx Any element in
[0098] Corresponding to time t i and time t i+1 The product of any two of the base pitch angle, base pitch angular velocity, and the difference between the linear velocity of the driving wheel and the target velocity, or the difference of the quadratic term of any one of them. xx Any element in Corresponding to time t i and time t i+1 The product of any two of the base pitch angle, the base pitch angular velocity, the difference between the linear velocity of the driving wheel and the target velocity, or the integral of the quadratic term of any one of them. xu Any element in Corresponding to time t i and time t i+1 The integral of the product of any one of the base pitch angle, the base pitch angular velocity, the difference between the linear velocity of the driving wheel and the target velocity and the driving force of the driving wheel. Different robot configurations will correspond to different data sets, which are shown above only as examples and the present disclosure is not limited thereto.
[0099] After obtaining the training data, it is necessary to further analyze the relationship between the training data to construct an iterative relationship function, so as to solve the linear equilibrium parameter matrix. Those skilled in the art should understand that different training data construction modes will correspondingly affect the form of the constructed iterative relationship function. The following is only an exemplary iterative relationship function obtained based on formula (8), and the present disclosure is not limited thereto.
[0100] For example, combined with formula (8), let M = [I xx ,2rI xu ], the interval [t1, t2] to [t l ,t l+1 ] is written in a compact form, resulting in the following formula (9).
[0101]
[0102] According to automatic control theory, M has full rank, that is, According to the least squares method, formula (9) can be further rewritten as:
[0103]
[0104] in,
[0105] By taking the differential of both sides of formula (10) with respect to s and combining it with formula (5), we can obtain formula (11).
[0106]
[0107] From this, the iterative relationship function can be further derived. Specifically, let the left side of equation (11) be the differential of the iterative target term y with respect to s, and the right side be f(y), that is, equation (11) can be simplified as For ease of expression, the iteration target terms y and f(y) in the iteration relationship function are further defined in formula (12).
[0108]
[0109] Specifically, at y = 0, f(y) is locally Lipschitz stable with respect to y. Therefore, according to automatic control theory, there exists some δ>0 such that In y(t f )=0 interval [t f -δ,t f ] has a unique solution. Further according to formula (6) to formula (11), we know that H(s) = A T P(s)+P(s)A and Satisfies formula (11), therefore, H(s) = A T P(s)+P(s)A and It is the only solution of formula (11).
[0110] That is, when the following conditions ① to ③ are satisfied, formula (13) can be obtained. Among them, conditions ① to ③ are: Condition ①: M has full rank; Condition ②: P(t f )=0,H(t f )=0,K(t f )=0; Condition ③: H(s), K(s) are the solutions of formula (11).
[0111]
[0112] Among them, P * is the solution of formula (4). That is, when s approaches negative infinity, K(s) can converge to According to formula (4), is u * (t) corresponds to the linear equilibrium parameter matrix. Further, in formula (11), M, D xx It can be constructed by the motion state data of the robot in the motion process of the quasi-equilibrium state and the control data of the first controller. Formula (11) can be solved inversely by Euler method.
[0113] Formula (5) to Formula (13) are solved to obtain the calculation method of the iterative target term y containing the linear equilibrium parameter matrix: It includes the differential form of the iterative step target term, so the linear balance parameter matrix corresponding to the dynamic characteristics of the robot can be approximated by numerical iteration.
[0114] Optionally, step S2022 further includes: when the iterative target item converges during the numerical iteration process, stopping the numerical iteration; and reconstructing the linear balance parameter matrix corresponding to the dynamic characteristics of the robot according to the converged iterative target item.
[0115] For example, since both the motion data and the control data are in discrete form, the iterative relationship function can be constructed in discrete form at least according to the above formulas (5) to (13), that is, f -kh)=y(t f -(k-1)h)-hf(y(t f -(k-1)h)).
[0116] That is, the iterative relationship function is expressed as y(t f -kh)=y(t f -(k-1)h)-
[0117] hf(y(t f -(k-1)h)) is used to calculate the iteration target item y(t) corresponding to the k-1th iteration. f -
[0118] (k-1)h) iteratively calculates the iteration target item y(t f -kh), where h is the iteration step size.
[0119] As the number of iterations increases, the iteration target item y(t f -kh) will gradually converge. For this purpose, refer to Figure 4 In industrial operation, the iteration step size h and the convergence condition can be further set. For example, the convergence condition can be set as follows: when the difference between the iteration target item obtained in the kth iteration and the iteration target item obtained in the k-1th iteration is small enough (for example, less than a sufficiently small threshold ε), the iteration is stopped.
[0120] As shown in formula (12), Based on the convergence of the iterative objective term y(t f -kh) can directly obtain the iterative parameter term K(t f -kh). Therefore, the above-mentioned reconstruction of the linear balance parameters corresponding to the dynamic properties of the robot according to the converged iterative objective term may further include: according to the converged iterative objective term y(t f -kh), and the iterative target term y(t f -kh) and the iteration parameter K(t f -kh), and obtain the convergence K(t f -kh); the convergence K(t f-kh) is set as the linear balance parameter matrix; based on the linear balance parameter matrix, a controller is constructed, wherein the controller is u=-K(t f -kh)x, the controller corresponds to the dynamic characteristics of the robot.
[0121] As described above, the embodiment of the present disclosure only collects the motion state data of the robot with unknown dynamic characteristics in the quasi-equilibrium state and the control data of the first controller, and constructs the first data set D by performing an integral operation on these data. xx , the second data set I xx and the third data set I xu As training data. Therefore, the amount of training data in the embodiment of the present disclosure is much smaller than the amount of data required by the traditional reinforcement learning algorithm. The embodiment of the present disclosure also constructs an iterative relationship function accordingly, so that the target iteration term gradually converges with the increase of the learning step size. Moreover, the converged target iteration term can obtain a controller, which converges to the controller corresponding to the optimal solution of the linear quadratic regulation problem, thereby ensuring the stability of the closed-loop system and greatly simplifying the training process. The entire process does not require additional restrictions on the training data, thereby simplifying the design process of the robot's controller.
[0122] Next reference Figures 5 to 6B Further description of Figure 1 The method for controlling the robot shown. Figure 5 Still another structural view of the robot 100 is shown. Fig. 6A The motion state data and control data of the robot in the process of using the first controller to control the robot's motion in a quasi-balanced state are shown. Figure 6B The process of constructing a controller corresponding to the dynamic characteristics of the robot is shown in K(t f -kh) and H(t f -kh) convergence process, where the base height of the robot is 0.65 meters. Figure 6C The process of constructing a controller corresponding to the dynamic characteristics of the robot is shown in K(t f -kh) and
[0123] H(t f -kh) convergence process, where the base height of the robot is 0.38 meters. Fig.6D The motion state data and control data of the robot using the constructed controller to control the robot to walk in a straight line when the base height is 0.38 meters. Fig. 6E The motion state data and control data of the robot using the constructed controller to control the robot to yaw when the base height is 0.65 meters. Fig. 6F The motion state data of the robot using the constructed controller to control the robot to move along the "S" path is shown. Figure 6G The data shows the motion state of the robot on the speed bump controlled by the constructed controller. Figure 6H The motion state data of the robot is shown using the constructed controller to control the robot to resist external forces.
[0124] like Figure 5 As shown, the robot 100 has Figure 1 In addition to the wheel leg part and the base part described in the text, it also includes a data acquisition device, a data processing device and a drive motor.
[0125] The data acquisition device may be configured to: when a robot with unknown dynamic characteristics is controlled to move in a quasi-equilibrium state by a first controller, obtain the motion state data of the robot in the quasi-equilibrium state, wherein the robot in the quasi-equilibrium state is near the equilibrium point. For example, the data acquisition device may include: a first sensor for measuring the pitch angle θ of the base part and its angular velocity The second sensor is used to measure the rotational angular velocity of the left and right driving wheels and The first sensor may be an inertial measurement unit (IMU), which may include a three-axis gyroscope, a three-axis accelerometer, or a three-axis magnetometer. The second sensor may be a motor encoder with a sampling frequency of 200 Hz.
[0126] The data processing device is configured to: obtain control data corresponding to the motion state data; calculate a linear balance parameter matrix using a numerical iteration method based on the motion state data of the robot in a quasi-equilibrium state and the control data; and construct a controller corresponding to the dynamic characteristics of the robot based on the linear balance parameter matrix.
[0127] The data processing device may include a microprocessor, a digital signal processor ("DSP"), an application specific integrated circuit ("ASIC"), a field programmable gate array, a state machine, or other processing device for processing the electrical signals received from the sensor line. Such processing devices may include programmable electronic devices, such as PLCs, programmable interrupt controllers ("PIC"), programmable logic devices ("PLD"), programmable read-only memories ("PROM"), electronic programmable read-only memories, etc.
[0128] For example, the data processing device can be further configured to further process the data collected by the data collection device. and The data processing device can calculate the linear velocity of the robot 100 where r w is the radius of the driving wheel. Optionally, r w =0.1m. Optionally, the data processing device can also calculate the yaw angular velocity of the robot where w d is the robot width, optionally, w d =0.47m.
[0129] For ease of explanation, the present disclosure only gives an example of using the first controller or the second controller to control the driving wheel 104. Those skilled in the art should understand that the solution of the present disclosure can also be used to control other components of the robot. Since the driving wheel is only used to control the forward and backward movement of the robot, for a curved target trajectory, a controller for controlling the yaw angle is also required to control the robot's steering. To simplify the description, the controller is set to in is the target yaw rate. Then by and Calculate the torque on the left and right wheels. τ ψ The force along the longitudinal direction of the robot is not changed. Therefore, the yaw movement does not affect the balance of the robot. Afterwards, the angle unit is converted to "degrees (deg)" for easier reading.
[0130] Next, the data processing device calculates the control data of the first controller based on the given target trajectory. Figure 3 As mentioned above, the present disclosure does not limit the specific control law of the first controller. For the convenience of explanation, a simple PID controller is used as an example for explanation in the following. Furthermore, the control frequency of the data processing device is 1000 Hz.
[0131] As described above, the motion data and control data will be used to calculate the first data set D xx , the second data set I xx and the third data set I xu These data require continuous signals of x and u, so when the first controller and the controller corresponding to the dynamic characteristics of the robot control the movement of the robot 100, the data processing device can further use trapezoidal integration to calculate the integral. The step size of the trapezoidal integration is 0.01s, which is the same as the sampling period.
[0132] The data processing device may be further configured with the following instructions to perform calculation of control data of the first controller and construction of a controller corresponding to the dynamic characteristics of the robot. For ease of description, the instructions are shown in the form of pseudo code, and those skilled in the art can calculate the control data of the first controller and construct a controller corresponding to the dynamic characteristics of the robot using any programming language based on the following pseudo code.
[0133] ********************************************************
[0134] 1: Select the initial controller u0(x) and noise β(t) to calculate the control number of the first controller
[0135] According to: u(t)=u0(x)+β(t).
[0136] 2: Apply u(t) to the robot 100 and use the data acquisition device to collect the data from t0 to t l+1 Data
[0137] x(t)
[0138] 3: Construction D xx ,I xx ,I xu
[0139] 4: Set y(t f )=0,k=0,iteration step size h and small threshold ε
[0140] 5: Repeat steps 6 and 7 until |y(t f -kh)-y(t f -(k-1)h|<ε
[0141] 6: k+1 is assigned to k
[0142] 7:y(t f -kh)=y(t f -(k-1)h)-hf(y(t f -(k-1)h))
[0143] 8: From y(t f -kh) reconstruction K(t f -kh)
[0144] 10: Use u=-K(t f -kh)x as a controller corresponding to the dynamic characteristics of the robot.
[0145] ********************************************************
[0146] refer to Fig. 6A Given the example, Fig. 6A The height of the base of the robot 100 is l = 0.65 m. Let Q = diag[2000, 100, 100], r = 1, t i+1 -t i =0.1s, h=0.001, ε=0.001, β(t)=0.3sin(πt). Fig. 6A The upper left figure shows the change of the pitch angle (unit: degree) of the base over time, the upper right figure shows the change of the pitch angle velocity (unit: degree / second) of the base over time, the lower left figure shows the change of the linear velocity (unit: meter / second) of the robot over time, and the lower right figure shows the change of the driving force (unit: Newton) of the active wheel over time. In the above four figures, the horizontal axis is time, and the unit is second.
[0147] like Fig. 6A As shown in Figure 2, due to the influence of sensor noise and β(t), the pitch angle θ and velocity x fluctuate greatly near the equilibrium point, so the full rank condition of M can be satisfied.
[0148] The process of constructing a controller corresponding to the dynamic characteristics of the robot is as follows Figure 6B As shown. Among them, Figure 6B The left figure shows K(t f -s) with the iteration of s. For the convenience of illustration, u = K′x, K′ = -K, and the negative gain in K′ represents negative feedback. Figure 6B The right figure shows H(t f -s) changes with the iteration of s. Figure 6B In the right figure, six curves are shown, from top to bottom, respectively: H(t f -s) in the first row and first column, the first row and second column, the first row and third column, the second row and first column, the second row and second column, and the second row and third column. Figure 6B It is shown that when s>2, K(t f -s) converges to [-116.93,-42.96,-20.18], H(t f -s) converges to
[0149] refer to Figure 6C, a controller corresponding to the dynamic characteristics of the robot can also be constructed similarly when the height of the base of the robot 100 is l = 0.38m. The training data is obtained based on the following parameters. Let Q = diag[200, 100, 100], r = 1, t i+1 -t i =0.1s, h=0.001, ε=0.001, β(t)=0.2sin(1.5πt).
[0150] in, Figure 6C The left figure shows K(t f -s) with the iteration of s. For the convenience of illustration, u = K′x, K′ = -K, and the negative gain in K′ represents negative feedback. Figure 6C The right figure shows H(t f -s) changes with the iteration of s. Figure 6C In the right figure, six curves are shown, from top to bottom, respectively: H(t f -s) in the first row and first column, the first row and second column, the first row and third column, the second row and first column, the second row and second column, and the second row and third column. Figure 6C It is shown that when s>1, K(t f -s) converges to [-88.91,-30.22,-16.48], H(t f -s) converges to
[0151]
[0152] First, use K = [-88.91, -30.22, -16.48] to construct the controller u = -K (t f -kh)x to control the robot 100 to perform linear motion when the base height is 0.38 meters. For test data in linear motion, see Fig.6D shown. Fig.6D The upper left figure shows the change of the pitch angle (unit: degree) of the base over time, the upper right figure shows the change of the pitch angle velocity (unit: degree / second) of the base over time, the lower left figure shows the change of the linear velocity (unit: meter / second) of the robot over time, and the lower right figure shows the change of the driving force (unit: Newton) of the active wheel over time. In the above four figures, the horizontal axis is time, and the unit is seconds. It can be seen that when using K=
[0153] [-88.91,-30.22,-16.48] to build the controller, the pitch angle and pitch velocity are almost around the equilibrium point 0. In addition, because the robot moves slowly backward and forward in the test, the experimenter adjusts the remote control to discontinuously send the reference signal to indicate the target linear velocity. Therefore, the driving force does not remain at zero, and other states also change around zero.
[0154] Next, we use K = [-116.93, -42.96, -20.18] to construct the controller u = -K (t f -kh)x to control the robot 100 to perform yaw motion (i.e., rotation in place) when the base portion is at a height of 0.65 meters. For test data in yaw motion, see Fig.6D shown. Fig.6D The upper left figure shows the pitch angle of the base (unit: degree) over time, the upper right figure shows the linear velocity of the base (unit: degree / second) over time, the lower left figure shows the yaw angular velocity of the robot (unit: meter / second) over time, and the lower right figure shows the driving force of the left and right driving wheels (unit: Newton) over time. In the above four figures, the horizontal axis is time, in seconds. Rotation period The pitch angle approaches 0, and varies in a small range of [-1.6°, 1.3°]. It can be seen that the controller has excellent balancing performance.
[0155] Continue to use the controller to control the robot to move along the "S" path. Its motion state is as follows Fig. 6F shown. Fig. 6F The left figure shows the change of the linear velocity of the base (unit: degree / second) over time, and the right figure shows the change of the yaw angular velocity of the base (unit: meter / second) over time. According to the test, the pitch angle θ varies slightly between ±2.5°, and the actual linear velocity is almost the same as the target linear velocity.
[0156] Continue to use the controller to control the robot to move on the speed bump. Its motion state is as follows: Figure 6G shown. Figure 6G The upper left figure shows the pitch angle of the base (unit: degree) changing with time, the upper right figure shows the pitch angle velocity of the base (unit: degree / second) changing with time, the lower left figure shows the linear velocity of the robot (unit: meter / second) changing with time, and the lower right figure shows the driving force of the active wheel (unit: Newton) changing with time. In the above four figures, the horizontal axis is time, in seconds. Speed bumps can be considered as interference to the bottom of the robot. Figure 6GAs shown in the figure, the robot passes the speed bump 6 times, moving forward 3 times and moving backward 3 times. In particular, at each speed bump, the robot first accelerates and rushes to the speed bump. Then, when it reaches the top of the speed bump, it slows down. Next, the speed increases again while going downhill. Finally, it stops on the ground. The whole process is clearly reflected in Figure 6G In order to overcome the interference of speed bumps, the control path of the controller changes sharply at the speed bumps.
[0157] As a final example, let's test the robustness of the robot. While the robot is being controlled by the controller, kick the base of the robot to apply an external force to the top of the robot. Figure 6H shown. Figure 6H The upper left figure shows the change of the pitch angle (unit: degree) of the base over time, the upper right figure shows the change of the pitch angle velocity (unit: degree / second) of the base over time, the lower left figure shows the change of the linear velocity (unit: meter / second) of the robot over time, and the lower right figure shows the change of the driving force (unit: Newton) of the active wheel over time. In the above four figures, the horizontal axis is time, and the unit is seconds. Figure 6H As shown in , the robot was kicked six times. Here, the process of the robot being kicked for the first time is discussed in detail, which corresponds to Figure 6H When the top of the robot is acted upon by forces in the negative z direction and the positive x direction, the robot moves in the positive x direction, and due to inertia, the pitch angle is negative. Figure 6H Sudden changes in mid-pitch angle in the negative direction and Figure 6H The sudden change in linear velocity in the positive direction around t=8 seconds is consistent. In this case, as the controller controls the robot to slow down (t=8.5s to 9s), the pitch angle becomes more negative. Then, the dominant negative pitch angle (reaching a pitch angle of almost -10° at t=9s) leads to a large negative control law, so the robot moves in the negative x direction (t=9.1s). Then, similar to the normal situation, the controller adjusts the linear velocity, pitch angle, and pitch angular velocity to zero before the next kick (t=10s). In this example, as the robot is subjected to larger forces, the controller outputs larger control forces, and the robustness is also good.
[0158] Those skilled in the art will appreciate that the controller can also be used to control other motions, and the present disclosure is not limited thereto. In addition, according to tests, the robustness of the controller is much higher than that of the PID controller, that is, when the robot 100 is disturbed externally, the robot under the control of the controller can quickly restore balance.
[0159] Therefore, the embodiment of the present disclosure is based on reinforcement learning and ADP technology in artificial intelligence, and uses a numerical iteration scheme to solve the optimal balance control problem of the robot under the condition of unknown dynamic characteristics of the robot. The process of constructing a controller in the embodiment of the present disclosure only requires the wheel-legged robot to move for a period of time / a section of trajectory under the control of a non-optimal controller / an arbitrary controller, and collects motion state data and control data corresponding to the time period / trajectory as training data. Therefore, the amount of training data in the embodiment of the present disclosure is much smaller than the amount of data required by the traditional reinforcement learning algorithm. Furthermore, the trained controller of the embodiment of the present disclosure gradually converges to the controller that best corresponds to the optimal solution of the linear quadratic regulation problem as the learning step size increases, thereby ensuring the stability of the closed-loop system, and its training process is greatly simplified, and no additional restrictions are required on the training data, thereby simplifying the design process of the controller of the wheel-legged robot.
[0160] According to actual needs, the robot may also include, for example, a bus, a memory, a sensor assembly, a communication module, an input and output device, etc. The embodiments of the present disclosure are not limited by the specific components of the robot.
[0161] The bus may be a circuit that interconnects the components of the robot and passes communication information (eg, control messages or data) among the components.
[0162] The sensor component can be used to perceive the physical world, such as a camera, an infrared sensor, an ultrasonic sensor, etc. In addition, the sensor component can also include a device for measuring the current operation and motion state of the robot, such as a Hall sensor, a laser position sensor, or a strain sensor.
[0163] The communication module can be connected to a network, for example, by wire or invalid, so as to communicate with the physical world (e.g., a server). The communication module can be wireless and can include a wireless interface, such as an IEEE802.11, Bluetooth, wireless local area network ("WLAN") transceiver, or a radio interface for accessing a cellular telephone network (e.g., a transceiver / antenna for accessing a CDMA, GSM, UMTS or other mobile communication network). In another example, the communication module can be wired and can include an interface such as Ethernet, USB or IEEE 1394.
[0164] The input-output device may transmit commands or data input, for example, from a user or any other external device to one or more other components of the robot, or may output commands or data received from one or more other components of the robot to the user or other external device.
[0165] A plurality of robots may form a robot system to collaboratively complete a task, the plurality of robots being communicatively connected to a server and receiving collaborative robot instructions from the server.
[0166] According to one aspect of an embodiment of the present disclosure, there is provided a wheel-legged robot, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the above method.
[0167] According to one aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above method.
[0168] The program part of the technology can be considered as a "product" or "manufactured product" in the form of executable code and / or related data, which is participated in or realized by computer-readable media. Tangible and permanent storage media can include any memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives or any similar devices that can provide storage functions for software.
[0169] All software or parts of it may sometimes be communicated over a network, such as the Internet or other communications network. Such communications can load software from one computer device or processor to another. Therefore, another medium capable of transmitting software elements may also be used as a physical connection between local devices, such as light waves, radio waves, electromagnetic waves, etc., transmitted through cables, optical cables or air. Physical media used to carry carriers, such as cables, wireless connections or optical cables and the like, can also be considered as media that carry software. As used herein, unless limited to tangible "storage" media, other terms referring to computer or machine "readable media" refer to media that participate in the process of executing any instructions by the processor.
[0170] The present disclosure uses specific words to describe the embodiments of the present disclosure. For example, "first / second embodiment", "one embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present disclosure. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different locations in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of the present disclosure may be appropriately combined.
[0171] In addition, it will be appreciated by those skilled in the art that various aspects of the present disclosure may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present disclosure may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present disclosure may be represented as a computer product located in one or more computer-readable media, the product including computer-readable program code.
[0172] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.
[0173] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.
Claims
1. A method for constructing a controller of a wheel-legged robot, the method comprising: Controlling a wheel-legged robot with unknown dynamic characteristics to move in a quasi-equilibrium state to obtain motion state data and control data during the movement, wherein the wheel-legged robot in the quasi-equilibrium state is near a balance point during the movement; Calculating a linear balance parameter matrix using a numerical iteration method according to the motion state data and the control data; as well as Based on the linear balance parameter matrix, a controller corresponding to the dynamic characteristics of the wheel-legged robot is constructed.
2. The method of claim 1, wherein: The quasi-balanced motion state of the wheel-legged robot is controlled by a first controller. The wheel-legged robot under the control of the controller corresponding to the dynamic characteristics of the robot has a better control effect during the motion process than the wheel-legged robot under the control of the first controller.
3. The method of claim 1, wherein: The controller is a linear controller, and the control force provided by the controller at a moment is negatively correlated to the product of the linear balance parameter matrix and the motion state data of the wheel-legged robot at the moment.
4. The method of claim 2, wherein: The control of the wheel-legged robot to move in a quasi-balanced state to obtain motion state data and control data during the motion process includes: The first controller determines an initial control instruction according to the current motion state of the wheel-legged robot, wherein the initial control instruction is used to control the wheel-legged robot to move in a near-balanced state or in a quasi-balanced state; applying disturbance to the control data indicated by the initial control instruction to obtain control data of the first controller; and According to the control data of the first controller, the wheel-legged robot is controlled to move in a quasi-balanced state to obtain motion state data during the movement.
5. The method of claim 1, wherein: The motion state data and the control data correspond to a plurality of time intervals.
6. The method of claim 5, wherein: The method of calculating the linear balance parameter matrix by numerical iteration according to the motion state data of the wheel-legged robot in a quasi-balanced state and the control data comprises: respectively performing operations on the motion state data and the control data corresponding to the plurality of time intervals to construct an iterative relationship function; and According to the iterative relationship function, the iterative target item is numerically iterated to approximate the linear balance parameter matrix corresponding to the dynamic characteristics of the wheel-legged robot.
7. The method of claim 6, wherein: The method of numerically iterating the iterative target according to the iterative relationship function to approximate the linear balance parameter matrix corresponding to the dynamic characteristics of the wheel-legged robot includes: When the iteration target item converges during the numerical iteration process, stopping the numerical iteration; and According to the converged iterative objective term, the linear balance parameter matrix corresponding to the dynamic characteristics of the wheel-legged robot is reconstructed.
8. The method of claim 7, wherein: The iterative relationship function is expressed as y(t f -kh)=y(t f -(k-1)h)-hf(y(t f -(k-1)h)) is used to calculate the iteration target item y(t) corresponding to the k-1th iteration. f -(k-1)h) iteratively calculates the iteration target item y(t f -kh), where h is the iteration step length, t f is a time point, and f(·) is an equation about y(·).
9. The method of claim 8, wherein: f(y)=M in the iterative relationship function + D xx vecs(-H(s)+rK T (s)K(s)-Q), M and D xx Corresponding to the nonlinear combination of the motion state data and the control data in the multiple time intervals, and M has full rank, the corresponding relationship between H(s) and K(s) and the iteration target item y is Q is a real symmetric and positive semidefinite matrix.
10. The method of claim 8, wherein: The step of reconstructing the linear balance parameters corresponding to the dynamic properties of the wheel-legged robot according to the converged iterative objective term comprises: According to the converged iteration target term y(t f -kh), and the iterative target term y(t f -kh) and the iteration parameter K(t f -kh), and obtain the iterative target item y(t f -kh) when the iteration parameter K(t f -kh); The iterative parameter term K(t f -kh) is set to the linear equilibrium parameter matrix; The controller is constructed based on the linear equilibrium parameter matrix, wherein the controller is u=-K(t f -kh)x, the controller corresponds to the dynamic characteristics of the wheel-legged robot.
11. A wheel-legged robot, comprising: The data acquisition device is configured to: when controlling a wheel-legged robot with unknown dynamic characteristics to move in a quasi-equilibrium state, obtain motion state data of the wheel-legged robot in the quasi-equilibrium state, wherein the wheel-legged robot in the quasi-equilibrium state is near a balance point; A data processing device configured to: Acquiring control data corresponding to the motion state data; Calculating a linear equilibrium parameter matrix using numerical iteration according to the motion state data of the robot in a quasi-equilibrium state and the control data; and Based on the linear balance parameter matrix, a controller corresponding to the dynamic characteristics of the wheel-legged robot is constructed.
12. The wheel-legged robot according to claim 11, further comprising: A wheel leg portion including a plurality of joints; a base portion connected to the wheel leg portion; A driving motor is provided on the robot to control the driving wheel in the wheel leg based on the controller.
13. A non-transitory computer-readable medium having at least one instruction stored thereon, wherein the at least one instruction is loaded and executed by a processor to implement the method of any one of claims 1 to 10.
14. A method for controlling a wheel-legged robot, wherein the wheel-legged robot is driven by active wheels and the dynamic characteristics of the wheel-legged robot are unknown, the method comprising: receiving a motion instruction, wherein the motion instruction indicates a motion trajectory of the wheel-legged robot; According to the motion instruction, the driving force of the active wheel is controlled so that the wheel-legged robot moves in a quasi-balanced state and obtains motion state data and control data during the movement, and the wheel-legged robot in the quasi-balanced state is near the balance point during the movement; Based on the motion state data and the control data, constructing a controller corresponding to the dynamic characteristics of the wheel-legged robot by using a numerical iteration method; and The controller is used to control the driving force of the active wheel so that the wheel-legged robot moves smoothly.
