Tumbling and turning-over control method and device of foot type robot and storage medium

Abstract

The invention discloses a fall and turn-over control method and device of a foot type robot and a storage medium. The fall and turn-over control method comprises the steps of firstly obtaining fall posture information of the foot type robot; calling an action selection model to perform multi-stage iterative turnover action prediction based on the falling posture information to obtain an optimal turnover action of each iteration stage, controlling the foot type robot to execute the optimal turnover action of the current iteration stage in each iteration stage, and controlling the foot type robot to execute the optimal turnover action of the current iteration stage after the foot type robot executes each optimal turnover action. The foot-type robot is in a stable posture when the foot-type robot turns over, and the foot-type robot turns over successfully after executing the optimal turning-over action of the last iteration stage. Wherein when the action selection model is called to carry out multi-stage iterative turning-over action prediction based on the falling posture information, the action selection model predicts the optimal turning-over action of the next iteration stage according to the stable posture of the foot robot in the current iteration stage. According to the embodiment, the manual design cost can be reduced, and the applicability is high.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a method, device, and storage medium for controlling a legged robot from falling and turning over. Background Art

[0002] Currently, robots are widely used in industrial manufacturing, as well as in agricultural operations, environmental exploration, and disaster relief. Legged robots are prone to losing balance and falling due to terrain and environmental factors when performing tasks. In such cases, legged robots need to be able to roll over after a fall.

[0003] In the prior art, a legged robot's fall-and-turn motion scheme is generally pre-set manually, so that when the legged robot falls, it can control each joint to perform the corresponding turning movement according to the fall-and-turn motion scheme, so that it can stand up from the fallen state. However, the fall-and-turn motion scheme in this prior art requires a long period of manual design and debugging, and most fall-and-turn motion schemes can only successfully turn over for specific fall postures. If the legged robot is to be able to successfully turn over in different fall postures, it needs to be manually designed and debugged for each fall posture, but this will undoubtedly greatly increase the manual design cost of the legged robot. Summary of the Invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide a method, device, and storage medium for controlling a fall and flipping of a legged robot, which can effectively reduce the cost of manual design and have high applicability.

[0006] In one aspect, an embodiment of the present application provides a method for controlling a legged robot from falling and turning over, comprising the following steps:

[0007] Acquiring falling posture information of the legged robot;

[0008] calling an action selection model to perform a multi-stage iterative turning action prediction based on the falling posture information to obtain an optimal turning action in each iterative stage, and controlling the legged robot to perform the optimal turning action in the current iterative stage in each iterative stage, wherein the legged robot is in a stable posture after performing the optimal turning action in each iterative stage, and the legged robot successfully turns over after performing the optimal turning action in the last iterative stage;

[0009] Among them, when calling the action selection model to perform the multi-stage iterative turning action prediction based on the falling posture information, the action selection model predicts the optimal turning action in the next iterative stage according to the stable posture of the legged robot in the current iterative stage.

[0010] On the other hand, an embodiment of the present application further provides a fall and turn control device for a legged robot, comprising:

[0011] A control module, configured to obtain falling posture information of the legged robot;

[0012] The control module is further configured to call an action selection model to perform a multi-stage iterative turning action prediction based on the falling posture information, obtain an optimal turning action in each iterative stage, and control the legged robot to perform the optimal turning action in the current iterative stage in each iterative stage, wherein the legged robot is in a stable posture after performing the optimal turning action in each iterative stage, and the legged robot successfully turns over after performing the optimal turning action in the last iterative stage;

[0013] Among them, when the control module calls the action selection model to perform the multi-stage iterative turning action prediction based on the falling posture information, the action selection model predicts the optimal turning action in the next iterative stage according to the stable posture of the legged robot in the current iterative stage.

[0014] Optionally, the control module is further configured to:

[0015] Determine an initial strategy for turning over;

[0016] determining a candidate turning action according to the stable posture of the legged robot in the current iteration stage and the initial turning action strategy, and controlling the legged robot to perform the candidate turning action;

[0017] determining a turning action reward according to the posture of the legged robot after performing the candidate turning action, the candidate turning action, and the optimal turning action in the current iteration stage;

[0018] Adjusting the initial strategy of the turning action according to the turning action reward to obtain an optimized strategy of the turning action;

[0019] Re-determining a candidate turning action according to the stable posture of the legged robot in the current iteration stage and the turning action optimization strategy, until the re-determined candidate turning action maximizes the turning action reward;

[0020] The candidate turning action when the turning action reward is the largest is determined as the optimal turning action in the next iterative stage.

[0021] Optionally, the control module is further configured to:

[0022] determining a body turning reward according to the posture of the legged robot after performing the candidate turning action;

[0023] Determining an action smoothness reward according to the candidate turning action and the optimal turning action in the current iteration stage;

[0024] A turning action reward is determined according to the body turning reward and the action smoothness reward.

[0025] Optionally, the control module is further configured to:

[0026] determining a body rotation angle of the legged robot according to the posture of the legged robot after performing the candidate turning action;

[0027] A body turning reward is determined according to the body rotation angle.

[0028] Optionally, the control module is further configured to:

[0029] Get the reference gravity vector;

[0030] Vectorizing the body rotation angle to obtain a body rotation angle vector;

[0031] A body turning reward is obtained according to a Euclidean norm of a difference between the body rotation angle vector and the reference gravity vector.

[0032] Optionally, the control module is further configured to:

[0033] Obtaining a first action parameter of the candidate turning action;

[0034] Obtaining a second action parameter of the optimal turning action in the current iteration stage;

[0035] An action smoothness reward is determined according to the first action parameter and the second action parameter.

[0036] Optionally, the control module is further configured to:

[0037] calculating an action parameter difference between the first action parameter and the second action parameter;

[0038] The action smoothness reward is obtained according to the square of the Euclidean norm of the action parameter difference.

[0039] Optionally, the control module is further configured to:

[0040] Obtaining a first weight corresponding to the body turning reward and a second weight corresponding to the action smoothness reward;

[0041] The body turning reward and the action smoothness reward are weightedly summed according to the first weight and the second weight to obtain a turning action reward.

[0042] Optionally, the control module is further configured to:

[0043] Determining a first weight corresponding to the body turning reward according to the stable posture of the legged robot in the current iteration stage and the posture of the legged robot after performing the candidate turning action;

[0044] A second weight corresponding to the action smoothness reward is determined according to the smoothness of a process in which the legged robot executes the candidate turning action.

[0045] Optionally, the calling of the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information is calling the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction; in the process of calling the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction, an overall cumulative reward corresponding to the overall turning strategy direction is also obtained; the control module is further used to:

[0046] Based on the different overall turning strategy directions, re-execute the aforementioned legged robot fall and turn control method to obtain the overall cumulative rewards corresponding to the different overall turning strategy directions;

[0047] Determine the optimal overall turning action strategy according to the overall turning strategy direction with the largest overall cumulative reward;

[0048] The optimal overall turning-over action strategy is applied to the legged robot, so that the legged robot can turn over after falling.

[0049] Optionally, each iterative stage corresponds to a stage-optimal turning strategy, and the control module is further configured to:

[0050] The overall turnaround strategy direction with the largest overall cumulative reward is used as the target turnaround strategy direction;

[0051] Obtaining the optimal turning strategy for each stage corresponding to all iterative stages under the target turning strategy direction;

[0052] According to the optimal turning strategies of all iterative stages corresponding to the target turning strategy direction, an overall turning action strategy under the target turning strategy direction is constructed;

[0053] The overall turning action strategy under the target turning strategy direction is determined as the optimal overall turning action strategy.

[0054] On the other hand, an embodiment of the present application further provides a legged robot, comprising:

[0055] at least one processor;

[0056] at least one memory for storing at least one program;

[0057] When at least one of the programs is executed by at least one of the processors, the above-mentioned method for controlling the fall and turn over of the legged robot is implemented.

[0058] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program executable by a processor. When the processor-executable computer program is executed by the processor, it is used to implement the fall and turnover control method of the legged robot as described above.

[0059] On the other hand, an embodiment of the present application also provides a computer program product, including a computer program or computer instructions, wherein the computer program or the computer instructions are stored in a computer-readable storage medium, the processor of the legged robot reads the computer program or the computer instructions from the computer-readable storage medium, and the processor executes the computer program or the computer instructions, so that the legged robot executes the fall and turnover control method of the legged robot as described above.

[0060] The embodiments of the present application include at least the following beneficial effects: first, the falling posture information of the legged robot is obtained; then, the action selection model is called to perform a multi-stage iterative turning action prediction based on the falling posture information, and the optimal turning action of each iterative stage is obtained, and in each iterative stage, the legged robot is controlled to perform the optimal turning action of the current iterative stage, wherein the legged robot is in a stable posture after performing the optimal turning action of each iterative stage, and the legged robot successfully turns over after performing the optimal turning action of the last iterative stage; wherein, the action selection model is called to perform a multi-stage iterative turning action based on the falling posture information. When making predictions, the action selection model predicts the optimal turning motion for the next iteration based on the stable posture of the legged robot in the current iteration. This means that the goal of each iteration is to transition the legged robot from its current stable posture to the next stable posture until it successfully turns over. The end of each iteration means that the legged robot has reached a stable posture, providing new data for action prediction in the next iteration. This allows the legged robot to autonomously learn and adjust the optimal turning motion required to achieve the next stable posture based on the stable posture of each iteration, thereby enabling the legged robot to successfully turn over. Because the entire control process is implemented based on the turning motion prediction made by the action selection model, no manual design and debugging are required, which can effectively reduce manual design costs and has high applicability.

[0061] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained through the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0063] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0064] Figure 2 is a schematic diagram of another implementation environment provided by an embodiment of the present application;

[0065] Figure 3 is a schematic diagram of another implementation environment provided by an embodiment of the present application;

[0066] Figure 4 This is a flow chart of a method for controlling a legged robot falling and turning over provided in an embodiment of the present application;

[0067] Figure 5 This is a schematic diagram of action prediction of the action selection model provided in an embodiment of the present application;

[0068] Figure 6 Schematic diagram of the update optimization process of the action selection model provided in the embodiment of the present application;

[0069] Figure 7 Schematic diagram of the optimization process of the turning action optimization strategy provided in the embodiment of the present application;

[0070] Figure 8 This is a schematic diagram of the process of a legged robot performing a fall and turning over provided by an embodiment of the present application;

[0071] Figure 9 This is a schematic diagram of the calculation process of the body turning reward provided in an embodiment of the present application;

[0072] Figure 10 This is a schematic diagram of the effect of a legged robot falling and turning over provided by an embodiment of the present application;

[0073] Figure 11 This is a detailed flow chart of a method for controlling a legged robot falling and turning over, provided as a specific example;

[0074] Figure 12 is a detailed flow chart of a method for controlling a fall and rollover of a legged robot provided by another specific example;

[0075] Figure 13 1 is a schematic diagram of a fall and turn control device for a legged robot provided in an embodiment of the present application;

[0076] Figure 14 This is a schematic diagram of a legged robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be considered as limiting the present application. All other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0078] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0080] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0081] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0082] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field of computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI domains. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0083] Pose is used to characterize the posture of a legged robot in a specified coordinate system. Pose refers to the movement of the legged robot in space, and the posture information of the legged robot can be represented by a matrix.

[0084] In the related art, the legged robot's fall and turn motion scheme is generally pre-set manually, so that when the legged robot falls, it can control each joint to perform the corresponding turning movement according to the fall and turn motion scheme, so that it can stand up from the fallen state. However, the fall and turn motion scheme in this related art requires a long period of manual design and debugging, and most fall and turn motion schemes can only successfully turn over for specific fall postures. If the legged robot is to be able to turn over successfully in different fall postures, it is necessary to manually design and debug it for different fall postures separately, but this will undoubtedly greatly increase the manual design cost of the legged robot.

[0085] In order to effectively reduce the cost of manual design, the embodiment of the present application provides a method for controlling the fall and turning over of a legged robot, a device for controlling the fall and turning over of a legged robot, a legged robot, a computer-readable storage medium, and a computer program product. The method first obtains the fall posture information of the legged robot; then calls the action selection model to perform multi-stage iterative turning action prediction based on the fall posture information, obtains the optimal turning action in each iterative stage, and in each iterative stage, controls the legged robot to perform the optimal turning action of the current iterative stage, wherein the legged robot will be in a stable posture after performing the optimal turning action in each iterative stage, and the legged robot will turn over successfully after performing the optimal turning action in the last iterative stage. When the action selection model is called to perform multi-stage iterative turning action prediction based on the falling posture information, the action selection model predicts the optimal turning action for the next iterative stage based on the stable posture of the legged robot in the current iterative stage. This means that the goal of each iterative stage is to transform the legged robot from the current stable posture to the next stable posture until the turning is successful. The end of each iterative stage means that the legged robot has reached a stable posture, providing new data for the action prediction of the next iterative stage. This allows the legged robot to autonomously learn and adjust the optimal turning action required to achieve the next stable posture through the stable posture of each iterative stage, thereby enabling the legged robot to turn successfully. Since the entire control process is implemented based on the turning action prediction made by the action selection model, no manual design and debugging are required, thus effectively reducing the cost of manual design and having high applicability.

[0086] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1The implementation environment includes a control module 101 of the legged robot and multiple sensor modules installed on the legged robot, wherein the multiple sensor modules can measure the current posture information of the legged robot, and the control module 101 can communicate with each sensor module, so that the control module 101 can obtain the posture information of the legged robot through the multiple sensor modules. The control module 101 and each sensor module can be directly or indirectly connected through wired or wireless communication, and this embodiment does not specifically limit this. Among them, the legged robot may include three or more supporting feet, and the control module 101 may control the joints of each supporting foot separately. Specifically, the legged robot may include a left forefoot, a right forefoot, a left hind foot and a right hind foot. The control module 101 may control the joints of the left forefoot, the right forefoot, the left hind foot and the right hind foot to rotate or swing sideways. The control module 101 obtains the posture of the legged robot through multiple sensor modules, and then the control module 101 can control the rotation angle and swing amplitude of each supporting foot according to the posture information of the legged robot, so that the legged robot moves to the target posture required by the target, such as forward, backward, turning or turning over.

[0087] Reference Figure 1As shown, in an application scenario, the control module 101 can obtain the posture information of the legged robot in the current state through multiple sensor modules, and when the legged robot is in a falling state, the control module 101 can obtain the falling posture information of the legged robot in the current falling state through multiple sensor modules, and call the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information, and obtain the optimal turning action in each iterative stage. In each iterative stage, the control module 101 can control the legged robot to perform the optimal turning action in the current iterative stage, wherein After executing the optimal turning action in each iteration, the legged robot will be in a stable posture. For example, the optimal turning action can be achieved by simultaneously rotating the left and right forefoots so that the legged robot is in a stable posture with the knee joints of the left and right forefoots in contact with the ground; or by simultaneously rotating the left and right hindfoots so that the legged robot is in a stable posture with the knee joints of the left and right hindfoots in contact with the ground; or by swinging the left and left hindfoots so that the legged robot is in a stable posture with the left and left hindfoots in contact with the ground and the right and right hindfoots off the ground. When the legged robot is in a stable standing posture after executing the optimal turning action in an iteration, it can be considered that the turning action of the legged robot is successful, i.e., the iteration is the last iteration. When the action selection model is called to perform multi-stage iterative turning action prediction based on the falling posture information, the action selection model can predict the optimal turning action for the next iteration based on the stable posture of the legged robot in the current iteration. Therefore, the action selection model can continuously predict the stable posture of the legged robot after performing the turning action as the initial state, autonomously learn and adjust the action strategy to adapt to different falling postures, so that the legged robot can switch to the next stable posture after performing the turning action until the turning is successful.

[0088] Figure 2 This is a schematic diagram of another implementation environment provided by the embodiment of the present application. Figure 2The implementation environment includes a control module 201 of a legged robot and multiple sensor modules installed on the legged robot. The multiple sensor modules can measure the current posture information of the legged robot. The control module 201 can communicate with each sensor module, so that the control module 201 can obtain the posture of the legged robot through the multiple sensor modules. The control module 201 and each sensor module can be directly or indirectly connected through wired or wireless communication. This embodiment does not specifically limit this. The legged robot may include a robotic arm and two supporting legs. The control module 201 can control the joints of the robotic arm and the two supporting legs respectively. Specifically, the legged robot may include a left leg, a right leg, and at least one robotic arm. The legged robot contacts the ground through the left and right legs to maintain the legged robot in a standing posture. The control module 201 can control the joints of the left and right legs to rotate or swing sideways to control the legged robot to move forward, backward, and turn. The control module 201 can also control the joints of the robotic arms to move the legged robot to the desired posture and perform the target task.

[0089] Reference Figure 2As shown, in an application scenario, the control module 201 can obtain the posture information of the legged robot in the current state through the sensor module, and when the legged robot is in a falling state, the control module 201 can obtain the falling posture information of the legged robot in the current falling state through the sensor module, and call the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information to obtain the optimal turning action of each iterative stage, and in each iterative stage, the control module 201 can control the legged robot to perform the optimal turning action of the current iterative stage, wherein the legged robot will be in a stable posture after performing the optimal turning action of each iterative stage. For example, the optimal turning action can be by swinging and rotating the support arm so that the legged robot is in a stable posture with the knee joint of the left foot and the knee joint of the right foot contacting the ground, or by rotating the left foot and the right foot at the same time so that the legged robot is in a stable posture with the knee joint of the left foot and the knee joint of the right foot contacting the ground, or by rotating the left foot and the support arm so that the legged robot is in a stable posture with the knee joint of the left foot and the support arm contacting the ground and the knee joint of the right foot leaving the ground. When the legged robot is in a stable standing position after executing the optimal turning action in an iterative phase, the turning is considered successful, and the iterative phase is considered the final phase. When the action selection model is invoked to perform multi-stage iterative turning action prediction based on the falling posture information, the action selection model can predict the optimal turning action for the next iterative phase based on the stable posture of the legged robot in the current iterative phase. Therefore, the action selection model can continuously predict the stable posture of the legged robot after performing the turning action as the initial state, autonomously learn and adjust the action strategy to adapt to different falling postures, and ensure that the legged robot transitions to the next stable posture after performing the turning action until the turning is successful.

[0090] Figure 3 This is a schematic diagram of another implementation environment provided by the embodiment of the present application. Figure 3 The implementation environment includes a control module 301 of the legged robot, a posture detection terminal 302 for measuring the current posture information of the legged robot, and a server 303, wherein the control module 301, the posture detection terminal 302 and the server 303 are directly or indirectly connected through wired or wireless communication, which is not specifically limited in this embodiment.

[0091] In one embodiment, the posture detection terminal 302 can detect the posture information of the legged robot in its current environment. For example, the posture detection terminal 302 can generate a thermal image of the legged robot using thermal imaging technology, analyze temperature changes in different parts of the legged robot, and obtain the posture information of the legged robot. Alternatively, the posture detection terminal 302 can analyze visual information of the legged robot's environment using computer vision technology to measure the current posture information of the legged robot. This embodiment does not specifically limit this. After detecting and obtaining the posture information of the legged robot, the posture detection terminal 302 can upload the posture information to the server 303, which then transmits the posture information of the legged robot to the control module 301.

[0092] Server 303 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0093] In one embodiment, the legged robot may include at least two supporting feet, and the control module 301 may control the joints of each supporting foot separately. Specifically, the legged robot contacts the ground through each supporting foot to maintain the legged robot in a standing posture, and the control module 301 may control the joints of each supporting foot to rotate or swing sideways to control the legged robot to move forward, backward and turn.

[0094] Reference Figure 3As shown, in one application scenario, the posture detection terminal 302 can detect the posture information of the legged robot in its current state and upload this posture information to the server 303. The server 303 then sends this posture information to the control module 301 of the legged robot. When the legged robot is in a falling state, the control module 301 can obtain the falling posture information of the legged robot from the posture detection terminal 302 through the server 303, and call the action selection model to perform a multi-stage iterative turning action prediction based on the falling posture information to obtain the optimal turning action for each iterative stage. In each iterative stage, the control module 301 can control the legged robot to perform the optimal turning action for the current iterative stage. After performing the optimal turning action in each iterative stage, the legged robot will be in a stable posture. For example, the optimal turning action can be to swing the supporting foot so that the legged robot is in a stable posture with the knee joint of the supporting foot in contact with the ground, or to rotate the supporting foot so that the legged robot is in a stable posture with the knee joint of part of the supporting foot in contact with the ground and the knee joint of the remaining supporting foot off the ground. When the legged robot is in a stable posture, the posture detection terminal 302 can again detect and obtain the posture information of the legged robot in its current state and upload this posture information to the server 303. The control module 301 of the legged robot can then use the posture information of the stable posture to iteratively predict the turning action. When the legged robot is in a stable standing posture after performing the optimal turning action in an iterative phase, it can be considered that the turning action has been successful, i.e., the iterative phase is the last iterative phase. When the action selection model is called to perform multi-stage iterative turning action prediction based on the falling posture information, the action selection model can predict the optimal turning action for the next iterative phase based on the stable posture of the legged robot in the current iterative phase. Therefore, the action selection model can continuously predict the action of the legged robot using the stable posture after performing the turning action as the initial state, autonomously learn and adjust the action strategy to adapt to different falling postures, so that the legged robot can transition to the next stable posture after performing the turning action until the turning action is successful.

[0095] Figure 4 This is a flow chart of a legged robot fall and turn control method provided in an embodiment of the present application. The legged robot fall and turn control method can be executed by the control module of the legged robot, or can be executed by the control module of the legged robot and the server together. In the embodiment of the present application, the method is described as being executed by the control module of the legged robot. Figure 4 The method for controlling the fall and turn over of the legged robot includes but is not limited to steps 410 to 420.

[0096] Step 410: Acquire the falling posture information of the legged robot;

[0097] Step 420: Call the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information, obtain the optimal turning action in each iterative stage, and in each iterative stage, control the footed robot to perform the optimal turning action of the current iterative stage, wherein the footed robot will be in a stable posture after performing the optimal turning action in each iterative stage, and the footed robot will successfully turn over after performing the optimal turning action in the last iterative stage.

[0098] In one embodiment, the falling posture information may refer to the posture information of the legged robot in a falling state, and the falling state of the legged robot may refer to a state in which the legged robot deviates significantly from a pre-designed normal standing posture or working posture; the posture information of the legged robot may include the rotation angle of the legged robot flipping forward and backward or left and right, the joint acceleration, the force condition of the supporting foot, etc., wherein the legged robot may be installed with a sensor module, and the falling posture information of the legged robot may be obtained through the sensor module. Specifically, the sensor module may include an inertial measurement unit, a visual sensor, a laser radar or a force sensor, wherein the inertial measurement unit, the visual sensor, the laser radar and the force sensor are all commonly used sensors in this field, and the working principles of these sensors can refer to the relevant introduction and description in the relevant technology, and will not be described in detail here. The inertial measurement unit can be used to detect the acceleration, angular velocity and gravity direction readings of the legged robot, and then the posture information of the legged robot can be estimated from the data measured by the inertial measurement unit through a data fusion algorithm, such as Kalman filtering, complementary filtering or nonlinear state estimation method. Among them, the principles and processes of Kalman filtering, complementary filtering and nonlinear state estimation methods can be referred to the relevant introduction and description in the relevant technology, and will not be described in detail here; or, the current surrounding environment of the legged robot is perceived by a visual sensor, and compared with the ground features to estimate the current posture information; or, the current force condition of the legged robot is detected by a tactile sensor, and the pressure distribution of each joint is judged to estimate the posture information of the legged robot.

[0099] In one embodiment, the data measured by the sensor module can be used to determine whether the legged robot is in a falling state, and then determine whether the obtained posture information is falling posture information. Among them, by comparing the posture information with a pre-set parameter threshold, when a data value in the posture information exceeds the set parameter threshold, it can be considered that the legged robot is currently in an abnormal falling state, and the posture information in the current state can be used as falling posture information. For example, by comparing whether the rotation angle of the legged robot flipping back and forth is greater than the angle threshold corresponding to the standing posture, if the rotation angle of the legged robot flipping back and forth is greater than the angle threshold corresponding to the standing posture, it can be considered that the legged robot is currently in a falling state; or, by comparing the changing trend of the posture information, it can be determined whether the legged robot is in a falling state. When the legged robot is in a falling state, When a robot falls, due to the effect of gravity acceleration, the joint acceleration will change sharply in one or more directions, and the speed and displacement data of some joints will suddenly increase or decrease. Therefore, it is possible to compare whether the change amplitude of the joint acceleration, speed and displacement data exceeds the set change amplitude threshold; in addition, the force conditions of each joint of the legged robot can be judged. When there is no force on the soles of all supporting feet or a large area of ​​force is applied to parts other than the soles of the supporting feet, it can be considered that the legged robot is currently in a falling state, and the current posture information can be used as the falling posture information.

[0100] In one embodiment, the falling posture information of the legged robot can be measured by a sensor module installed on the robot itself, or it can be obtained by an external posture detection terminal by analyzing the data of the legged robot in the current environment, and the posture information of the legged robot is returned to the legged robot, so that the legged robot can act based on the posture information. At the same time, the posture detection terminal can determine whether the legged robot is in a falling posture based on the posture information. When it is determined that the legged robot is in a falling state, the posture detection terminal can return the current posture information of the legged robot as falling posture information to the legged robot, and notify the legged robot to perform a falling and turning action.

[0101] In one embodiment, the action selection model can be a network model pre-trained based on pre-obtained posture-action sample pairs. When using this action selection model to predict a turning action based on posture information, the posture information of the legged robot can first be feature extracted to obtain key features of the legged robot's posture information. The turning action can then be predicted based on these key features, and the desired turning action can be output. It should be noted that the action selection model involved in this embodiment can be constructed using a network such as a deep neural network or a convolutional neural network. The model structure and operating principles of deep neural networks and convolutional neural networks can be referenced in the relevant art and will not be detailed here.

[0102] In one embodiment, the action selection model can also be a selection model composed of a non-neural network. A pre-set reward function can be stored in the action selection model. At the same time, the action selection model can select multiple executable actions based on the input foot-type robot posture information, calculate the reward value of each executable action through the reward function, and select the executable action with the highest reward value as the optimal turning action.

[0103] In one embodiment, referring to Figure 5 , Figure 5 This is a schematic diagram of the action prediction of the action selection model provided in the embodiment of the present application. Figure 5 In this context, the action selection model can refer to the aforementioned non-neural network selection model that selects the optimal turning action based on the current legged robot's fall state information. The action selection model can be pre-deployed in the legged robot's control module or sent by a server. In the scenario where the legged robot falls and turns over, the action selection model's goal is to help the legged robot recover from the fallen position to a standing position. The action selection model can perform multi-stage iterative turning action prediction based on the fall information, obtaining the optimal turning action for each iterative stage. In each iterative stage, the legged robot is controlled to execute the optimal turning action for the current iterative stage. After executing the optimal turning action in each iterative stage, the legged robot is in a stable posture. The action selection model then predicts the optimal turning action for the next iterative stage based on the stable posture of the legged robot in the current iterative stage, until the legged robot successfully turns over. The end of each iterative stage means that the legged robot is in a stable posture after executing the turning action.

[0104] In one embodiment, the action selection model outputs multiple turning actions during each iteration of the turning action prediction. Each turning action is associated with a reward value, and the optimal turning action is the one with the highest reward value among these multiple turning actions. The turning action can be an action trajectory consisting of a series of turning postures. During the process of performing a turning action, the legged robot may be in multiple turning postures, each of which corresponds to a moment, and the turning postures corresponding to all these moments constitute the turning action.

[0105] It should be noted that the stable posture is a randomly selected target posture from a series of turning postures that constitute the optimal turning action. Each moment in the process of the legged robot performing the optimal turning action corresponds to a turning posture. For each moment, the turning posture corresponding to that moment is determined, so for that moment, the turning posture is a posture with a certain action or a stable action. Therefore, the turning posture corresponding to each moment in the process of the legged robot performing the optimal turning action can be considered as the stable posture at that moment. For example, the stable posture can be the target posture corresponding to the legged robot when it begins to perform the optimal turning action, which is equivalent to the target posture corresponding to the first moment in the motion trajectory of the optimal turning action; the stable posture can also be the target posture corresponding to the legged robot when it completes the optimal turning action, which is equivalent to the target posture corresponding to the last moment in the motion trajectory of the optimal turning action.

[0106] like Figure 5As shown, when the legged robot is in a falling state, the falling posture information is obtained, and the action selection model is called to perform a multi-stage iterative turning action prediction based on the falling posture information. In the first iterative stage, the action selection model can perform a turning action prediction based on the falling posture information, and obtain the optimal turning action A1 of the first iterative stage; after the legged robot performs the optimal turning action of the first iterative stage, it is judged whether the current legged robot is in a stable posture of a standing posture. If so, it is considered that the turning is successful and the turning action prediction is stopped. Otherwise, the action selection model randomly selects a stable posture S1 (or target posture S1) from the optimal turning action of the first iterative stage as the initial posture of the next iterative stage to perform a turning action prediction of the next iterative stage, and obtain the optimal turning action A2 of the second iterative stage. Therefore, after the legged robot performs the optimal turning action A2 of the second iterative stage, it is judged whether the current legged robot is in a stable posture of a standing posture. If , then the turning over is considered successful and the turning over action prediction is stopped. Otherwise, the action selection model randomly selects a stable posture S2 (or target posture S2) from the optimal turning over action in the second iterative stage as the initial posture for the next iterative stage to predict the turning over action in the next iterative stage, and obtains the optimal turning over action A3 for the third iterative stage. After the legged robot executes the optimal turning over action A2 in the third iterative stage, it continues to judge whether the current legged robot is in a stable posture of standing posture. If so, the turning over is considered successful and the turning over action prediction is stopped. Otherwise, the action selection model randomly selects a stable posture S3 (or target posture S3) from the optimal turning over action in the third iterative stage as the initial posture for the next iterative stage to predict the turning over action in the next iterative stage, until the legged robot is in a stable posture of turning over and standing after executing the optimal turning over action AN in the Nth iterative stage. At this time, the Nth iterative stage can be determined as the last iterative stage. Since the optimal turning motion for each iteration is predicted based on the current legged robot's posture, the optimal turning motion for the current iteration can achieve a stable turning-over-standing posture for the legged robot. Therefore, the motion selection model can discover and explore the stable postures that the legged robot experiences during the transition from falling to turning over and standing, and use these stable postures to predict the next turning motion. Since the legged robot spends most of the time in an unbalanced state during the transition from falling to turning over and standing, it is difficult to maintain stability for long periods of time. For example, performing a partial turning motion can bring the legged robot's posture close to standing, but it cannot maintain a stable posture, and its posture changes rapidly within a short period of time. Therefore, it is difficult to predict turning motions in these unbalanced states to derive a motion that is helpful for the legged robot to turn over and stand.In this embodiment, the action selection model focuses on the stable posture of the legged robot after performing a rollover, rather than the rollover itself. This helps achieve a more accurate rollover action, allowing the robot's stable posture after performing the optimal rollover to be closer to its standing stable posture. The robot then autonomously learns and adjusts the optimal rollover action required to achieve the next stable posture, ultimately enabling the robot to successfully roll over. Because the entire control process is based on the rollover action predictions made by the action selection model, no manual design and debugging are required, effectively reducing manual design costs and increasing applicability.

[0107] In one embodiment, the optimal turning over action may refer to a sequence of actions required to help the legged robot stably turn over and stand or be in a stable posture closer to turning over and standing under the current posture of the legged robot. The optimal turning over action may vary depending on factors such as the design of the legged robot, the specific posture it is currently in, and the range of motion of the joints. Specifically, the optimal turning over action may include independent actions of each movable joint of the legged robot. For example, the legged robot includes a left forefoot, a right forefoot, a left hindfoot and a right hindfoot. The optimal turning over action in the first iterative stage may be the rotation of the left forefoot and the left hindfoot. After the current legged robot executes the optimal turning over action in the first iterative stage, the legged robot is in a stable posture in which the knee joint of the left forefoot and the knee joint of the left hindfoot are both in contact with the ground. Therefore, based on the current stable posture of the legged robot, a turning over action prediction is performed, and the optimal turning over action obtained can be obtained by swinging the right forefoot and the right hindfoot, so that after the legged robot executes the optimal turning over action, the right forefoot and the right hindfoot are both in contact with the ground, while the left forefoot and the left hindfoot are suspended in the air. Therefore, it can be seen that the optimal turning over action is different based on different postures, which can enable the legged robot to switch from one stable posture to another.

[0108] In one embodiment, the action selection model is called to perform multi-stage iterative turning action prediction based on the falling posture information, and the optimal turning action in each iterative stage can be obtained. After executing the optimal turning action in each iterative stage, the footed robot can be in a stable posture. Therefore, the optimal turning action in each iterative stage and the stable postures before and after the optimal turning action are executed can be recorded, and the action selection model is updated and optimized using the optimal turning action and the two stable postures before and after the optimal turning action is executed. For example, the action selection model predicts the current optimal turning action Aa based on the stable posture Sa (or target posture Sa) before executing the optimal turning action, indicating that the confidence of the optimal turning action Aa is the highest, or it can be understood that the reward value of the optimal turning action Aa is the highest. Therefore, the stable posture Sb (or target posture Sb) of the legged robot after executing the optimal turning action Aa can be evaluated, and the learning rate of the action selection model for selecting the optimal turning action Aa in the stable posture Sa can be adjusted. If the turning effect of the stable posture Sb of the legged robot after executing the optimal turning action Aa does not reach the turning effect predicted by the action selection model, the learning rate of selecting the optimal turning action Aa in the stable posture Sa can be reduced. The turning effect can be represented by the degree of difference between the actual posture information and the target standing posture information. The smaller the degree of difference between the actual posture information and the target standing posture information, the higher the turning effect, and the greater the degree of difference between the actual posture information and the target standing posture information, the lower the turning effect.

[0109] In one embodiment, referring to Figure 6 , Figure 6Schematic diagram of the update optimization process of the action selection model provided in an embodiment of the present application. When the legged robot is in the current stable posture S1, the action selection model predicts a turning action based on the stable posture S1, predicts the optimal turning action A1, and believes that the legged robot can reach the stable posture S2 after executing the optimal turning action A1 in the stable posture S1. The action selection model evaluates the reward return value R1 for the legged robot in the stable posture S2 relative to the process of the legged robot falling and turning over. In actual conditions, after the legged robot executes the optimal turning action A1 in the stable posture S1, the legged robot is actually in the stable posture S3, and the action selection model evaluates the reward return value R2 for the legged robot in the stable posture S3 relative to the process of the legged robot falling and turning over. By comparing the reward return value R1 of the stable posture S2 with the reward return value R2 of the stable posture S3 and calculating the difference between the two, the learning rate of the action selection model for executing the optimal turning action A1 in the stable posture S1 is adjusted. If the reward return value R1 of the stable posture S2 is less than the reward return value R2 of the stable posture S3, it can be considered that the turning effect of the optimal turning action A1 performed in the stable posture S1 is not ideal, and the learning rate needs to be reduced so that the remaining turning actions can be explored when the legged robot is in the stable posture S1 later; if the reward return value R1 of the stable posture S2 is greater than the reward return value R2 of the stable posture S3, it can be considered that the turning effect of the optimal turning action A1 performed in the stable posture S1 exceeds the ideal effect, and the learning rate can be increased, and the amplitude of the learning rate adjustment can be determined according to the difference between the reward return value R1 of the stable posture S2 and the reward return value R2 of the stable posture S3. It should be noted that, when executing the optimal turning action in each iterative stage, the optimal turning action can be evaluated when the legged robot is in a stable posture, and the action selection model can be optimized and updated. Alternatively, after the legged robot executes the optimal turning action in the last iterative stage and turns over successfully, the optimal turning action in each iterative stage can be evaluated based on the overall turning process of the legged robot, and then the action selection model can be optimized and updated.

[0110] In one embodiment, in the process of the action selection model predicting the optimal turning action in the next iterative stage based on the stable posture of the legged robot in the current iterative stage, an initial turning action strategy can be determined first; then, based on the stable posture of the legged robot in the current iterative stage and the initial turning action strategy, a candidate turning action is determined, and the legged robot is controlled to perform the candidate turning action; then, based on the posture of the legged robot after performing the candidate turning action, the candidate turning action and the optimal turning action in the current iterative stage, a turning action reward is determined; based on the turning action reward, the initial turning action strategy is adjusted to obtain the turning action optimization strategy; then, based on the stable posture of the legged robot in the current iterative stage and the turning action optimization strategy, a candidate turning action is re-determined until the re-determined candidate turning action maximizes the turning action reward; the candidate turning action when the turning action reward is maximized is determined as the optimal turning action in the next iterative stage. Since the turning action is determined based on the current initial turning action strategy and the stable posture of the legged robot, the turning action is applicable to the stable posture of the legged robot and can provide a good turning effect. Moreover, by continuously selecting turning actions for testing and guiding the continuous adjustment of the initial turning action strategy by calculating the turning action reward, the selected turning action can be optimized towards the goal of turning over and standing up, providing clear directionality for the optimization and adjustment process of the initial turning action strategy and the selection of turning actions. In addition, after determining a candidate turning action, the legged robot can be simulated and controlled to perform the candidate turning action in a simulation environment to deduce the posture of the legged robot after performing the candidate turning action, which can reduce the cost and risk of conducting a large number of action tests in the actual environment. It should be noted that in the process of controlling the legged robot to perform the candidate turning action, the control module of the legged robot performs the candidate turning action in a simulation environment constructed based on the current environment of the legged robot.

[0111] In one embodiment, the initial strategy for the turning action may refer to a strategy in the current iteration phase, i.e., the initial strategy for the turning action may be determined based on the stable posture of the legged robot in the current iteration phase; or, the initial strategy for the turning action may be the turning action optimization strategy in the previous iteration phase. Both the initial strategy for the turning action and the optimization strategy for the turning action are used to enable the legged robot to transition from one stable posture to another. Based on the stable posture of the legged robot in the current iteration phase and the initial strategy for the turning action, a candidate turning action may be determined. In the process of determining the candidate turning action, the turning actions that can be executed by the legged robot may be screened based on the strategy direction of the initial strategy for the turning action to obtain a candidate turning action that is suitable for the legged robot to execute in the current stable posture. Specifically, the reward value of each executable turning action may be predicted, and the turning action with the largest predicted reward value may be selected as the candidate turning action; or, the predicted reward value of each executable turning action may be used as its probability value, and a turning action may be randomly selected from each executable turning action as the candidate turning action.

[0112] In one embodiment, after the legged robot is controlled to perform the candidate turning action, the legged robot will be in a stable posture. Therefore, the turning action reward can be determined based on the posture of the legged robot after performing the candidate turning action, the candidate turning action and the optimal turning action in the current iteration stage. The turning action reward refers to the signal fed back based on the legged robot's action performance when the legged robot performs the candidate turning action. It is used to guide the legged robot to learn to effectively complete the behavior of turning over and standing, and drive the action selection model to autonomously discover and optimize the turning action during the learning process. Specifically, the action performance of the legged robot can be understood as the stable posture of the legged robot after the legged robot performs the candidate turning action, as well as the fluency, complexity and stability of the legged robot in performing the candidate turning action. For example, after performing the candidate turning action, if the posture of the legged robot deviates more from the target standing posture than the posture in the current iteration stage, it is considered to reduce the turning action reward; if the legged robot performs the candidate turning action with high fluency and low complexity, that is, the legged robot can complete the candidate turning action in a shorter time, it can be considered to increase the turning action reward; if the legged robot performs the candidate turning action with poor stability, that is, the legged robot becomes unstable or falls again when performing the candidate turning action, the turning action reward can be reduced.

[0113] In one embodiment, referring to Figure 7 , Figure 7This is a schematic diagram of the optimization process of the turning action optimization strategy provided by the embodiment of the present application. Based on the current initial strategy for the turning action, the candidate turning actions of the stable posture of the legged robot in the current iteration stage are determined, the legged robot is controlled to perform the candidate turning actions in the simulation environment, and the posture of the legged robot after performing the candidate turning action in the simulation environment is simulated and deduced, and then the turning action reward of the candidate turning action is determined. In the process of predicting and selecting the candidate turning actions, the action selection model selects the turning action with the largest predicted reward value as the candidate turning action based on the current initial strategy for the turning action. Therefore, the action selection model can be corrected by comparing the value error between the predicted reward value and the actual turning action reward, and the initial strategy for the turning action is optimized to obtain the turning action optimization strategy. As the turning action optimization strategy is continuously iteratively optimized, the more selectable candidate turning actions of the stable posture of the legged robot in the current iteration stage, the more postures can be changed, such as Figure 7 As shown, based on the turning action optimization strategy and the stable posture of the legged robot in the current iteration stage, the candidate turning action determined can enable the legged robot to be in three different stable postures after executing the candidate turning action; and in the third iteration of the initial turning action strategy, that is, after the iterative optimization of the turning action optimization strategy, the legged robot can be in six different stable postures instead of the original three after executing the candidate turning action, thereby being able to explore and discover multiple different stable postures, which helps to speed up the turning efficiency of the legged robot.

[0114] In one embodiment, the turning action optimization strategy is gradually improved through interaction with the environment. Starting from the initial turning action strategy, the strategy is adjusted and updated based on the rewards obtained by the legged robot from interacting with the environment after executing a candidate turning action. Therefore, the initial turning action strategy can be adjusted using the turning action reward. Specifically, the learning rate of the initial turning action strategy for the candidate turning actions can be adjusted using the turning action reward to balance the exploration of new turning actions with the utilization of the currently known optimal turning actions. Furthermore, the turning action reward can be updated using a model-free reinforcement learning method such as Q-Learning to update the Q-value. The Q-value represents the expected reward of executing a specific action in a given state, i.e., the expected action reward of executing the candidate turning action in the stable posture of the legged robot in the current iteration. Therefore, the Q-value can be updated using the actual turning action reward obtained after the legged robot executes the candidate turning action, correcting the expected action reward of executing the candidate turning action in the stable posture of the legged robot in the current iteration, so that the candidate turning action selected by the turning action optimization strategy can help the legged robot complete a turning action.

[0115] In one embodiment, referring to Figure 8 , Figure 8 This is a schematic diagram of the process of a legged robot performing a fall and turning over provided in an embodiment of the present application. Figure 8 (a) shows the process of a legged robot performing a fall and turning over in the related technology. The dotted line shows the turning motion trajectory that is preset manually. When the legged robot is in a fallen state, the joints are first swung or rotated until the legged robot moves to a specified initial posture S1 in the preset turning motion trajectory. Then, the turning motion is performed according to the turning motion trajectory, so that the legged robot stands up from the fallen state. However, this turning motion trajectory requires manual design and consumes a lot of time and energy. At the same time, it is impossible to start turning over from any falling state. The legged robot must be controlled to move to the specified initial posture S1 before it can start turning over. Therefore, the turning process takes a long time and has low applicability. Figure 8 (b) represents the process of the legged robot performing a fall and turn over under the fall and turn over method of the legged robot provided by the embodiment of the present application. The dotted line represents the turn over motion trajectory obtained by the legged robot after multi-stage iterative optimization. For example, in the case of posture S1, the legged robot can use the optimized turn over motion optimization strategy based on the stable posture of the legged robot in the current iterative stage to continuously determine the candidate turn over motions such as Figure 8(b) The turning actions A1 to A5 shown in (b) are performed until the re-determined candidate turning action A3 maximizes the turning action reward, and the candidate turning action A3 when the turning action reward is maximized is determined as the optimal turning action for the next iterative stage. Therefore, when the footed robot is in posture S1, it executes the optimal turning action A3, and randomly selects a stable posture S2 from the optimal turning action A3 as the initial posture for the next iterative stage; then, in the case of the stable posture S2, action exploration and strategy optimization are performed for the stable posture S2, and the candidate turning action A8 with the largest turning action reward is selected from turning actions A6 to A8 as the optimal turning action for the next iterative stage. After the footed robot executes the optimal turning action A8, a stable posture S3 is randomly selected from the optimal turning action A8 as the initial posture for the next iterative stage to predict the turning action, and so on, until the posture of the footed robot is transformed to the turning-over standing posture. By continuously optimizing the turning action optimization strategy, invalid action exploration is gradually reduced, so that each turning attempt can be closer to the actual application requirements, improving learning efficiency and turning efficiency. At the same time, by selecting the turning action with the largest turning action reward, the legged robot can learn the optimal action sequence for turning over under the stable posture of the current iteration stage, thereby improving the probability of successful turning over, and can independently discover the turning action trajectory that enables the legged robot to successfully turn over; and since the selection of each optimal turning action and the optimization of the turning action optimization strategy are all based on the stable posture of the current iteration stage, it can ensure that the legged robot can still find an effective turning strategy and select the optimal turning action when it is in different postures. Therefore, when the legged robot is in different falling postures, there is no need to uniformly reset to the pre-set initial posture and then perform the turning action according to the preset turning action trajectory. The turning action can be directly performed based on the current posture.

[0116] In one embodiment, in the process of re-determining a candidate turning action according to the stable posture of the legged robot in the current iteration stage and the turning action optimization strategy, until the re-determined candidate turning action maximizes the turning action reward, a threshold value for the number of explorations of the candidate turning action can be set, that is, when the number of re-determined candidate turning actions reaches the threshold value for the number of explorations, the re-determination of the candidate turning action is stopped, and the candidate turning action with the largest turning action reward is selected from the multiple candidate turning actions selected historically as the optimal turning action; or, since the turning action optimization strategy is continuously updated, the turning action reward corresponding to the re-determined candidate turning action can be continuously improved. If the turning action reward of the currently re-determined candidate turning action is less than the turning action reward of the previously determined candidate turning action, it can be considered that the re-determined candidate turning action maximizes the turning action reward.

[0117] In one embodiment, when determining a rollover reward based on the legged robot's posture after executing a candidate rollover action, the candidate rollover action, and the optimal rollover action in the current iteration, a body turning reward can be first determined based on the legged robot's posture after executing the candidate rollover action; a motion smoothness reward can then be determined based on the candidate rollover action and the optimal rollover action in the current iteration; and finally, a rollover reward can be determined based on the body turning reward and the motion smoothness reward. In the process of learning to roll over after a fall, the rollover reward is key to guiding the legged robot to quickly roll over. To optimize the rollover action, a composite reward function consisting of different components can be established, including a body turning reward and a motion smoothness reward. The body turning reward and the motion smoothness reward jointly determine the rollover reward for the rollover action.

[0118] In one embodiment, a body turning reward is determined based on the stable posture of the legged robot after executing a candidate turning action. The goal of the body turning reward is to enable the legged robot to turn over from a fallen position to a standing position. The body turning reward is determined by the stable posture. The closer the posture of the legged robot after executing the candidate turning action is to the ideal standing position, the greater the body turning reward, thereby encouraging the legged robot to execute a turning action that effectively turns to the ideal standing position. Specifically, the body turning reward can be determined by comparing the orientation angle of the legged robot after completing the candidate turning action with the orientation angle of the standing position. The body turning reward can be determined by comparing the angular difference between the orientation angle after completing the candidate turning action and the orientation angle of the standing position.

[0119] In one embodiment, the motion smoothness reward is determined by comparing the difference between the current turning action and the optimal turning action in the previous iteration stage. The motion smoothness reward encourages the legged robot to perform overly smooth turning actions and reduces the legged robot from performing rapid movements or abrupt movements. Smooth turning actions can reduce mechanical loss and energy loss, and help avoid excessive stress and control errors, thereby protecting the mechanical structure of the legged robot.

[0120] In one embodiment, a rollover reward is derived by integrating the body steering reward and the motion smoothness reward. This factor in the legged robot's posture change effect and motion smoothness to comprehensively evaluate the overall performance of a candidate rollover. This can also help select a more accurate rollover when multiple satisfying rollovers exist. Therefore, the rollover reward reflects whether a candidate rollover ensures both steering accuracy and smoothness. This provides an optimization target for rollover optimization strategies, helping legged robots perform rollovers from falls in various postures and enhancing the practicality and reliability of rollovers in complex environments.

[0121] In one embodiment, when determining a body turning reward based on the posture of a legged robot after executing a candidate turning action, the legged robot's body rotation angle can be first determined based on the posture of the legged robot after executing the candidate turning action; then, the body turning reward can be determined based on the body rotation angle. After the legged robot executes the candidate turning action, the relative position and orientation of the legged robot's torso with respect to the ground in the current posture is measured to determine the body rotation angle of the legged robot. In this case, the body rotation angle represents a quantified result of the legged robot's posture. For example, the body rotation angle can be calculated as the angle between the direction vector horizontal to the ground and the direction vector of the legged robot's torso's orientation. Therefore, when determining the body turning reward based on the body rotation angle, the body turning reward can be calculated based on the difference between the body rotation angle of the legged robot and the quantified result in the standing posture, i.e., the initial body angle. The smaller the difference between the body rotation angle and the initial body angle, the closer the legged robot's orientation is to the orientation in the standing posture, and thus the greater the body turning reward.

[0122] In addition, a first orientation of the trunk of the legged robot in the posture of the current iteration stage can be obtained, and after the legged robot performs the candidate turning action, a second orientation of the trunk of the legged robot in the current posture can be obtained. Based on the difference between the angle of the first orientation and the angle of the second orientation, the body rotation angle of the legged robot is determined. In this case, the body rotation angle represents the degree of change in the posture of the legged robot after the legged robot performs the candidate turning action. Therefore, the body turning reward can also be determined based on the degree of change in the posture of the legged robot, wherein the body turning reward can be adjusted according to different iteration stages. For example, when the legged robot just starts to perform the turning action, that is, in the initial iteration stage, the greater the degree of posture change, the greater the body turning reward can be, thereby encouraging posture change. If the posture of the legged robot is close to the standing posture, the body turning reward can be adjusted so that the smaller the degree of posture change, the greater the body turning reward, thereby reducing posture change and avoiding excessive movement.

[0123] In one embodiment, in the process of determining the body turning reward based on the body rotation angle, a reference gravity vector can be obtained first; then, the body rotation angle is vectorized to obtain the body rotation angle vector; and then, the body turning reward is obtained based on the Euclidean norm of the difference between the body rotation angle vector and the reference gravity vector.

[0124] In one embodiment, referring to Figure 9 , Figure 9This is a schematic diagram of the calculation process for the body steering reward provided in an embodiment of the present application. A body coordinate system for the legged robot is established with the center of the legged robot's trunk as the origin, the axis parallel to the legged robot's forward direction as the X-axis, the axis perpendicular to the X-axis in the horizontal plane and parallel to the right of the X-axis as the Y-axis, and the axis perpendicular to the plane of the X-axis and pointing upward as the Z-axis. The reference gravity vector refers to the unit vector of gravity in the body coordinate system of the legged robot in a standing posture, which can be understood as a unit vector pointing downward along the Z-axis. The reference gravity vector is measured when the legged robot is in a stable standing posture, free from motion interference or external interference. The body rotation angle is the relative direction between the body and the ground in the current posture. The body rotation angle vector is obtained by vectorizing the body rotation angle. The body rotation angle vector can represent the unit vector of the gravity in the body coordinate system of the legged robot in the current posture. Then, by calculating the difference between the body rotation angle vector and the reference gravity vector and calculating the Euclidean norm of the difference vector, the straight-line distance between the two vectors of the body rotation angle vector and the reference gravity vector of the legged robot is obtained. This can accurately measure the change in the orientation of the legged robot, effectively quantify the deviation between the posture change of the legged robot and the ideal standing posture, and obtain a more accurate body turning reward. Specifically, the calculation formula of the body turning reward is shown in the following formula (1):

[0125] r o =-||e g -[0,0,-1]|| (1)

[0126] Among them, r o Expressed as the body turning reward, e g It is represented as the body rotation angle vector, and [0,0,-1] is represented as the reference gravity vector.

[0127] In one embodiment, in determining a motion smoothness reward based on a candidate turning action and the optimal turning action in the current iteration, the first motion parameter of the candidate turning action can be first obtained; then the second motion parameter of the optimal turning action in the current iteration can be obtained; and then the motion smoothness reward can be determined based on the first motion parameter and the second motion parameter. The first motion parameter is used to describe the specific characteristics of the candidate turning action, such as joint motion angle, joint motion speed, duration, etc. Correspondingly, the second motion parameter is used to describe the specific characteristics of the optimal turning action in the current iteration. The first motion parameter and the second motion parameter are both represented by an action parameter matrix. The motion smoothness reward is determined by evaluating the coherence between the first motion parameter and the second motion parameter, and the coherence of the action is then used to determine the motion smoothness reward. Turning actions with high coherence will receive a higher value of the motion smoothness reward. The motion smoothness reward encourages the turning action strategy to generate smooth action sequences, reduce drastic changes, improve motion coherence and stability, help provide a stable learning process, maintain good performance in dynamic and complex environments, and facilitate rapid strategy convergence to obtain the candidate turning action with the highest turning action reward.

[0128] In one embodiment, in the process of determining the action smoothing reward based on the first action parameter and the second action parameter, the action parameter difference between the first action parameter and the second action parameter can be calculated first; then, the action smoothing reward is obtained based on the square of the Euclidean norm of the action parameter difference. The first action parameter and the second action parameter can be represented by a parameter vector, wherein the parameter vector can be multidimensional. Therefore, the action parameter difference between the first action parameter and the second action parameter can also be a parameter vector, and the action smoothing reward is calculated by the Euclidean norm of the negative exponent. The specific calculation formula of the action smoothing reward is shown in the following formula (2):

[0129] r a =-‖a t -a t-1 ‖ 2 (2)

[0130] Among them, r a Represented as the body turning reward, a t The parameter vector represented as the first action parameter, a t-1 The parameter vector of the second action parameter is represented as follows. Therefore, when the parameter vector of the first action parameter is very close to the parameter vector of the second action parameter, that is, the difference in action parameters is small, the smaller the Euclidean norm value is, and the setting of a negative exponent will lead to a larger action smoothing reward. Conversely, the larger the difference in action parameters, the smaller the action smoothing reward is.

[0131] In one embodiment, when determining a flipping action reward based on a body turning reward and a motion smoothness reward, a first weight corresponding to the body turning reward and a second weight corresponding to the motion smoothness reward can be first obtained. Then, based on the first and second weights, a weighted sum of the body turning reward and the motion smoothness reward is performed to obtain the flipping action reward. By setting the first and second weights and calculating the flipping action reward using a weighted summation of the body turning reward and the motion smoothness reward, both the quality of the body turning and the smoothness of the motion are considered. The resulting flipping action reward more comprehensively characterizes the quality and effectiveness of the flipping action. By adjusting the first and second weights, a balance can be achieved between the body turning reward and the motion smoothness reward. Specifically, the first and second weights can be adjusted based on the environmental complexity of the current legged robot. For example, in a scenario requiring a quick response, a higher weight can be given to the body turning reward (i.e., the first weight is increased), while in a scenario requiring smooth movement, a higher weight can be given to the motion smoothness reward (i.e., the second weight is increased).

[0132] In one embodiment, in the process of obtaining the first weight corresponding to the body turning reward and the second weight corresponding to the action smoothness reward, the first weight corresponding to the body turning reward can be determined based on the stable posture of the legged robot in the current iteration stage and the posture of the legged robot after performing the candidate turning action; the second weight corresponding to the action smoothness reward can be determined based on the smoothness of the process of the legged robot performing the candidate turning action.

[0133] In one embodiment, the stability and posture change magnitude of the legged robot before and after performing a turning action can be analyzed by calculating the posture difference between the stable posture of the legged robot in the current iteration and the posture of the legged robot after performing a candidate turning action. Then, the body turning reward is determined based on the influence of the posture change magnitude before and after performing the turning action and the overall emphasis on stability. The overall emphasis on stability can be determined based on the posture deviation between the stable posture of the legged robot in the current iteration and the ideal standing posture. For example, when the stable posture of the legged robot in the current iteration is closer to the ideal standing posture, the overall emphasis on stability is higher. When the posture deviation between the stable posture of the legged robot in the current iteration and the ideal standing posture is larger, the overall emphasis on stability is higher, and the legged robot can be encouraged to perform turning actions with large posture changes to quickly approach the ideal standing posture.

[0134] In one embodiment, a parameter change curve of the process of executing the candidate turning action can be drawn using the first action parameter and the second action parameter, and then the parameter change curve can be differentiated to obtain the smoothness of the process of executing the candidate turning action; or, the smoothness of the process of executing the candidate turning action can be judged by calculating the parameter difference between the first action parameter and the second action parameter. The larger the parameter difference, the lower the smoothness, and the smaller the parameter difference, the higher the smoothness. Accordingly, the smoother the process of executing the candidate turning action, the higher the determined second weight.

[0135] In one embodiment, calling the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information is calling the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction; in the process of calling the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction, an overall cumulative reward corresponding to the overall turning strategy direction is also obtained; therefore, based on different overall turning strategy directions, the falling and turning control method of the legged robot provided in the above embodiment can be re-executed to obtain the overall cumulative rewards corresponding to different overall turning strategy directions; according to the overall turning strategy direction with the largest overall cumulative reward, the optimal overall turning action strategy is determined; the optimal overall turning action strategy is applied to the legged robot, so that the legged robot can achieve falling and turning over.

[0136] In one embodiment, the overall turning strategy direction may refer to different action plans and motion paths that a legged robot can choose during a fall and turn over process, for example, prioritizing a specific action sequence, prioritizing posture direction adjustment, prioritizing turning speed, prioritizing action smoothness, etc. The overall cumulative reward is the sum of the turning action rewards corresponding to the optimal turning action in each iteration stage under the same overall turning strategy direction. Since the action selection model is called to predict the turning action using different overall turning strategy directions under the same fall posture, the optimal turning actions in each iteration stage may be different. Therefore, the turning action trajectories obtained under different overall turning strategy directions are also different, and the overall cumulative rewards obtained are also different. By comparing the overall cumulative rewards corresponding to different overall turning strategy directions, the overall turning strategy direction with the largest overall cumulative reward is selected as the overall turning strategy direction of the optimal overall turning action strategy.

[0137] In one embodiment, the overall cumulative reward can be obtained by weighting the turning action rewards corresponding to the optimal turning action in each iteration stage. Specifically, the turning action rewards of the optimal turning action in each iteration stage all correspond to a reward depreciation factor. The reward depreciation factor is related to the order of each iteration stage. The higher the order of the iteration stage, the larger the reward depreciation factor, and the lower the order of the iteration stage, the smaller the reward depreciation factor. That is, the reward depreciation factor corresponding to the first iteration stage is the largest, the reward depreciation factor corresponding to the second iteration stage is smaller than the reward depreciation factor corresponding to the first iteration stage, but larger than the reward depreciation factor corresponding to the third iteration stage, and so on. The reward depreciation factor corresponding to the last iteration stage is the smallest. Then, according to the reward depreciation factors of each iteration stage, the turning action rewards corresponding to the optimal turning action in each iteration stage are weighted and summed to obtain the weighted sum, and then the weighted sum is averaged to obtain the overall cumulative reward. Specifically, the calculation formula for the overall cumulative reward can be shown as follows (3):

[0138]

[0139] Among them, E j It is represented as the overall cumulative reward under the overall turning strategy direction j, T is represented as the total number of iteration stages under the overall turning strategy direction j, and γ t Expressed as the reward depreciation factor of the t-th iteration stage, R(s t ,s t+1 ) represents the turning action reward of the optimal turning action in the t-th iteration stage, that is, the posture s in the t-th iteration stage t Change to the posture s of the t+1th iteration stage t+1 The rollover action reward.

[0140] In one embodiment, after determining the optimal overall turning motion strategy, the legged robot's control module can invoke the motion selection model to perform a multi-stage iterative turning motion prediction based on the optimal overall turning motion strategy for the current fall posture information, thereby guiding the legged robot to achieve a turn after a fall. It should be noted that the legged robot's control module can upload the optimal overall turning motion strategy to a server, which can then be shared with other legged robots via the server for reference and learning from.

[0141] In one embodiment, when determining the optimal overall turning action strategy based on the overall turning strategy direction with the highest overall cumulative reward, the overall turning strategy direction with the highest overall cumulative reward is used as the target turning strategy direction. The optimal turning strategies corresponding to all iteration stages within the target turning strategy direction are then obtained. Based on the optimal turning strategies corresponding to all iteration stages within the target turning strategy direction, an overall turning action strategy within the target turning strategy direction is constructed. The overall turning action strategy within the target turning strategy direction is determined as the optimal overall turning action strategy. Each iteration stage corresponds to a corresponding optimal turning strategy. The optimal turning strategy may be the turning action optimization strategy corresponding to the candidate turning action with the highest turning action reward determined in the current iteration stage. The optimal turning strategies corresponding to all iteration stages within the target turning strategy direction are then used to construct an overall turning action strategy within the target turning strategy direction in the order of the iteration stages. The strategy generated in each iteration stage can be considered a local optimal solution within the current iteration stage. Therefore, these local optimal solutions are optimized and adjusted to obtain a globally optimal overall turning action strategy.

[0142] In one embodiment, referring to Figure 10 , Figure 10 This is a schematic diagram of the effect of a foot-type robot falling and turning over provided in an embodiment of the present application. Figure 10 As shown in (a), the legged robot falls, and the falling posture information of the legged robot is obtained. The action selection model is called to perform multi-stage iterative turning action prediction based on the falling posture information. First, the optimal turning action A1 in the first iterative stage is obtained, and the legged robot is controlled to perform the optimal turning action A1. Specifically, Figure 10 As shown in (b), the optimal turning action A1 can be the right front foot of the swing-legged robot. A stable posture S2 is randomly selected from the optimal turning action of the right front foot of the swing-legged robot. The stable posture S2 can be a turning posture of the legged robot in which the right front foot, left front foot and left hind foot are in contact with the ground and the right hind foot is suspended in the air. Then, the stable posture S2 is used as the initial state of the second iterative stage, and the turning action of the second iterative stage is predicted to obtain the optimal turning action A2 of the second iterative stage, and the legged robot is controlled to perform the optimal turning action A2. Specifically, as shown in FIG. Figure 10As shown in (c), the optimal turning action A2 can be to rotate the left front foot and the right front foot of the legged robot, and then swing the left front foot and the left hind foot. Therefore, a stable posture S3 can be randomly selected from the optimal turning action of the legged robot rotating the left front foot and the right front foot, and then swinging the left front foot and the left hind foot. The stable posture S3 can be a turning posture of the legged robot in which the left front foot and the left hind foot are in contact with the ground and the right front foot and the right hind foot are suspended in the air. Then, the stable posture S3 is used as the initial state of the third iterative stage, and the turning action of the third iterative stage is predicted to obtain the optimal turning action A3 of the third iterative stage, and the legged robot is controlled to perform the optimal turning action A3. Specifically, as shown in FIG. Figure 10 As shown in (d), the optimal turning action A3 can be to swing the left front foot and left hind foot of the legged robot so that the legged robot is in a turning standing posture SN with the left front foot, left hind foot, right front foot and right hind foot all in contact with the ground for support, and the turning is successful.

[0143] Reference Figure 11 As shown, Figure 11 This is a detailed flow chart of a method for controlling a legged robot falling and turning over, provided as a specific example. Figure 11 In the embodiment of the present invention, the method for controlling the fall and turning over of the legged robot may include but is not limited to steps 1101 to 1117.

[0144] Step 1101: Acquire the falling posture information of the legged robot.

[0145] Step 1102: Call the action selection model to determine an initial strategy for the turning action.

[0146] Step 1103: Determine a candidate turning action based on the stable posture of the legged robot in the current iteration stage and the initial strategy of the turning action, and control the legged robot to execute the candidate turning action.

[0147] Step 1104: Determine the body rotation angle of the legged robot according to the posture of the legged robot after performing the candidate turning action.

[0148] Step 1105: Obtain a reference gravity vector, vectorize the body rotation angle, and obtain a body rotation angle vector.

[0149] Step 1106: Obtain the body turning reward based on the Euclidean norm of the difference between the body rotation angle vector and the reference gravity vector.

[0150] Step 1107: Obtain the first action parameters of the candidate turning actions and the second action parameters of the optimal turning action in the current iteration stage.

[0151] Step 1108: Calculate the motion parameter difference between the first motion parameter and the second motion parameter.

[0152] Step 1109: Obtain the action smoothness reward based on the square of the Euclidean norm of the action parameter difference.

[0153] Step 1110: Determine a first weight corresponding to the body turning reward based on the stable posture of the legged robot in the current iteration stage and the posture of the legged robot after performing the candidate turning action.

[0154] Step 1111: Determine a second weight corresponding to the action smoothness reward based on the smoothness of the process of the legged robot executing the candidate turning action.

[0155] Step 1112: Based on the first weight and the second weight, perform weighted summation on the body turning reward and the action smoothness reward to obtain the turning action reward.

[0156] Step 1113: According to the reward for the turning action, the initial turning action strategy is adjusted to obtain an optimized turning action strategy.

[0157] Step 1114: Based on the stable posture of the legged robot in the current iteration stage and the turning action optimization strategy, a candidate turning action is re-determined until the re-determined candidate turning action maximizes the turning action reward.

[0158] Step 1115: The candidate turning action with the maximum turning action reward is determined as the optimal turning action for the next iteration stage.

[0159] Step 1116: Control the legged robot to perform the optimal turning action in the next iteration stage.

[0160] Among them, the legged robot will be in a stable posture after performing the optimal turning action in each iterative stage.

[0161] Step 1117: Determine whether the legged robot has successfully turned over. If so, end the step process; otherwise, return to step 1102.

[0162] In this embodiment, the fall and turn control method of the legged robot of the above-mentioned steps 1101 to 1117 is firstly performed by obtaining the fall posture information of the legged robot; then the action selection model is called to perform multi-stage iterative turn action prediction based on the fall posture information, and the optimal turn action of each iterative stage is obtained, and in each iterative stage, the legged robot is controlled to perform the optimal turn action of the current iterative stage, wherein the legged robot is in a stable posture after performing the optimal turn action of each iterative stage, and the legged robot successfully turns over after performing the optimal turn action of the last iterative stage; wherein the action selection model is called based on the fall posture information When performing multi-stage iterative turning motion prediction, the action selection model predicts the optimal turning motion for the next iterative stage based on the stable posture of the legged robot in the current iterative stage. This means that the goal of each iterative stage is to transition the legged robot from its current stable posture to the next stable posture until it successfully turns over. The end of each iterative stage means that the legged robot has reached a stable posture, providing new data for motion prediction in the next iterative stage. This allows the legged robot to autonomously learn and adjust the optimal turning motion required to achieve the next stable posture based on the stable posture of each iterative stage, thereby enabling the legged robot to successfully turn over. Because the entire control process is implemented based on the turning motion prediction made by the action selection model, no manual design and debugging are required, which can effectively reduce manual design costs and has high applicability.

[0163] Reference Figure 12 As shown, Figure 12 This is another specific example of a detailed flow chart of a method for controlling a legged robot falling and turning over. Figure 12 In the embodiment of the present invention, the method for controlling the fall and turning over of the legged robot may include but is not limited to steps 1201 to 1217.

[0164] Step 1201: Obtain the falling posture information of the legged robot.

[0165] Step 1202: Call the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction, obtain the optimal turning action in each iterative stage, and in each iterative stage, control the legged robot to perform the optimal turning action in the current iterative stage until the legged robot turns over successfully.

[0166] Among them, the legged robot will be in a stable posture after executing the optimal turning action in each iterative stage. When calling the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information, the action selection model predicts the optimal turning action in the next iterative stage based on the stable posture of the legged robot in the current iterative stage.

[0167] Step 1203: Calculate the overall cumulative reward corresponding to the overall turning strategy direction in the process of calling the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction.

[0168] Step 1204: Determine whether all overall turning strategy directions have been traversed. If so, execute step 1205; otherwise, return to execute step 1202.

[0169] In this step, if step 1202 is returned to be executed, the overall turning strategy corresponding to the overall cumulative reward that has not been obtained is reselected as the overall turning action strategy direction to be explored, so as to call the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning action strategy direction to be explored.

[0170] Step 1205: The overall turnaround strategy direction with the largest overall cumulative reward is used as the target turnaround strategy direction.

[0171] Step 1206: Obtain the optimal turning strategies corresponding to all iterative stages under the target turning strategy direction.

[0172] Step 1207: constructing an overall turning action strategy under the target turning strategy direction according to the optimal turning strategies corresponding to all iterative stages under the target turning strategy direction.

[0173] Step 1208: Determine the overall turning action strategy under the target turning strategy direction as the optimal overall turning action strategy.

[0174] Step 1209: Apply the optimal overall turning action strategy to the legged robot, so that the legged robot can turn over after falling.

[0175] In this embodiment, the fall and turnover control method of the legged robot of the above-mentioned steps 1201 to 1209 is firstly obtained by obtaining the fall posture information of the legged robot; then the action selection model is called to perform multi-stage iterative turning action prediction on the fall posture information based on the selected overall turning strategy direction, and the optimal turning action of each iterative stage is obtained. In each iterative stage, the footed robot is controlled to perform the optimal turning action of the current iterative stage until the turning over is successful. At the same time, the overall cumulative reward of the turning action under the currently selected overall turning strategy direction is evaluated. In addition, by traversing all the overall turning strategy directions, the overall turning strategy direction with the largest overall cumulative reward is selected as the target turning strategy direction, so that a more accurate turning action can be obtained in the subsequent turning action prediction process, thereby accelerating the turning speed and improving the athletic performance of falling and turning over.

[0176] The following are some practical examples to illustrate the application scenarios of the embodiments of the present application.

[0177] It should be noted that the method for controlling the fall and flipping of a legged robot provided in the embodiment of the present application can be applied to different application scenarios, such as scenarios in which the legged robot performs field operations, scenarios in which the legged robot performs exploration and rescue tasks, and scenarios in which the legged robot performs industrial manufacturing tasks. The following will be explained using the execution scenarios of research and development and debugging of the legged robot, the execution scenarios in which the legged robot performs environmental monitoring tasks, and the execution scenarios in which the legged robot performs industrial manufacturing tasks as examples.

[0178] Scene 1

[0179] The legged robot fall and turn control method provided in the embodiments of the present application can also be applied to scenarios where the legged robot is performing field work tasks. For example, when the legged robot falls during field work tasks such as plowing, planting, harvesting, and fertilizing, the control module of the legged robot can obtain the legged robot's fall posture information, then call a pre-stored action selection model to perform a multi-stage iterative turn action prediction based on the fall posture information, obtain the optimal turn action for each iterative stage, and control the legged robot to perform the optimal turn action for the current iterative stage in each iterative stage. After performing the optimal turn action for each iterative stage, the legged robot is in a stable posture, and after performing the optimal turn action for the last iterative stage, the legged robot successfully turns over. When the action selection model is called to perform a multi-stage iterative turn action prediction based on the fall posture information, the action selection model predicts the optimal turn action for the next iterative stage based on the stable posture of the legged robot in the current iterative stage. When the legged robot successfully turns over, it can continue to perform the interrupted field work task.

[0180] Scene 2

[0181] The legged robot fall and turn control method provided in the embodiments of the present application can also be applied to scenarios where the legged robot is performing exploration and rescue missions. For example, a legged robot is performing a search and rescue mission in complex terrain such as mountains, forests, and wilderness, searching for missing persons or exploring the terrain. If the legged robot falls during the mission, the control module of the legged robot can obtain the legged robot's fall posture information, then call a pre-stored action selection model to perform a multi-stage iterative turn action prediction based on the fall posture information, obtain the optimal turn action for each iterative stage, and control the legged robot to perform the optimal turn action for each iterative stage in each iterative stage. After performing the optimal turn action in each iterative stage, the legged robot is in a stable posture, and the legged robot successfully turns over after performing the optimal turn action in the last iterative stage. When calling the action selection model to perform a multi-stage iterative turn action prediction based on the fall posture information, the action selection model predicts the optimal turn action for the next iterative stage based on the stable posture of the legged robot in the current iterative stage. If the legged robot successfully turns over, it can resume the interrupted exploration and rescue mission.

[0182] Scenario 3

[0183] The legged robot fall and turn control method provided in the embodiments of the present application can also be applied to scenarios where the legged robot performs industrial manufacturing tasks. For example, the legged robot can perform heavy-load handling tasks, inspect and maintain equipment, or participate in assembly tasks on a production line. If the legged robot falls while performing the industrial manufacturing task, the legged robot's control module can obtain the legged robot's fall posture information, then call a pre-stored action selection model to perform a multi-stage iterative turn action prediction based on the fall posture information, obtain the optimal turn action for each iterative stage, and control the legged robot to perform the optimal turn action for the current iterative stage in each iterative stage. After performing the optimal turn action in each iterative stage, the legged robot is in a stable posture, and after performing the optimal turn action in the last iterative stage, the legged robot successfully turns over. When the action selection model is called to perform a multi-stage iterative turn action prediction based on the fall posture information, the action selection model predicts the optimal turn action for the next iterative stage based on the stable posture of the legged robot in the current iterative stage. When the legged robot successfully turns over, it can resume executing the interrupted industrial manufacturing task.

[0184] It will be appreciated that, although the various steps in the above-mentioned various flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless clearly stated in the present embodiment, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts can include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0185] Reference Figure 13 The present application also discloses a legged robot fall and turn control device. The legged robot fall and turn control device 1300 can implement the legged robot fall and turn control method of the previous embodiment. The legged robot fall and turn control device 1300 includes:

[0186] The control module 1310 is used to obtain the falling posture information of the legged robot;

[0187] The control module 1310 is also used to call the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information, obtain the optimal turning action in each iterative stage, and control the footed robot to perform the optimal turning action in the current iterative stage in each iterative stage, wherein the footed robot will be in a stable posture after performing the optimal turning action in each iterative stage, and the footed robot will successfully turn over after performing the optimal turning action in the last iterative stage; wherein, when calling the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information, the action selection model predicts the optimal turning action in the next iterative stage according to the stable posture of the footed robot in the current iterative stage.

[0188] In one embodiment, the control module 1310 is further configured to:

[0189] Determine an initial strategy for turning over;

[0190] Determine a candidate turning action based on the stable posture of the legged robot and the initial turning action strategy in the current iteration stage, and control the legged robot to execute the candidate turning action;

[0191] Determine a turning action reward based on the posture of the legged robot after executing the candidate turning action, the candidate turning action, and the optimal turning action in the current iteration stage;

[0192] According to the reward for the turning action, the initial turning action strategy is adjusted to obtain the turning action optimization strategy;

[0193] Based on the stable posture of the mid-legged robot and the turning action optimization strategy in the current iteration stage, a candidate turning action is re-determined until the re-determined candidate turning action maximizes the turning action reward;

[0194] The candidate turning action with the largest turning action reward is determined as the optimal turning action in the next iteration stage.

[0195] In one embodiment, the control module 1310 is further configured to:

[0196] Determine a body turning reward according to the posture of the legged robot after executing the candidate turning action;

[0197] Determine the action smoothness reward based on the candidate turning action and the optimal turning action in the current iteration;

[0198] Determine the reward for the turning action based on the body's turning reward and action smoothness reward.

[0199] In one embodiment, the control module 1310 is further configured to:

[0200] determining a body rotation angle of the legged robot according to the posture of the legged robot after performing the candidate turning action;

[0201] Determine the body's steering reward based on the body's rotation angle.

[0202] In one embodiment, the control module 1310 is further configured to:

[0203] Get the reference gravity vector;

[0204] Vectorize the body rotation angle to obtain the body rotation angle vector;

[0205] The body steering reward is obtained based on the Euclidean norm of the difference between the body rotation angle vector and the reference gravity vector.

[0206] In one embodiment, the control module 1310 is further configured to:

[0207] Obtain the first action parameter of the candidate turning action;

[0208] Obtain the second action parameter of the optimal turning action in the current iteration stage;

[0209] An action smoothness reward is determined according to the first action parameter and the second action parameter.

[0210] In one embodiment, the control module 1310 is further configured to:

[0211] calculating an action parameter difference between the first action parameter and the second action parameter;

[0212] The action smoothing reward is obtained based on the square of the Euclidean norm of the action parameter difference.

[0213] In one embodiment, the control module 1310 is further configured to:

[0214] Obtain the first weight corresponding to the body turning reward and the second weight corresponding to the action smoothness reward;

[0215] According to the first weight and the second weight, the body turning reward and the action smoothness reward are weightedly summed to obtain the turning action reward.

[0216] In one embodiment, the control module 1310 is further configured to:

[0217] Determining a first weight corresponding to the body turning reward based on the stable posture of the legged robot in the current iteration phase and the posture of the legged robot after performing the candidate turning action;

[0218] A second weight corresponding to the action smoothness reward is determined according to the smoothness of a process in which the legged robot executes the candidate turning action.

[0219] In one embodiment, calling the action selection model to perform multi-stage iterative turning action prediction based on the falling posture information is calling the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction; in the process of calling the action selection model to perform multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction, an overall cumulative reward corresponding to the overall turning strategy direction is also obtained; the control module 1310 is further configured to:

[0220] Based on different overall turning strategy directions, the fall and turning control method of the legged robot in the above embodiment is re-executed to obtain the overall cumulative rewards corresponding to the different overall turning strategy directions;

[0221] Determine the optimal overall turning action strategy based on the overall turning strategy direction with the largest overall cumulative reward;

[0222] The optimal overall turning motion strategy is applied to the legged robot, enabling the legged robot to turn over after falling.

[0223] In one embodiment, each iteration stage corresponds to an optimal turning strategy for that stage, and the control module 1310 is further configured to:

[0224] The overall turnaround strategy direction with the largest overall cumulative reward is used as the target turnaround strategy direction;

[0225] Obtain the optimal turning strategy for each stage corresponding to all iteration stages under the target turning strategy direction;

[0226] According to the optimal turning strategies corresponding to all iterative stages under the target turning strategy direction, the overall turning action strategy under the target turning strategy direction is constructed;

[0227] The overall turning over action strategy under the target turning over strategy direction is determined as the optimal overall turning over action strategy. It should be noted that since the fall and turning over control device 1300 of the footed robot in this embodiment can realize the fall and turning over control method of the footed robot in the previous embodiment, the fall and turning over control device 1300 of the footed robot in this embodiment and the fall and turning over control method of the footed robot in the previous embodiment have the same technical principles and the same beneficial effects. In order to avoid repetition of content, they will not be repeated here.

[0228] Reference Figure 14 The present application also discloses a legged robot. The legged robot 1400 includes:

[0229] at least one processor 1401;

[0230] At least one memory 1402, configured to store at least one program;

[0231] When at least one program is executed by at least one processor 1401, the above-mentioned method for controlling the fall and turning over of the legged robot is implemented.

[0232] An embodiment of the present application further discloses a computer-readable storage medium, which stores a computer program executable by a processor. When the computer program executable by the processor is executed by the processor, it is used to implement the fall and turnover control method of the legged robot as described above.

[0233] An embodiment of the present application also discloses a computer program product, including a computer program or computer instructions, which are stored in a computer-readable storage medium. The processor of the legged robot reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the legged robot executes the fall and turnover control method of the legged robot as described above.

[0234] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0235] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0236] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0237] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0238] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0239] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0240] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0241] The step numbers in the above method embodiment are only provided for the convenience of explanation and description, and do not limit the order of the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

Claims

1. A method for controlling a legged robot when it falls and turns over, characterized in that: The following steps are involved: Acquiring falling posture information of the legged robot; calling an action selection model to perform a multi-stage iterative turning action prediction based on the falling posture information to obtain an optimal turning action in each iterative stage, and controlling the legged robot to perform the optimal turning action in the current iterative stage in each iterative stage, wherein the legged robot is in a stable posture after performing the optimal turning action in each iterative stage, and the legged robot successfully turns over after performing the optimal turning action in the last iterative stage; Among them, when calling the action selection model to perform the multi-stage iterative turning action prediction based on the falling posture information, the action selection model predicts the optimal turning action in the next iterative stage according to the stable posture of the legged robot in the current iterative stage.

2. The method according to claim 1, characterized in that The process of predicting the optimal turning action in the next iteration phase according to the stable posture of the legged robot in the current iteration phase by the action selection model includes the following steps: Determine an initial strategy for turning over; determining a candidate turning action according to the stable posture of the legged robot in the current iteration stage and the initial turning action strategy, and controlling the legged robot to perform the candidate turning action; determining a turning action reward according to the posture of the legged robot after performing the candidate turning action, the candidate turning action, and the optimal turning action in the current iteration stage; Adjusting the initial strategy of the turning action according to the turning action reward to obtain an optimized strategy of the turning action; Re-determining a candidate turning action according to the stable posture of the legged robot in the current iteration stage and the turning action optimization strategy, until the re-determined candidate turning action maximizes the turning action reward; The candidate turning action when the turning action reward is the largest is determined as the optimal turning action in the next iterative stage.

3. The method according to claim 2, characterized in that The step of determining a turning action reward according to the posture of the legged robot after executing the candidate turning action, the candidate turning action, and the optimal turning action in the current iteration stage includes: determining a body turning reward according to the posture of the legged robot after performing the candidate turning action; Determining an action smoothness reward according to the candidate turning action and the optimal turning action in the current iteration stage; A turning action reward is determined according to the body turning reward and the action smoothness reward.

4. The method according to claim 3, characterized in that The determining of a body turning reward according to the posture of the legged robot after performing the candidate turning action includes: determining a body rotation angle of the legged robot according to the posture of the legged robot after performing the candidate turning action; A body turning reward is determined according to the body rotation angle.

5. The method according to claim 4, characterized in that The determining of the body turning reward according to the body rotation angle includes: Get the reference gravity vector; Vectorizing the body rotation angle to obtain a body rotation angle vector; A body turning reward is obtained according to a Euclidean norm of a difference between the body rotation angle vector and the reference gravity vector.

6. The method according to claim 3, characterized in that The determining of the action smoothness reward according to the candidate turning action and the optimal turning action in the current iteration stage includes: Obtaining a first action parameter of the candidate turning action; Obtaining a second action parameter of the optimal turning action in the current iteration stage; An action smoothness reward is determined according to the first action parameter and the second action parameter.

7. The method according to claim 6, characterized in that The determining of the action smoothness reward according to the first action parameter and the second action parameter includes: calculating an action parameter difference between the first action parameter and the second action parameter; The action smoothness reward is obtained according to the square of the Euclidean norm of the action parameter difference.

8. The method according to claim 3, characterized in that The step of determining the turning action reward according to the body turning reward and the action smoothness reward includes: Obtaining a first weight corresponding to the body turning reward and a second weight corresponding to the action smoothness reward; The body turning reward and the action smoothness reward are weightedly summed according to the first weight and the second weight to obtain a turning action reward.

9. The method according to claim 8, characterized in that The obtaining of the first weight corresponding to the body turning reward and the second weight corresponding to the action smoothness reward includes: Determining a first weight corresponding to the body turning reward according to the stable posture of the legged robot in the current iteration stage and the posture of the legged robot after performing the candidate turning action; A second weight corresponding to the action smoothness reward is determined according to the smoothness of a process in which the legged robot executes the candidate turning action.

10. The method according to claim 1, characterized in that The calling of the action selection model to perform a multi-stage iterative turning action prediction based on the falling posture information is to call the action selection model to perform a multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction; in the process of calling the action selection model to perform a multi-stage iterative turning action prediction on the falling posture information based on the overall turning strategy direction, an overall cumulative reward corresponding to the overall turning strategy direction is also obtained; The method further comprises: Based on the different overall turning strategy directions, re-executing the fall and turning control method of the legged robot according to any one of claims 1 to 9 to obtain the overall cumulative reward corresponding to the different overall turning strategy directions; Determine the optimal overall turning action strategy according to the overall turning strategy direction with the largest overall cumulative reward; The optimal overall turning-over action strategy is applied to the legged robot, so that the legged robot can turn over after falling.

11. The method according to claim 10, characterized in that Each iteration stage corresponds to an optimal turnaround strategy; Determining the optimal overall turning action strategy according to the overall turning strategy direction with the largest overall cumulative reward includes: The overall turnaround strategy direction with the largest overall cumulative reward is used as the target turnaround strategy direction; Obtaining the optimal turning strategy for each stage corresponding to all iterative stages under the target turning strategy direction; According to the optimal turning strategies of all iterative stages corresponding to the target turning strategy direction, an overall turning action strategy under the target turning strategy direction is constructed; The overall turning action strategy under the target turning strategy direction is determined as the optimal overall turning action strategy.

12. A fall and turn control device for a legged robot, characterized in that: include: A control module, configured to obtain falling posture information of the legged robot; The control module is further configured to call an action selection model to perform a multi-stage iterative turning action prediction based on the falling posture information, obtain an optimal turning action in each iterative stage, and control the legged robot to perform the optimal turning action in the current iterative stage in each iterative stage, wherein the legged robot is in a stable posture after performing the optimal turning action in each iterative stage, and the legged robot successfully turns over after performing the optimal turning action in the last iterative stage; Among them, when the control module calls the action selection model to perform the multi-stage iterative turning action prediction based on the falling posture information, the action selection model predicts the optimal turning action in the next iterative stage according to the stable posture of the legged robot in the current iterative stage.

13. A legged robot, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the method for controlling a legged robot falling and turning over as described in any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium, characterized in that A computer program executable by a processor is stored therein, and when the computer program executable by the processor is executed by the processor, it is used to implement the fall and turnover control method of the legged robot as described in any one of claims 1 to 11.

15. A computer program product comprising a computer program or computer instructions, characterized in that The computer program or the computer instructions are stored in a computer-readable storage medium, the processor of the legged robot reads the computer program or the computer instructions from the computer-readable storage medium, and the processor executes the computer program or the computer instructions, so that the legged robot executes the fall and turnover control method of the legged robot as described in any one of claims 1 to 11.

Images