State Estimation Method and Device for Legged Robot, and Computer Equipment

Abstract

Disclosed are a state estimation method, a state estimation device, a computer device, a computer-readable storage medium, and a computer program product for a legged robot, which can be applied to scenarios such as artificial intelligence, legged robot technology, and mechatronics. Each aspect of the present disclosure integrates the output information of different sensors operating at different frequencies, and uses two Kalman filters to solve problems such as different frequencies and different delays in fusing information from different sensors, realizing multi-sensor fusion state estimation with high real-time performance, high robustness, and high effectiveness, and significantly reducing the cumulative error of state estimation during long-term movement of the legged robot.

Description

Technical Field

[0001] The present disclosure relates to the technical field of legged robots, specifically to the technical field of planning and control of legged robots, and particularly to a state estimation method for legged robots, a state estimation device for legged robots, a computer device, and a computer storage medium. Background Art

[0002] Currently, during the movement of a legged robot, various means can usually be used to determine the state of the legged robot in real time (such as position and attitude information, etc.). For example, multiple sensor fusions can be used to estimate the body state of the legged robot. However, when using sensors based on body perception such as IMU and leg encoders for state estimation,

[0003] since the legged robot often experiences foot tip side slip, deformation, mechanical structure errors, etc. during movement, and sensor noise, drift, etc. will also affect the state estimation result, resulting in inaccurate estimated values. As the movement duration of the legged robot increases, the error value will also accumulate. Since the control of the legged robot usually requires the current state estimation value to control each joint motor, inaccurate state estimation values may make it difficult to accurately control the legged robot.

[0004] Therefore, it is necessary to improve the existing state estimation method to eliminate the cumulative error as much as possible while ensuring real-time acquisition of the state estimation value. Summary of the Invention

[0005] Embodiments of the present disclosure provide a state estimation method for a legged robot and related devices, which can improve the accuracy of the state estimation value when the state estimation value of the legged robot is acquired in real time.

[0006] The present disclosure provides a state estimation method for a legged robot, the method comprising: acquiring first sensor information and second sensor information of the legged robot, and based on the first sensor information and the second sensor information, using a first Kalman filter to determine first state information of the legged robot, and saving the first state information for a period of time as historical information for a second Kalman filter; acquiring third sensor information of the legged robot, and based on the third sensor information and the historical information, using the second Kalman filter to determine second state information of the legged robot; and based on the second state information of the legged robot, updating the first state information of the legged robot at the current moment to determine the state information of the legged robot at the current moment.

[0007] In another aspect, the present disclosure provides a state estimation device for a legged robot, the device including: a first sensor configured to obtain first sensor information of the legged robot; a second sensor configured to obtain second sensor information of the legged robot; a third sensor configured to obtain third sensor information of the legged robot; a first Kalman filter configured to determine first state information of the legged robot based on the first sensor information and the second sensor information, and save the first state information for a period of time as historical information; a second Kalman filter configured to determine second state information of the legged robot based on the third sensor information and the historical information by using the second Kalman filter; wherein, the first Kalman filter is further configured to determine the state information of the legged robot at the current moment based on the second state information and the first state information corresponding to the legged robot at the current moment.

[0008] In another aspect, the present disclosure provides a computer device including an input interface and an output interface, characterized by further including: a processor adapted to implement one or more instructions; and a computer storage medium; the computer storage medium stores one or more instructions, and the one or more instructions are adapted to be loaded and executed by the processor to perform the above-mentioned state estimation method for a legged robot.

[0009] In another aspect, the present disclosure provides a computer-readable storage medium, characterized in that the computer storage medium stores one or more instructions, and the one or more instructions are adapted to be loaded and executed by a processor to perform the above-mentioned state estimation method for a legged robot.

[0010] In another aspect, the present disclosure provides a computer program product, characterized in that the computer program product includes a computer program; when the computer program is executed by a processor, the above-mentioned state estimation method for a legged robot is implemented.

[0011] Thus, various aspects of the present disclosure integrate the output information of different sensors operating at different frequencies, and use two Kalman filters to solve problems such as different frequencies and different delays in fusing different sensor information, achieving multi-sensor fusion state estimation with high real-time performance, high robustness, and high effectiveness, and significantly reducing the cumulative error caused by state estimation during long-term movement of the legged robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some exemplary embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. The following drawings are not deliberately drawn to scale in actual size, and the focus is on showing the gist of the present disclosure.

[0013] Figure 1 It is a schematic diagram showing a legged robot according to an embodiment of the present disclosure.

[0014] Figure 2 It is a flowchart showing a state estimation method for a legged robot according to an embodiment of the present disclosure.

[0015] Figure 3 It is a schematic diagram showing a state estimation method for a legged robot according to an embodiment of the present disclosure, in which the first state information and the second state information are schematically shown.

[0016] Figure 4 It is a partial structural diagram showing a state estimation device for a legged robot according to an embodiment of the present disclosure.

[0017] Figure 5 It is a schematic diagram showing the execution of a method by a state estimation device for a legged robot according to an embodiment of the present disclosure.

[0018] Figure 6 It is a comparison diagram showing the execution effect of the method according to an embodiment of the present disclosure and the traditional solution.

[0019] Figure 7 It shows a schematic diagram of an electronic device according to an embodiment of the present disclosure.

[0020] Figure 8 It shows a schematic diagram of the architecture of an exemplary computing device according to an embodiment of the present disclosure.

[0021] Figure 9 It shows a schematic diagram of a storage medium according to an embodiment of the present disclosure. Detailed implementation manners

[0022] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some embodiments of the present disclosure, rather than all embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present disclosure.

[0023] In the description of the present disclosure, it should be understood that the orientation or positional relationship indicated by terms such as "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present disclosure, "a plurality of" means two or more unless otherwise specifically defined.

[0024] In the description of the present disclosure, it should be noted that unless otherwise clearly specified and defined, the terms "mounted", "connected" and "coupled" shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection, an electrical connection or a connection capable of mutual communication; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the internal communication of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

[0025] In the present disclosure, unless otherwise clearly specified and defined, the first feature being "on" or "under" the second feature may include the direct contact between the first and second features, or may include the situation where the first and second features are not in direct contact but in contact through other features therebetween. Moreover, the first feature being "above", "over" and "on top of" the second feature includes that the first feature is directly above and obliquely above the second feature, or merely means that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "beneath" and "underneath" the second feature includes that the first feature is directly below and obliquely below the second feature, or merely means that the horizontal height of the first feature is lower than that of the second feature.

[0026] The following disclosure provides many different embodiments or examples for implementing different structures of the present disclosure. To simplify the disclosure of the present disclosure, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present disclosure. In addition, the present disclosure may repeat reference numerals and / or reference letters in different examples. Such repetition is for the purpose of simplification and clarity and does not itself indicate the relationship between the various embodiments and / or settings discussed. In addition, the present disclosure provides examples of various specific processes and materials, but those of ordinary skill in the art may be aware of the application of other processes and / or the use of other materials.

[0027] Although the present disclosure makes various references to certain modules in the devices according to the embodiments of the present disclosure, any number of different modules may be used and run on the user terminal and / or the server. The modules are only illustrative, and different aspects of the device and method may use different modules.

[0028] Flowcharts are used in the present disclosure to illustrate the operations performed by the methods and devices according to the embodiments of the present disclosure. It should be understood that the operations before or below do not necessarily need to be executed precisely in sequence. On the contrary, various steps may be processed in reverse order or simultaneously as needed. At the same time, other operations may also be added to these processes, or one or more steps may be removed from these processes.

[0029] To facilitate the description of the present disclosure, the following concepts related to the present disclosure are introduced.

[0030] The legged robot of the present disclosure is a legged robot that uses legs for movement. It takes animals as the bionic object, and its purpose is to simulate the movement form of animals and replicate the movement ability of animals based on the achievements of engineering technology and scientific research. The legged robot has strong adaptability to various environments (including structured environments such as roads, railways, treated flat roads, etc. and unstructured environments such as mountains, swamps, rough roads, etc.). It can adapt to various changes in terrain, cross over higher obstacles, and can effectively reduce the load and improve the energy utilization efficiency of the system. The legged robot can be divided into single-legged, two-legged, four-legged, six-legged, eight-legged, etc. according to the number of legs. Among them, the four-legged legged robot has super movement ability. It has better static stability than the two-legged legged robot and is simpler and more flexible in movement than the six-legged and eight-legged legged robots. Therefore, the four-legged legged robot is a common choice for studying legged robots. The gait of the four-legged legged robot is the coordination relationship of its four legs in time and space in order to be able to move continuously. The gait of the four-legged legged robot comes from the gait of four-legged mammals and may include but is not limited to the following three simplified forms: walk, trot, and bound.

[0031] The method for controlling a legged robot according to the present disclosure may be based on Artificial Intelligence (AI). Artificial intelligence is the theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. For example, for the method of controlling a legged robot based on artificial intelligence, it can plan the movement trajectory and gait of the legged robot in a way similar to how humans guide the movement of living animals, making the movement of the legged robot more flexible and bionic. By studying the design principles and implementation methods of various intelligent machines, artificial intelligence enables the method for controlling a legged robot according to the present disclosure to automatically and efficiently design the subsequent movement trajectory and gait of the legged robot based on the current movement state of the legged robot.

[0032] In summary, the solution provided by the embodiments of the present disclosure involves technologies such as artificial intelligence and machine learning. The embodiments of the present disclosure will be further described below with reference to the accompanying drawings.

[0033] Figure 1 FIG. is a schematic diagram showing a legged robot 100 according to an embodiment of the present disclosure. As Figure 1 shown, taking a quadruped legged robot as an example, Figure 1 the left and right figures in FIG. respectively show the internal perspective view and the external structure diagram of the exemplary legged robot.

[0034] The exemplary legged robot is capable of moving based on four mechanical legs. Each mechanical leg may include a thigh and a calf, and may include at least one joint. For example, each mechanical leg may include multiple lower limb joints, such as two degrees of freedom for the hip joint and one degree of freedom for the knee joint. It should be noted that the term "joint" in the present disclosure indicates a movable connection between two or more components of the legged robot. The joint can move under the control of the force and torque output by the joint motor. For example, the joint can rotate by an angle so that other joints and their linkage mechanisms can generate a certain amount of movement in space, thereby causing a change in the posture / movement state of the legged robot 100.

[0035] In addition, each mechanical leg can be configured with multiple motors, which can be used to control or jointly control two degrees of freedom of the hip joint and one degree of freedom of the knee joint of the quadruped robot. It should be noted that various measurement components can also be configured on the quadruped robot, such as IMU (Inertial Measurement Unit) sensors and joint encoders, etc.; among them, the IMU sensor can provide the acceleration and attitude information of the quadruped robot in real time, and the joint encoder can provide the joint angle information of each joint of the quadruped robot in real time (such as the angle of the joint angle, the feedback value of the angular velocity, etc.).

[0036] Under the control of the multiple motors mentioned above, the exemplary quadruped robot has been able to achieve various motions, for example, including acrobatic flips or jumping and other action motions. However, accurately controlling the quadruped robot to perform these actions often requires accurate state data. If the state data is inaccurate, it is difficult for various control schemes to achieve accurate and flexible control of the quadruped robot.

[0037] Currently, during the motion of the quadruped robot, various means are usually used to determine the state of the quadruped robot in real time (such as position and attitude information, etc.). For example, various sensors can be used to estimate the body state of the quadruped robot. For example, in the industrial field, various methods have been proposed to determine the body state of the quadruped robot by fusing IMU data, joint encoder data, laser data, motion capture data, etc., but these methods cannot solve the state drift of the body perception sensor state estimator during long-term operation, resulting in inevitable cumulative errors.

[0038] Therefore, in view of the above problems, the present disclosure provides a state estimation method for a quadruped robot, the method including: obtaining first sensor information and second sensor information of the quadruped robot, and based on the first sensor information and the second sensor information, using a first Kalman filter to determine first state information of the quadruped robot, and saving the first state information for a period of time as historical information for a second Kalman filter; obtaining third sensor information of the quadruped robot, and based on the third sensor information and the historical information, using the second Kalman filter to determine second state information of the quadruped robot; and based on the second state information of the quadruped robot, updating the first state information of the quadruped robot at the current moment to determine the state information of the quadruped robot at the current moment.

[0039] In another aspect, the present disclosure also provides a state estimation device for a legged robot, the device including: a first sensor configured to acquire first sensor information of the legged robot; a second sensor configured to acquire second sensor information of the legged robot; a third sensor configured to acquire third sensor information of the legged robot; a first Kalman filter configured to determine first state information of the legged robot based on the first sensor information and the second sensor information, and save the first state information for a period of time as historical information; a second Kalman filter configured to determine second state information of the legged robot by using the second Kalman filter based on the third sensor information and the historical information; wherein, the first Kalman filter is further configured to determine state information of the legged robot at the current moment based on the second state information and the first state information corresponding to the legged robot at the current moment.

[0040] Compared with traditional state estimation schemes, various aspects of the present disclosure integrate sensor information of different sensors operating at different frequencies, and use two Kalman filters to solve problems such as high latency caused by only being able to fuse different sensor information at a relatively low frequency in traditional schemes, achieving multi-sensor fusion state estimation with high real-time performance, high robustness, and high effectiveness, and significantly reducing the cumulative error caused by state estimation during long-term movement of the legged robot.

[0041] The following refers to Figures 2 to 9 to further describe examples of various aspects of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited thereto.

[0042] Figure 2 is a flowchart showing a state estimation method 20 for a legged robot according to an embodiment of the present disclosure. Figure 3 is a schematic diagram showing the state estimation method 20 for a legged robot according to an embodiment of the present disclosure, in which first state information and second state information are schematically shown.

[0043] The state estimation method 20 for a legged robot according to an embodiment of the present disclosure may include operations S201 - S203 as Figure 2 shown. As Figure 2As shown, method 20 includes one or all of operations S201 to S203, and may also include more operations. The execution subjects of operations S201 to S203 can be various components located on the legged robot (for example, various sensors and computing components located on the legged robot), or can be various components located outside the legged robot (for example, various sensors and computing components located outside the legged robot). The present disclosure does not limit the execution subjects of operations S201 to S203. Method 20 can be processed by device 40 detailed later.

[0044] As an example, method 20 can be executed by any computing device. The computing device here can be a terminal or a server; alternatively, the computing device here can also be jointly executed by a terminal and a server, and this is not limited. Among them, the terminal can be a smart phone, a computer (such as a tablet computer, a notebook computer, a desktop computer, etc.), a smart wearable device (such as a smart watch, smart glasses), a smart voice interaction device, a smart home appliance (such as a smart TV), a vehicle-mounted terminal or an aircraft, etc.; the server can be an independent physical server, or can be a server cluster or distributed system composed of multiple physical servers, or can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and big data and artificial intelligence platforms, and so on. Further, the terminal and the server can be located inside or outside the blockchain network, and this is not limited; furthermore, the terminal and the server can also upload any data stored internally to the blockchain network for storage to prevent the data stored internally from being tampered with and improve data security.

[0045] For example, in operation S201, obtain the first sensor information and the second sensor information of the legged robot, and based on the first sensor information and the second sensor information, use a first Kalman filter to determine the first state information of the legged robot, and use the first state information saved for a period of time as the historical information of the second Kalman filter.

[0046] Optionally, the first sensor information may be any information collected by the first sensor. Optionally, the first sensor information carries a timestamp. The first sensor may operate at a first frequency to obtain the first sensor information corresponding to multiple timestamps in the first timestamp set. That is, the first sensor information may include the information collected by the first sensor at each timestamp in the first timestamp set. The first frequency may be relatively high, thereby ensuring the real-time performance of the first sensor. An exemplary first frequency may be from 0.01 to 1 kHz. Optionally, the real-time performance of the first sensor information collected by the first sensor is relatively high, but there may be a relatively high cumulative error as the legged robot moves.

[0047] For example, the first sensor may include at least one of the following devices: a tactile sensor, a force and torque sensor, an inertial measurement unit (IMU), etc. The present disclosure does not limit the type of sensor information and the specific signal form. For another example, in the case where the legged robot 100 further includes a robotic arm, the first sensor may further include a force and torque sensor corresponding to the robotic arm. In the case where the robotic arm further includes a finger tip, the first sensor may further include a tactile sensor at the finger tip, etc. In some embodiments, the first sensor may be physically attached to the legged robot, and its attachment position may change with the change of the configuration of the legged robot. The present disclosure is not limited thereto. In still other embodiments, the first sensor may not be physically attached to the legged robot. The present disclosure is not limited thereto.

[0048] Optionally, the second sensor information may be any information collected by the second sensor. Optionally, the second sensor also carries a timestamp. The second sensor may operate at a second frequency to obtain the second sensor information corresponding to multiple timestamps in the second timestamp set. The second sensor information may be used to calculate information related to the actions and postures of the legged robot. For example, the second sensor may be a joint encoder, which is used to collect the second sensor information of the legged robot, so as to obtain the joint angle information (such as the angle of the joint angle, the angular velocity feedback value, etc.) of each joint of the legged robot. Optionally, the joint encoder may also operate at a second frequency. An exemplary second frequency may be from 0.1 to 1 kHz. The second frequency may be slightly lower than the first frequency or may be equal to the first frequency. The present disclosure is not limited thereto.

[0049] The first Kalman Filter can be either a standard Kalman Filter or an Extended Kalman Filter. Specifically, a Kalman Filter is a highly efficient recursive filter (autoregressive filter) that can estimate the state of a legged robot from a series of incomplete and noisy measurement information (e.g., first sensor information and second sensor information). Kalman filtering will consider the joint distribution at different times based on the values of each measurement information at different times, and then generate an estimate of the unknown variable (e.g., first state information). Therefore, it will be more accurate than the estimation method based on a single measurement quantity. The so-called Extended Kalman Filter is an extended form of the standard Kalman Filter in the non-linear case. It can perform a Taylor expansion on the non-linear function, omit the high-order terms, and retain the first-order terms of the expansion terms to linearize the non-linear function. Although the following examples are described with a standard Kalman Filter as an example, those skilled in the art should understand that the present disclosure is not limited thereto.

[0050] In a specific implementation, the determining the first state information of the legged robot by using the first Kalman Filter includes: determining the first sensor information and the second sensor information with time stamps, sequentially using the first sensor information and the second sensor information as inputs to the first Kalman Filter; and performing state estimation through the first Kalman Filter to obtain the first state information of the legged robot with time stamps, and saving the first state information with time stamps as historical information. The first state information includes multiple state estimation results with time stamps output by the first Kalman Filter.

[0051] Optionally, referring to Figure 3 , assume that the current time is T b , and multiple time stamps are included in the time period corresponding to from time T a to time T b . Each time stamp corresponding first sensor information and second sensor information in the above multiple time stamps can be sequentially used as inputs to the first Kalman Filter to perform state estimation through the first Kalman Filter, so as to obtain the first state information corresponding to each time stamp of the legged robot in the time period [T a , T b . It should be noted that the first state information includes multiple state estimation results corresponding to multiple time stamps output by the first Kalman Filter.

[0052] In one example, the first state information may be represented by a time-series numerical sequence composed of the centroid and foot-end related information corresponding to each time step (each time step corresponds to a time stamp), and each element in the time-series numerical sequence corresponds to a state estimation result. Optionally, each element in the time-series numerical sequence may have multiple dimensions, which respectively represent the position of the centroid and the foot end in the x-axis direction, the position of the centroid in the y-axis direction, the position of the centroid in the z direction (gravity direction), the pitch angle of the legged robot, the yaw angle of the legged robot, and the roll angle of the legged robot, etc. The present disclosure is not limited thereto.

[0053] The above process can be carried out in real time, that is, every time the first sensor information and the second sensor information corresponding to a time stamp are collected, they are input into the first Kalman filter to obtain a state estimation result (for example, Figure 3 the first state information represented by a curve in

[0054] ), and each point on the curve corresponds to the state information at a moment. a For example, at time stamp T a , the state estimation result output by the first Kalman filter is S b-1 . At time stamp T b-1 , the state estimation result output by the first Kalman filter is S a to S b-1 . Hereinafter, the term "state estimation results S a to S b-1 " includes the state estimation results S i corresponding to each time stamp T a (T i <T b-1 ) within the historical time period corresponding to the time from moment T i . After that, the example scheme for obtaining the first state information through the first Kalman filter will be further described with reference to the examples described in Figure 4 and Figure 5 , and the present disclosure will not elaborate herein.

[0055] Then, the first state information for a period of time can be saved (for example, the state estimation results S a to S b-1 corresponding to each time stamp within the time period [T a to T b-1 ) as the historical information of the second Kalman filter.

[0056] Next, in operation S202, third sensor information of the legged robot is obtained, and based on the third sensor information and the historical information, the second state information of the legged robot is determined using the second Kalman filter.

[0057] Optionally, the third sensor information may be any information collected by a third sensor. The third sensor may operate at a third frequency to obtain the third sensor information corresponding to only one of the plurality of moments in a fixed time period. The third frequency may be relatively low, for example, lower than the first frequency. An exemplary third frequency may be 10 to 50 HZ. Optionally, the real-time performance of the third sensor information collected by the third sensor is low but the accuracy is high, and there is no cumulative error as the legged robot moves.

[0058] Optionally, the third sensor may include at least one of the following devices: a distance sensor (TOF), a camera or camera providing visual information, a radar, a position sensor (such as a GPS sensor), a laser sensor, etc.

[0059] Similarly, the second Kalman filter may also be a standard Kalman filter or an extended Kalman filter. Optionally, the timestamp corresponding to the third sensor information may be determined, and the historical information and the third sensor information are used as inputs to the second Kalman filter; and state estimation is performed through the second Kalman filter, and the state estimation result corresponding to the timestamp of the third sensor information is obtained as the second state information. In addition, in another embodiment, to further reduce the calculation amount, only the state estimation result corresponding to the timestamp corresponding to the third sensor information in the above historical information and the third sensor information may be used as inputs to the second Kalman filter to perform state estimation through the second Kalman filter, and the state estimation result corresponding to the timestamp of the third sensor information is obtained as the second state information.

[0060] Specifically, the process of state estimation by the second Kalman filter can be briefly described as follows: First, based on the timestamp corresponding to the third sensor information, the state estimation result corresponding to the timestamp in the historical information is determined; then, based on the third sensor information and the state estimation result corresponding to the timestamp in the historical information, state estimation is performed using the second Kalman filter to obtain the state estimation result corresponding to the timestamp as the second state information.

[0061] For example, referring to Figure 3 , taking the third sensor as a camera or camera providing visual information as an example, the camera or camera may open the shutter at time T a to capture the legged robot at time T aof the image, and then process the image, and finally at the current time T b , provide the third sensor information corresponding to the time T a to the second Kalman filter. At the time T b , the second Kalman filter can, according to the first state information corresponding to the time T a , combined with the third sensor information at the time T a , estimate the state estimation results of each timestamp in the time period Ta to T b-1 .

[0062] As Figure 3 shown, at the time T a , there is a certain state increment ΔS between the state estimation result Sa of the first Kalman filter and the state estimation result of the second Kalman filter. In one example, the second Kalman filter can superimpose the state increment ΔS on the first state information Sb corresponding to the current time Tb of the legged robot to determine the state information of the legged robot at the current time

[0063] In addition, in another example, the second Kalman filter can, based on the state increment ΔS, correct one by one the state estimation results S a to S b-1 determined by the first Kalman filter at each timestamp in the time period T a to S b-1 to obtain the second state information to wherein, is the state estimation result corresponding to the time T b-1 estimated by the third sensor information. For example, it can be through the formula where a < i < b, to correct the state estimation result corresponding to each timestamp in the first state information. In another example, the second Kalman filter can use the offset between the state estimation result S a of the first Kalman filter corresponding to the time T a and the state estimation result of the second Kalman filter as the initial value, and iteratively update the state estimation results of each timestamp in the time period T a to T b-1 to obtain the second state information to In another example, the second Kalman filter can further combine the kinematic model or dynamic model of the legged robot, and based on the state estimation result of the second Kalman filter, sequentially update the time period T a to T b-1The state estimation results of each timestamp in are obtained to obtain the second state information to The present disclosure does not limit this

[0064] After that, reference will be made to Figure 4 and Figure 5 The example described in further illustrates an example scheme for obtaining the second state information through the second Kalman filter, which will not be elaborated herein by the present disclosure

[0065] In operation S203, the first state information corresponding to the current moment of the legged robot is obtained, and based on the second state information of the legged robot, the state information of the legged robot at the current moment is determined

[0066] Refer to Figure 3 and continue with the current moment as t b as an example for illustration. In order to obtain the state S b at the current moment T b at moment T b the first sensor can collect the first sensor information corresponding to moment T b and the joint encoder can collect the second sensor information corresponding to moment T b At this time, the first Kalman filter can take the first sensor information and the second sensor information corresponding to moment T b as inputs to obtain the first state estimation result S b corresponding to moment T b and correct the first state estimation result S b with the state increment ΔS in operation S202 to obtain the cumulative error in S b is eliminated by the state increment ΔS, and then relatively accurate That is, the state information of the legged robot at the current moment is the information fused by the first Kalman filter and the second Kalman filter, and its accuracy is relatively high

[0067] Thus, the method 20 of the present disclosure fuses the sensor information of different sensors operating at different frequencies and the second sensor information, and uses two Kalman filters to solve the problems of low frequency and high latency in fusing different sensor information, realizing multi-sensor fusion state estimation with high real-time performance, high robustness, and high effectiveness, and significantly reducing the cumulative error caused by state estimation during long-term movement of the legged robot. In particular, method 20 simplifies the analysis and adjustment of the Kalman filter and reduces the running time. In addition, the second Kalman filter can be directly turned off when the third sensor information is unavailable without affecting the operation of the entire state estimation device

[0068] Next, refer toFigure 4 , in combination with Figure 1 describe the legged robot 100 and Figures 2 to 3 describe method 20, and further describe some details of operations S201 to S203. Among them, Figure 4 is a partial structural diagram showing a state estimation device 40 for a legged robot according to an embodiment of the present disclosure.

[0069] As Figure 4 shown, the state estimation device 40 for a legged robot optionally includes a first sensor, a second sensor, a third sensor, a first Kalman filter, and a second Kalman filter. Among them, the first Kalman filter can optionally operate in a high-frequency and low-latency state, while the second Kalman filter can optionally operate in a low-frequency and high-latency state.

[0070] The first sensor is configured to obtain the first sensor information corresponding to each timestamp in the first timestamp set of the legged robot and the first sensor information corresponding to the current moment of the legged robot. As can be seen from the foregoing, the first sensor in the legged robot may include an IMU sensor, and the IMU sensor can collect the attitude information and acceleration of the legged robot at multiple moments at a first frequency as the first sensor information corresponding to that moment. The IMU sensor can be configured to obtain the three-axis attitude angles (or angular rates) and accelerations of each joint corresponding to these moments at multiple different discrete moments. The data output by the IMU sensor includes timestamps, three-dimensional accelerations, three-dimensional angular velocities, orientations, etc., and the present disclosure is not limited thereto.

[0071] For example, the joint encoder as the second sensor is optionally configured to obtain the second sensor information corresponding to each timestamp in the second timestamp set of the legged robot and the second sensor information corresponding to the current moment of the legged robot. As an example, the joint encoder can collect and provide the joint angle information (such as joint angle, angular velocity feedback value, etc.) of each joint on each mechanical leg of the legged robot at each moment as the second sensor information corresponding to that moment. For example, the joint encoder can be configured to obtain the angles q[] and joint angular velocities of all joints corresponding to these timestamps for multiple different timestamps

[0072] The first Kalman filter is configured to determine the first state information of the legged robot based on the first sensor information and the second sensor information corresponding to the plurality of timestamps. Optionally, the first Kalman filter further includes an observation model and a prediction model. The observation model is used for state observation, and state observation data can be obtained through state observation. The prediction model is used for state prediction, and state prediction data can be obtained through state prediction. As will be described in detail later, the prediction model obtains a predicted state and corresponding covariance through the first sensor information, and the observation model obtains an observed state and corresponding covariance through the second sensor information. Of course, the present disclosure is not limited thereto.

[0073] Specifically, it is assumed that both the first timestamp set and the second timestamp set include the timestamp T i . For the timestamp T i , the first Kalman filter performs state estimation to obtain the state estimation result corresponding to the timestamp T i , including: inputting the first sensor information corresponding to the timestamp T i into the observation model to obtain the state observation data corresponding to the timestamp T i ; inputting the state estimation result corresponding to the previous timestamp T i of the timestamp T i-1 and the first sensor information corresponding to the timestamp T i into the prediction model to obtain the state prediction data corresponding to the timestamp T i ; and using the measurement value obtained by the second sensor corresponding to the timestamp T i to correct the state prediction data corresponding to the timestamp T i to obtain the state estimation result corresponding to the timestamp T i . For example, the measurement value is obtained by kinematic solution of the second sensor information corresponding to the timestamp T i .

[0074] As an example, it is assumed that the observation model is based on the observation matrix H. The state observation data Y i corresponding to the timestamp T i can be expressed as Y i = Hx i , where x i is the observable quantity corresponding to the timestamp T i , which is determined based on the first sensor information. As another example, the prediction model can be expressed as where m i ~ N(0, Q i ) is the prediction noise, A is the state transition matrix, and B is the control matrix.

[0075] In a specific implementation, the first Kalman filter determines the legged robot at time Ti The general principle of the first state information is as follows:

[0076] First, the IMU sensor in the legged robot can be called to collect the first sensor information of the legged robot at time T i where the first sensor information at time T i includes the acceleration information at time T i (which may include the accelerations of the legged robot in multiple directions (such as the vertical direction and the horizontal direction)) and the attitude information, and the second sensor joint encoder is called to determine the joint angle information (such as the angle of the joint angle, the angular velocity feedback value, etc.) of each joint of the legged robot at time T i

[0077] Then, the attitude information and the joint angle information at time T i can be incorporated into the observation model to calculate the state observation data, which may include: the values of multiple state observables. The values of multiple state observables include: the observed positions of the foot ends of each mechanical leg of the legged robot at time T i and the observed position of the centroid of the legged robot at time T i As an example, the observation model can output a set of pose data identified by timestamps. For example, the pose data may optionally include the XYZ axis values of the centroid and the foot ends in the world coordinate system / body coordinate system, the orientation data (represented by a quaternion), and the velocities of the legged robot 100 along the XYZ axes in the world coordinate system / body coordinate system, etc.

[0078] Meanwhile, at least one of the state estimation result at the previous time T i of the legged robot, the acceleration information at time T i-1 and the attitude information at time T i of the legged robot can also be incorporated into the prediction model to calculate the state prediction data at time T i The state prediction data may include the values of multiple state predictors and the prior covariance. The values of multiple state predictors include the predicted positions of the foot ends of each mechanical leg of the legged robot at time T i , the predicted position of the centroid of the legged robot at time T i , the velocity of the centroid of the legged robot at time T i etc.

[0079] Next, the first Kalman filter can determine the state estimation result corresponding to time T i based on the state observation data at time T i and the state prediction data at time T i The goal of the first Kalman filter is to, at a given time T i ​In the case of state observation data, measurement values, and state prediction data, the state prediction data is corrected using the state observation data and the measured quantity to obtain a corresponding state estimation result.

[0080] A third sensor, configured to obtain third sensor information corresponding to one of the plurality of timestamps of the legged robot. As described above, the third sensor in the legged robot may include a camera or a camera (hereinafter also called a visual sensor) that provides visual information. Visual odometry can convert the image information collected by the visual sensor into the position information and attitude information of the legged robot in the world coordinate system. The third sensor is, for example, a binocular single-channel camera, and each time it takes a photo of 1456 * 1088 pixels, and each pixel in the photo is represented by eight bits. The third sensor can trigger a shot every 0.1 s. Of course, the present disclosure is not limited thereto, and the third sensor can also trigger a shot at a longer or shorter time interval. The third sensor is also, for example, a monocular single-channel camera, and the resolution of the photo it takes is 640x576 or 512x512, and each pixel in the photo is represented by 16 bits. Of course, the present disclosure is not limited thereto.

[0081] A second Kalman filter, configured to determine the second state information of the legged robot based on the third sensor information corresponding to one of the plurality of timestamps and the first state information, using the second Kalman filter. Optionally, the second Kalman filter further includes visual odometry. Visual odometry can determine the orientation and position of the legged robot by analyzing a series of image sequences.

[0082] In a specific implementation, the second Kalman filter determines the second state information of the legged robot at timestamp T i The general principle is as follows:

[0083] First, the visual sensor can be called to collect the third sensor information of the legged robot at timestamp T i-c As described above, there is a delay in the data collected by the third sensor. The third sensor information collected at timestamp T i-c may be calculated and parsed by the visual odometry at timestamp T i to obtain the measurement information at timestamp T i-c As an example, the maximum delay of the third sensor information is the duration corresponding to c timestamps. Similarly, the measurement information may include: the values of a plurality of state observables. The plurality of measurement information values include: the position of the centroid of the legged robot at timestamp T i-c the velocity of the centroid of the legged robot at timestamp T i-c and so on.

[0084] Then, historical information can be obtained from the first Kalman filter, which includes the timestamp T calculated by the first Kalman filter. i-c Up to the timestamp T i of the state estimation result. That is, the historical data window of the first Kalman filter is the duration corresponding to c timestamps. The second Kalman filter can be based on the measurement information of the timestamp T i-c obtained by the visual odometer and the state estimation result of the timestamp T i-c from the first Kalman filter in the historical information to determine the corrected timestamp T i-c of the state estimation result as the second state information of the timestamp T i-c Based on the state estimation result of the corrected timestamp T i-c the second Kalman filter can predict the state estimation result of the timestamp T i-c+1 And so on, until the second Kalman filter calculates the state estimation result of the timestamp T i as the second state information.

[0085] Furthermore, the first Kalman filter is further configured to determine the state information of the legged robot at the current moment based on the second state information and the first state information corresponding to the legged robot at the current moment.

[0086] As shown above, assuming the current moment is T b , the above-mentioned first sensor has collected the first sensor information at the moment T b and the joint encoder has also collected the second sensor information at the moment T b At the same time, the second Kalman filter can also calculate the state increment ΔS based on the first state estimation result S a provided by the first Kalman filter and the third sensor information corresponding to the timestamp T a At this time, the first Kalman filter can calculate the state information corresponding to the moment T b Based on the above information. Compared with the first Kalman filter directly using the state estimation result estimated by the first Kalman filter at T b-1 to estimate the state information at the moment T b , the state information of the moment T b estimated using the state increment ΔS from the second Kalman filter is more accurate.

[0087] As Figure 4 shown, the first Kalman filter can output the state information of the current moment of the legged robot to the controller at a first frequency (for example, Figure 4 shown as 0.1 to 1 kHz). At the same time, the second Kalman filter can be at a second frequency (for example, Figure 4The output of the second state information to the first Kalman filter is at 10 to 50 Hz as shown. Therefore, the first Kalman filter can adjust the state information at the current moment based on the second state information at intervals of 0.02 s to 0.1 s, thus avoiding cumulative errors.

[0088] Thus, the device 40 of the present disclosure fuses the sensor information of different sensors operating at different frequencies and the second sensor information, and uses two Kalman filters to solve problems such as differences in fusing different sensor information and different delays, achieving multi-sensor fusion state estimation with high real-time performance, high robustness, and high effectiveness, and significantly reducing the cumulative error caused by state estimation during long-term movement of the legged robot. In particular, the device 40 implementing the method 20 simplifies the analysis and adjustment of the Kalman filter and reduces the running time. In addition, the second Kalman filter can be directly turned off when the third sensor information is unavailable without affecting the operation of the entire state estimation device.

[0089] Next, referring to Figure 5 and Figure 6 , in combination with Figure 1 describe the legged robot 100, Figures 2 to 3 the method 20 described, and referring to Figure 4 the device 40 described, further describe some details of the above-mentioned components performing the above-mentioned operations. Among them, Figure 5 is a schematic diagram showing the state estimation device 40 for a legged robot implementing the method 20 according to an embodiment of the present disclosure. Figure 6 is a comparison diagram showing the execution effect of the method 20 according to an embodiment of the present disclosure and a traditional solution.

[0090] Referring to Figure 5 , in combination with the above content, a prediction model can be used to observe each state quantity (this process is also called state prediction). The input of the prediction model can optionally include the first sensor information. Among them, the first sensor information includes: the rotation matrix R between the world coordinate system and the body coordinate system (hereinafter also denoted as W R B , representing the rotation matrix of the body coordinate system B relative to the world coordinate system W), the angular velocity ω in the body coordinate system, the acceleration a in the body coordinate system, and so on. Let the prediction model output x as x = [p T v T p1 T p2 T p3 T p4 T T .

[0091] Among them, p represents the position of the body center of mass of the legged robot in the world coordinate system, v represents the velocity of the body center of mass in the world coordinate system,​ Represents the foot end position in the world coordinate system, represents the foot end velocity in the world coordinate system (assuming the foot end velocity is zero, that is, there is no slip between the foot end and the ground), f i is the index of the robotic leg. Taking Figure 1 the legged robot 100 shown in as an example, at f i = 1, represents the foot end position of the left front leg in the world coordinate system, represents the foot end velocity of the left front leg in the world coordinate system. At f i = 2, represents the foot end position of the right front leg in the world coordinate system, represents the foot end velocity of the right front leg in the world coordinate system. At f i = 3, represents the foot end position of the left hind leg in the world coordinate system, represents the foot end velocity of the left hind leg in the world coordinate system. At f i = 4, represents the foot end position of the right hind leg in the world coordinate system, represents the foot end velocity of the right hind leg in the world coordinate system.

[0092] Specifically, the prediction model can respectively predict the body position, body velocity, and foot end position of the legged robot based on the first equation (1) to the third equation (3) described below to obtain the above x. The first equation (1) is also called the discrete prediction equation of the legged robot body position. The second equation (2) is also called the discrete prediction equation of the legged robot body velocity. The third equation (3) is also called the discrete prediction equation of the legged robot foot end position.

[0093]

[0094] v k+1 = v k +( W R B a k + g)dt (2)

[0095]

[0096] where a k is the acceleration in the body coordinate system, dt is the time step, g represents the acceleration due to gravity, and the subscript k represents the index of the time step.

[0097] Referring to Figure 5 , combining the above content, an observation model can also be used to observe each state quantity (this process is also called state observation). The observation model can be at least partially based on Figure 5The shown leg kinematic model is used for state observation. The observation model can take the first sensor information and the second sensor information as inputs and the state observation data as outputs.

[0098] The second sensor information includes joint angle information q. The state observation data includes the position p of the foot tip relative to the center of mass in the world coordinate system rel , and the velocity of the foot tip relative to the center of mass in the world coordinate system Optionally, the observation model can also take the foot tip touchdown detector as an input to obtain more accurate foot tip observation data.

[0099] Specifically, the leg kinematic model can indicate the relative position and velocity relationship between the center of mass of the legged robot and the foot tip, which is used to eliminate the state drift between the center of mass and the foot tip. For example, in a specific implementation, determining the first state information of the legged robot based on the first sensor information and the second sensor information using the first Kalman filter can further include: determining the motion information of the legged robot based on the second sensor information and the leg kinematic model of the legged robot; determining the first state information of the legged robot using the first Kalman filter at least partially based on the motion information.

[0100] In one example, the observation model can calculate the relative position measurement residual between the body of the legged robot and the foot tip and the relative velocity measurement residual between the body of the legged robot and the foot tip based on the leg kinematic model of the legged robot, and then use the relative position measurement residual between the body of the legged robot and the foot tip and the relative velocity measurement residual between the body of the legged robot and the foot tip as part of the state observation data.

[0101] The observation model can correspondingly calculate the measurement residuals using the fourth equation (4) and the fifth equation (5). Among them, the fourth equation (4) is used to calculate the relative position measurement residual between the body of the legged robot and the foot tip, and the fifth equation (5) is used to calculate the relative velocity measurement residual between the body of the legged robot and the foot tip. Among them, ω b is the angular velocity in the body coordinate system.

[0102]

[0103]

[0104] As described above, the first Kalman filter can be configured to determine the first state information of the legged robot based on the first sensor information corresponding to the multiple timestamps and the second sensor information. Assume that the first Kalman filter at time T k (corresponding to the k-th time step), obtains the states of each joint motor at time T kThe control input u k and the first sensor information z collected by the first sensor at time T k k . At the same time, the first Kalman filter also knows the prior state corresponding to time T k-1 The goal of the first Kalman filter is to estimate the state mean μ and covariance σ based on the above various values. The state space equation of the first Kalman filter can be shown by the sixth equation (6) and the seventh equation (7).

[0105] x k = Ax k-1 + Bu k + m k (6)

[0106] z k = Cx k-1 + n k (7)

[0107] where m k ~ N(0, Q k ) is the prediction noise, n k ~ N(0, R k ) is the observation noise, A is the state transition matrix, B is the control matrix, and C is the observation matrix. The first Kalman filter can process the state observation data and state prediction data with the eighth equation (8) to the twelfth equation (12) to fuse and obtain the state estimation result corresponding to time step k.

[0108]

[0109]

[0110]

[0111]

[0112]

[0113] where P is the error covariance matrix, and the initial value is set by itself. K is the Kalman gain matrix, I is the identity matrix, and x - represents the prior value of x. represents the estimated value of x. That is, identifies the prior estimated value of P k . identifies the prior estimated value of x k . Among them, The value is a combination of the relative position measurement residual between the legged robot body and the foot end calculated by the fourth equation (4) and the relative velocity measurement residual calculated by the fifth equation (5). Among them, z k is also called the measured quantity, which is obtained by kinematic solution of the second sensor information; is also called the observed quantity or state observation data. Thus, can be defined as the residual. Therefore, the first Kalman filter can estimate the state estimation result at time T k The corresponding state estimation result can be used as the first state information after information synchronization and output to the second Kalman filter. As an example, the first Kalman filter estimates the state estimation result at time T k The corresponding state estimation result includes the posterior state at time T k and the posterior covariance

[0114] Continue to refer to Figure 5 , assuming that the third sensor is the aforementioned vision sensor. The third sensor information z' output by the vision sensor s is z' s =[p vo T v vo T , where p vo is the body position in the world coordinate system measured by the vision sensor at time step s, and v vo is the body velocity in the world coordinate system measured by the vision sensor at time step s. Thus, the state space equation corresponding to the second Kalman filter can be shown by the thirteenth equation (13).

[0115] z' s =C' s x s +n' s (13)

[0116] where n' k ~N(0,R' k ), s = k - N. That is, the output of the third sensor information is delayed by N time steps (timestamps) relative to the information of the above-mentioned first Kalman filter. Therefore, the third sensor and the first state information can be synchronized to obtain the measurement error for correcting the first state information.

[0117] ​Specifically, as described above, since the visual data has a certain delay, preferably, the first state information and the error covariance matrix can be time-synchronized with the third sensor information z and the error covariance matrix corresponding to the third sensor. That is, as described above, the third sensor information corresponding to the time step s, the state estimation result of the first Kalman filter corresponding to the time step s, and the error covariance matrix can be used as the iterative parameters in the following equations, and fusion is performed based on the principle of the above Kalman filter.

[0118] For example, the third sensor can use the fourteenth equation (14) to obtain the relative position measurement residual p for correcting the first state information vo -p and the relative velocity measurement residual v vo -v.

[0119]

[0120] Then, the second Kalman filter can fuse the above time-synchronized first state information and third sensor information based on the fifteenth equation (15) to the seventeenth equation (17) to obtain the second state information. Among them, the relative position measurement residual and the relative velocity measurement residual can be used to form the seventeenth equation

[0121]

[0122]

[0123]

[0124] where P′ is the error covariance matrix used by the second Kalman filter. K′ is the Kalman gain matrix used by the second Kalman filter, I is the identity matrix, and x - represents the prior value of x. Identifies the prior estimate of the state estimate value output by the second Kalman filter. As described above, is equal to the posterior state output by the first Kalman filter at the time step s is equal to the posterior covariance output by the first Kalman filter at the time step s Thus, it is realized that the posterior state and posterior covariance obtained by the first Kalman filter are used as the prior state and prior covariance of the second Kalman filter to be fused with the observation information of the second Kalman filter, and then the state estimation result corresponding to the time step s is obtained as the second state information.

[0125] Since the first Kalman filter can also combine the first sensor information and the second sensor information corresponding to time T after receiving the second state information corresponding to time T, and determine the state estimation result corresponding to time T based on the above first equation (1) to the twelfth equation (12). k After receiving the corresponding second state information at time T, it can also combine the first sensor information and the second sensor information corresponding to time T k and determine the state estimation result corresponding to time T based on the above first equation (1) to the twelfth equation (12). k At this time, the first Kalman filter can be based on the state increment ΔS between

[0126] and and superimpose the state increment ΔS on the state estimation result corresponding to the first Kalman filter at time T k to correct the state estimation result at time step k.

[0127] In addition, in some other examples, the second Kalman filter can also output the state estimation result corresponding to time T estimated from time step s as the second state information to the first Kalman filter through N repeated iterations to correct the current state information. The present disclosure is not limited thereto. k The state estimation result corresponding to time T estimated from time step s is output to the first Kalman filter as the second state information to correct the current state information. The present disclosure is not limited thereto.

[0128] As Figure 6 shown, three curves corresponding to the ground truth data of the motion capture system, the estimation results of the traditional method, and the estimation results obtained by using method 20 are shown. The horizontal axis is time and the vertical axis is position data. After the legged robot moves freely for 200 seconds, the cumulative errors in the X and Y directions of the position are reduced by 96.08% and 62.52% respectively, and the root mean square errors are reduced by 92.94% and 58.63% respectively. In addition, the root mean square errors of the speed are also reduced by 31.05% and 16.47% respectively. It can be seen that method 20 can significantly reduce the cumulative error after fusing visual data.

[0129] Therefore, the device 40 of the present disclosure fuses the sensor information of different sensors operating at different frequencies and the second sensor information, and uses two Kalman filters to solve problems such as different frequencies and different delays of fusing different sensor information, realizing multi-sensor fusion state estimation with high real-time performance, high robustness, and high effectiveness, and significantly reducing the cumulative error caused by state estimation during the long-term movement of the legged robot. In particular, the device 40 implementing method 20 simplifies the analysis and adjustment of the Kalman filter and reduces the running time. In addition, the second Kalman filter can be directly turned off when the third sensor information is unavailable without affecting the operation of the entire state estimation device 40.

[0130] Optionally, the present disclosure also provides a legged robot, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0131] The present disclosure also provides a computer-readable storage medium for storing a computer program. The computer-readable storage medium can be applied to a legged robot, and the computer program causes the legged robot to execute the corresponding processes in the state estimation method in the embodiments of the present disclosure. For the sake of brevity, it will not be elaborated here.

[0132] The present disclosure also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. The processor of the legged robot reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the legged robot to execute the corresponding processes in the state estimation method in the embodiments of the present disclosure. For the sake of brevity, it will not be elaborated here.

[0133] The present disclosure also provides a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the legged robot reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the legged robot to execute the corresponding processes in the state estimation method in the embodiments of the present disclosure. For the sake of brevity, it will not be elaborated here.

[0134] According to another aspect of the present disclosure, an electronic device is also provided for implementing the method according to the embodiments of the present disclosure. Figure 7 A schematic diagram of an electronic device 2000 according to an embodiment of the present disclosure is shown.

[0135] As Figure 7 shown, the electronic device 2000 may include one or more processors 2010 and one or more memories 2020. Among them, computer-readable code is stored in the memory 2020, and when the computer-readable code is run by the one or more processors 2010, the above-mentioned method can be executed.

[0136] The processor in the embodiments of the present disclosure can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, operations and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., and can be of the X86 architecture or the ARM architecture.

[0137] In general, the various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented as non-limiting examples in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0138] For example, the method or apparatus according to the embodiments of the present disclosure may also be implemented by means of Figure 8 the architecture of the computing device 3000 shown. As Figure 8 shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for the processing and / or communication of the method provided by the present disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 8 the architecture shown is only exemplary, and when implementing different devices, one or more components shown in the Figure 8 computing device may be omitted according to actual needs.

[0139] According to another aspect of the present disclosure, a computer-readable storage medium is also provided. Figure 9 A schematic diagram of the storage medium 4000 according to the present disclosure is shown.

[0140] As Figure 9As shown, computer-readable instructions 4010 are stored on the computer storage medium 4020. When the computer-readable instructions 4010 are run by a processor, the methods according to the embodiments of the present disclosure described with reference to the above figures can be executed. The computer-readable storage medium in the embodiments of the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM). It should be noted that the memories of the methods described herein are intended to include, but are not limited to, these and any other suitable types of memories. It should be noted that the memories of the methods described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0141] Embodiments of the present disclosure also provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods according to the embodiments of the present disclosure.

[0142] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0143] In general, the various example embodiments of the present disclosure may be implemented in hardware or a dedicated circuit, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that can be executed by a controller, a microprocessor, or other computing devices. When aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented as non-limiting examples in hardware, software, firmware, a dedicated circuit or logic, general hardware or a controller or other computing devices, or some combination thereof.

[0144] The example embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art should understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A state estimation method for a legged robot, the method comprising: Obtaining first sensor information and second sensor information of the legged robot, and based on the first sensor information and the second sensor information, using a first Kalman filter to determine first state information of the legged robot, and saving the first state information for a period of time as historical information of a second Kalman filter; Obtaining third sensor information of the legged robot, and based on the third sensor information and the historical information, using the second Kalman filter to determine second state information of the legged robot; And Based on the second state information of the legged robot, updating the first state information of the legged robot at the current moment to determine the state information of the legged robot at the current moment, wherein using the first Kalman filter to determine the first state information of the legged robot further includes: Based on the second sensor information and based on the leg kinematic model of the legged robot, determining motion information of the legged robot; At least partially based on the motion information, using the first Kalman filter to determine the first state information of the legged robot.

2. The method according to claim 1, wherein The first sensor information is collected by a first sensor operating at a first frequency, the second sensor information is collected by a second sensor operating at a second frequency, the third sensor information is collected by a third sensor operating at a third frequency, and the first frequency, the second frequency, and the third frequency are different from each other.

3. The method according to claim 2, wherein The second sensor information is collected by a joint encoder operating at the second frequency, based on the first sensor information and the second sensor information.

4. The method according to claim 1, wherein, The using the first Kalman filter to determine the first state information of the legged robot includes: Determining the first sensor information and the second sensor information with timestamps, and sequentially using the first sensor information and the second sensor information as inputs to the first Kalman filter; and Performing state estimation through the first Kalman filter to obtain the first state information of the legged robot with a timestamp, and saving the first state information with a timestamp as historical information, where the first state information includes multiple state estimation results with timestamps output by the first Kalman filter.

5. The method according to claim 4, wherein, The using the second Kalman filter to determine the second state information of the legged robot includes: Determining the timestamp corresponding to the third sensor information; Using the historical information corresponding to the timestamp of the third sensor information and the third sensor information as inputs to the second Kalman filter; and Performing state estimation through the second Kalman filter to obtain the state estimation result corresponding to the timestamp of the third sensor information as the second state information.

6. The method according to claim 5, wherein, The performing state estimation through the second Kalman filter to obtain the second state information of the legged robot includes: Based on the timestamp corresponding to the third sensor information, determining the state estimation result corresponding to the timestamp in the historical information; Based on the third sensor information and the state estimation result corresponding to the timestamp in the historical information, use the second Kalman filter for state estimation to obtain the state estimation result corresponding to the timestamp as the second state information.

7. The method according to claim 6, wherein, Updating the first state information of the legged robot at the current moment to determine the state information of the legged robot at the current moment includes: Superimpose the state increment obtained from the second state information onto the first state information corresponding to the current moment of the legged robot to determine the state information of the legged robot at the current moment.

8. The method according to claim 4, wherein The first Kalman filter includes a prediction model and an observation model. The prediction model obtains a predicted state and corresponding covariance through the first sensor information, and the observation model obtains an observed state and corresponding covariance through the second sensor information.

9. The method according to claim 6, wherein, Using the second Kalman filter for state estimation to obtain the state estimation result corresponding to the timestamp includes: Taking the maximum value of the delay of the third sensor information as the historical data window, and saving the posterior state and posterior covariance obtained by the first Kalman filter within the historical data window interval; Fuse the posterior state and posterior covariance as the prior state and prior covariance of the second Kalman filter with the observation information of the second Kalman filter to obtain the state estimation result corresponding to the timestamp as the second state information.

10. The method according to claim 8, wherein, The observation model performs state observation at least partially based on the leg kinematic model of the legged robot, and the leg kinematic model indicates the relative position and velocity relationship between the center of mass and the foot end of the legged robot.

11. The method according to claim 10, wherein, The observation model performing state observation includes: Based on the leg kinematic model of the legged robot, calculate the relative position measurement residual between the body and the foot end of the legged robot and the relative velocity measurement residual between the body and the foot end of the legged robot. Take the relative position measurement residual between the body and the foot end of the legged robot and the relative velocity measurement residual between the body and the foot end of the legged robot as part of the state observation data.

12. The method according to claim 6, wherein, Based on the third sensor information, correcting the state estimation result corresponding to the timestamp in the first state information to obtain the state estimation result corresponding to the timestamp in the second state information includes: Based on the third sensor information, obtain the position and velocity of the body of the legged robot in the world coordinate system at the timestamp corresponding to the third sensor information; and Based on the position and velocity of the body of the legged robot in the world coordinate system at the timestamp corresponding to the third sensor information, calculate the cumulative error for correcting the state estimation information of the first Kalman filter.

13. A state estimation device for a legged robot, the device includes: A first sensor configured to obtain the first sensor information of the legged robot; A second sensor configured to obtain the second sensor information of the legged robot; A third sensor configured to obtain the third sensor information of the legged robot; A first Kalman filter, configured to determine first state information of the legged robot based on the first sensor information and the second sensor information, and use the first state information saved for a period of time as historical information; A second Kalman filter, configured to determine second state information of the legged robot by using the second Kalman filter based on the third sensor information and the historical information; wherein the first Kalman filter is further configured to determine state information of the legged robot at the current moment based on the second state information and the first state information corresponding to the legged robot at the current moment.

14. A computer device, comprising an input interface and an output interface, characterized in that, Further comprising: a processor, adapted to implement one or more instructions; and a computer storage medium; the computer storage medium stores one or more instructions, and the one or more instructions are adapted to be loaded and executed by the processor to perform the state estimation method of the legged robot according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more instructions, and the one or more instructions are adapted to be loaded and executed by the processor to perform the state estimation method of the legged robot according to any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes a computer program; when the computer program is executed by the processor, it implements the state estimation method of the legged robot according to any one of claims 1-12.

Images