Quadruped robot dog inertial navigation method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a quadruped robot dog inertial navigation method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring inertial data of a mechanical leg through an inertial measurement unit mounted on the mechanical leg of the quadruped robot dog; determining the landing moment of the mechanical leg according to the inertial data; at the landing moment, the integral velocity of the mechanical leg is corrected to be zero; and updating the navigation state variable of the quadruped robot dog by taking the zero speed as an observed quantity.

Description

Technical Field

[0001] The present disclosure relates to the field of robotics technology, and in particular to an inertial navigation method, device, electronic device, computer-readable storage medium, and computer program product for a quadruped robot dog. Background Art

[0002] Robotics develops machines that can perform specific operations and plays an important role in manufacturing, disaster relief, resource exploration and other fields. Currently, wheeled robots occupy a major position in the robotics world. This type of robot controls movement by driving wheels with motors, and its implementation method and control scheme are relatively easy. However, the design of the wheels makes it impossible for wheeled robots to cross obstacles, cannot work on uneven surfaces, and are inefficient when dealing with turns. In view of the various shortcomings of wheeled robots, more and more scholars are turning their attention to legged robots.

[0003] Legged robots use legs instead of wheels, which gives them a higher degree of freedom and allows them to move freely in different environments. A quadruped robot dog is a type of legged robot. It is smaller in size and has a higher degree of freedom of movement. It has broad application prospects in disaster relief sites and resource exploration.

[0004] In order to further promote the quadruped robot dog, it is necessary to achieve stable and reliable positioning of the quadruped robot dog on different terrains. Since the movement of the quadruped robot dog is completely achieved by the motor-driven mechanical legs, some scholars use the output of the motor to determine the position of the quadruped robot dog. However, in practical applications, the quadruped robot dog moving on a smooth surface may slip; on grass or muddy ground, there may be factors such as ground deformation.

[0005] Since positioning solely by motors is not reliable, scholars have introduced the Inertial Measurement Unit (IMU) in the field of navigation. The inertial measurement unit can provide high-precision navigation results in a short time, but there is a problem of error drift, and huge navigation errors will occur when working for a long time. In order to solve the problem of cumulative errors, lidar, cameras, etc. are introduced, and the error of the inertial measurement unit divergence is limited by multi-source fusion methods. However, the above-mentioned lidar, camera and other sensors are expensive, and deploying them on a quadruped robot dog will increase the overall weight, and it is difficult to work normally in extreme environments. These factors limit the application scenarios of quadruped robot dogs.

[0006] In summary, the application of quadruped robot dogs is inseparable from stable and reliable navigation, but the current navigation methods have problems such as limited application scenarios and short reliable working time, making it difficult to achieve high-precision positioning of quadruped robot dogs in multiple scenarios. In actual use, quadruped robot dogs work in scenarios with degraded visual features, and their working time also needs to meet application requirements. The current methods are difficult to achieve long-term high-precision positioning in extreme environments, which restricts the further use and promotion of quadruped robot dogs. Summary of the invention

[0007] The present invention provides an inertial navigation technology solution for a quadruped robot dog.

[0008] According to one aspect of the present disclosure, there is provided an inertial navigation method for a quadruped robot dog, comprising:

[0009] By installing an inertial measurement unit on a mechanical leg of the quadruped robot dog, inertial data of the mechanical leg is collected;

[0010] Determining the landing moment of the mechanical leg according to the inertial data;

[0011] At the landing moment, correcting the integrated speed of the mechanical leg to zero;

[0012] Taking zero speed as an observation quantity, the navigation state variables of the quadruped robot dog are updated.

[0013] In a possible implementation, the inertial data includes acceleration and angular velocity;

[0014] Determining the landing time of the mechanical leg according to the inertial data includes:

[0015] For any moment, in response to the modulus of acceleration at the moment being less than the acceleration modulus threshold, and the modulus of angular velocity at the moment being less than the angular velocity modulus threshold, the moment is determined to be the landing moment of the mechanical leg.

[0016] In a possible implementation, taking the zero speed as an observation quantity and updating the navigation state variable of the quadruped robot dog includes:

[0017] Input zero velocity as an observation into the Kalman filter;

[0018] The navigation state variables of the quadruped robot dog are updated through the Kalman filter.

[0019] In a possible implementation manner, the navigation state variable includes at least part of the following: position, heading, and speed.

[0020] In a possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the method further includes:

[0021] Determining the scale factor matrix and zero bias of the accelerometer and the scale factor matrix and zero bias of the gyroscope according to the turntable calibration result;

[0022] Compensating the acceleration output by the accelerometer according to the scale factor matrix and zero bias of the accelerometer;

[0023] The angular velocity output by the gyroscope is compensated according to the scale factor matrix and the zero bias of the gyroscope.

[0024] In a possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the method further includes:

[0025] Controlling the four-legged robot dog to remain stationary for a preset period of time;

[0026] Determining the zero bias of the accelerometer according to the average value of the acceleration output by the accelerometer during the static phase;

[0027] Determining the zero bias of the gyroscope according to the average value of the angular velocity output by the gyroscope during the static phase;

[0028] Compensating the acceleration output by the accelerometer according to the zero bias of the accelerometer;

[0029] The angular velocity output by the gyroscope is compensated according to the zero bias of the gyroscope.

[0030] In a possible implementation, the method further includes:

[0031] Performing a dot product operation on the initial acceleration of the inertial measurement unit in the carrier coordinate system and the initial heading of the inertial measurement unit in the navigation coordinate system to obtain a dot product result;

[0032] Determine the angle between the carrier coordinate system and the navigation coordinate system according to the dot product result;

[0033] The initial heading of the inertial measurement unit is determined using the angle.

[0034] In a possible implementation, the method further includes:

[0035] Obtaining the latitude and longitude information of the location of the inertial measurement unit;

[0036] Calculate the angular velocity of the earth's rotation at the location according to the latitude and longitude information;

[0037] The initial heading of the inertial measurement unit relative to the earth fixed reference system is determined by using the earth rotation angular velocity and the angular velocity measured by the inertial measurement unit.

[0038] In a possible implementation, the method further includes:

[0039] Based on four inertial measurement units arranged on the four mechanical legs of the quadruped robot dog, navigation is performed respectively to obtain navigation state variables of the four mechanical legs;

[0040] Kinematic constraints are imposed on the navigation state variables of the four robotic legs.

[0041] In a possible implementation, kinematically constraining the navigation state variables of the four mechanical legs includes:

[0042] Dividing the four mechanical legs into two mechanical leg pairs;

[0043] For any pair of mechanical legs, in response to one mechanical leg in the pair of mechanical legs landing, determining a distance between the other mechanical leg in the pair of mechanical legs and the mechanical leg;

[0044] In response to the distance being greater than or equal to a distance threshold corresponding to the pair of mechanical legs, determining a reference position of the other mechanical leg according to the distance threshold;

[0045] The position of the other robot leg is corrected to the reference position.

[0046] In a possible implementation, the method further includes:

[0047] In response to two mechanical legs in the pair of mechanical legs landing at the same time, the distance between the two mechanical legs is obtained through a motor encoder to obtain a distance threshold corresponding to the pair of mechanical legs.

[0048] In a possible implementation, kinematically constraining the navigation state variables of the four mechanical legs includes:

[0049] In response to a heading deviation between any two of the four mechanical legs being greater than or equal to an angle threshold, a heading correction is performed according to the headings of the other two of the four mechanical legs.

[0050] In a possible implementation, the method further includes:

[0051] identifying the type of ground on which the quadruped robot dog is located based on the inertial data;

[0052] The acceleration modulus threshold and the angular velocity modulus threshold are determined according to the terrain type.

[0053] In a possible implementation, identifying the type of ground on which the quadruped robot dog is located according to the inertial data includes:

[0054] The inertial data is input into a pre-trained long short-term memory network, and the type of ground on which the quadruped robot dog is located is identified through the long short-term memory network.

[0055] In one possible implementation,

[0056] Before inputting the inertial data into the pre-trained long short-term memory network, the method further comprises: performing Gaussian filtering on the inertial data;

[0057] After identifying the type of ground on which the quadruped robot dog is located by using the long short-term memory network, the method further includes: performing mean filtering and continuity judgment on the output result of the long short-term memory network.

[0058] According to one aspect of the present disclosure, there is provided a quadruped robot dog inertial navigation device, comprising:

[0059] A collection module, used for collecting inertial data of the mechanical legs through an inertial measurement unit installed on the mechanical legs of the quadruped robot dog;

[0060] A first determination module, used to determine the landing moment of the mechanical leg according to the inertial data;

[0061] A correction module, used for correcting the integrated speed of the mechanical leg to zero at the landing moment;

[0062] The updating module is used to update the navigation state variables of the quadruped robot dog by taking the zero speed as an observation quantity.

[0063] In a possible implementation, the inertial data includes acceleration and angular velocity;

[0064] The first determining module is used for:

[0065] For any moment, in response to the modulus of acceleration at the moment being less than the acceleration modulus threshold, and the modulus of angular velocity at the moment being less than the angular velocity modulus threshold, the moment is determined to be the landing moment of the mechanical leg.

[0066] In a possible implementation, the update module is used to:

[0067] Input zero velocity as an observation into the Kalman filter;

[0068] The navigation state variables of the quadruped robot dog are updated through the Kalman filter.

[0069] In a possible implementation manner, the navigation state variable includes at least part of the following: position, heading, and speed.

[0070] In a possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the device further includes:

[0071] A second determination module, used to determine the scale factor matrix and zero bias of the accelerometer, and the scale factor matrix and zero bias of the gyroscope according to the result of the turntable calibration;

[0072] A first acceleration compensation module, used for compensating the acceleration output by the accelerometer according to the scale factor matrix and zero bias of the accelerometer;

[0073] The first angular velocity compensation module is used to compensate the angular velocity output by the gyroscope according to the scale factor matrix and zero bias of the gyroscope.

[0074] In a possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the device further includes:

[0075] A control module, used for controlling the four-legged robot dog to remain stationary for a preset period of time;

[0076] A third determination module is used to determine the zero bias of the accelerometer according to the average value of the acceleration output by the accelerometer in the static stage;

[0077] A fourth determination module, configured to determine the zero bias of the gyroscope according to an average value of the angular velocity output by the gyroscope during a stationary phase;

[0078] A second acceleration compensation module, used for compensating the acceleration output by the accelerometer according to the zero bias of the accelerometer;

[0079] The second angular velocity compensation module is used to compensate the angular velocity output by the gyroscope according to the zero bias of the gyroscope.

[0080] In a possible implementation manner, the device further includes:

[0081] A dot product module, used for performing a dot product operation on the initial acceleration of the inertial measurement unit in the carrier coordinate system and the initial heading of the inertial measurement unit in the navigation coordinate system to obtain a dot product result;

[0082] A fifth determination module, configured to determine an angle between the carrier coordinate system and the navigation coordinate system according to the dot product result;

[0083] A sixth determination module is used to determine an initial heading of the inertial measurement unit using the angle.

[0084] In a possible implementation manner, the device further includes:

[0085] A first obtaining module is used to obtain the longitude and latitude information of the location of the inertial measurement unit;

[0086] A calculation module, used for calculating the angular velocity of the earth's rotation at the position according to the latitude and longitude information;

[0087] The seventh determination module is used to determine the initial heading of the inertial measurement unit relative to the earth's fixed reference system by using the earth's rotation angular velocity and the angular velocity measured by the inertial measurement unit.

[0088] In a possible implementation manner, the device further includes:

[0089] A navigation module, used to perform navigation based on four inertial measurement units arranged on the four mechanical legs of the quadruped robot dog, respectively, to obtain navigation state variables of the four mechanical legs;

[0090] The kinematic constraint module is used to perform kinematic constraints on the navigation state variables of the four mechanical legs.

[0091] In a possible implementation, the kinematic constraint module is used to:

[0092] Dividing the four mechanical legs into two mechanical leg pairs;

[0093] For any pair of mechanical legs, in response to one mechanical leg in the pair of mechanical legs landing, determining a distance between the other mechanical leg in the pair of mechanical legs and the mechanical leg;

[0094] In response to the distance being greater than or equal to a distance threshold corresponding to the pair of mechanical legs, determining a reference position of the other mechanical leg according to the distance threshold;

[0095] The position of the other robot leg is corrected to the reference position.

[0096] In a possible implementation manner, the device further includes:

[0097] The second obtaining module is used for obtaining the distance between the two mechanical legs through the motor encoder in response to the two mechanical legs in the mechanical leg pair landing at the same time, so as to obtain the distance threshold corresponding to the mechanical leg pair.

[0098] In a possible implementation, the kinematic constraint module is used to:

[0099] In response to a heading deviation between any two of the four mechanical legs being greater than or equal to an angle threshold, a heading correction is performed according to the headings of the other two of the four mechanical legs.

[0100] In a possible implementation manner, the device further includes:

[0101] A ground type identification module, used to identify the ground type on which the quadruped robot dog is located according to the inertial data;

[0102] An eighth determination module is used to determine the acceleration modulus threshold and the angular velocity modulus threshold according to the terrain type.

[0103] In a possible implementation, the ground type identification module is used to:

[0104] The inertial data is input into a pre-trained long short-term memory network, and the type of ground on which the quadruped robot dog is located is identified through the long short-term memory network.

[0105] In a possible implementation manner, the device further includes:

[0106] A Gaussian filter module, used for performing Gaussian filter processing on the inertial data;

[0107] The outlier processing module is used to perform mean filtering and continuity judgment on the output results of the long short-term memory network.

[0108] According to one aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0109] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0110] According to one aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0111] In the disclosed embodiment, the inertial measurement unit installed on the mechanical leg of the quadruped robot dog is used to collect the inertial data of the mechanical leg. According to the inertial data, the landing time of the mechanical leg is determined. At the landing time, the integrated speed of the mechanical leg is corrected to zero, and the zero speed is used as the observation quantity to update the navigation state variable of the quadruped robot dog, thereby applying the zero speed correction algorithm to the odometer of the quadruped robot dog, and positioning the quadruped robot dog through the pedestrian navigation method. The disclosed embodiment fixes the inertial measurement unit on the mechanical leg of the quadruped robot dog, and the navigation information of the quadruped robot dog can be solved by using the acceleration information and angular velocity information. By introducing the zero speed correction algorithm, according to the characteristic that the mechanical leg of the quadruped robot dog has a zero speed at the moment of touching the ground, the accelerometer error is periodically corrected, thereby suppressing the problem of error accumulation of the inertial measurement unit. The disclosed embodiment relies on the inertial measurement unit, does not need to introduce more sensors (such as laser radar, camera, etc.), does not rely on the external environment, and performs error correction based on the motion characteristics of the quadruped robot dog, and has a wide range of application scenarios. Therefore, the embodiments of the present disclosure provide a convenient solution for autonomous dead reckoning of a quadruped robot dog.

[0112] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0113] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0115] Figure 1 A flow chart of the inertial navigation method for a quadruped robot dog provided by an embodiment of the present disclosure is shown.

[0116] Figure 2 A schematic diagram showing a layout of an inertial measurement unit on an odometer of a quadruped robot dog in an inertial navigation method for a quadruped robot dog provided in an embodiment of the present disclosure.

[0117] Figure 3 A flow chart showing a zero-speed correction algorithm in the inertial navigation method for a quadruped robot dog provided in an embodiment of the present disclosure is shown.

[0118] Figure 4 A schematic diagram showing positioning test results of a zero-speed correction algorithm on the odometer of a quadruped robot dog in the inertial navigation method of a quadruped robot dog provided in an embodiment of the present disclosure.

[0119] Figure 5A schematic diagram showing the arrangement of inertial measurement units on the four legs of a quadruped robot dog in the inertial navigation method for a quadruped robot dog provided by an embodiment of the present disclosure is shown.

[0120] Figure 6 A schematic diagram of a positioning error constraint algorithm for a quadruped robot dog combining kinematics and motors in an inertial navigation method for a quadruped robot dog provided in an embodiment of the present disclosure is shown.

[0121] Figure 7 Schematic diagram showing different road surfaces.

[0122] Figure 8 A schematic diagram showing the structure of a long short-term memory network in the inertial navigation method for a quadruped robot dog provided in an embodiment of the present disclosure is shown.

[0123] Fig. 9 The confusion matrix after the long short-term memory network training in the inertial navigation method for a quadruped robot dog provided by an embodiment of the present disclosure is shown.

[0124] Fig.10 An overall schematic diagram of a ground contact type detection algorithm for a quadruped robot dog in an inertial navigation method for a quadruped robot dog provided in an embodiment of the present disclosure is shown.

[0125] Fig.11 A block diagram of an inertial navigation device for a quadruped robot dog provided by an embodiment of the present disclosure is shown.

[0126] Fig.12 A block diagram of an electronic device 1900 provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0127] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0128] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0129] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0130] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0131] In summary, the inertial navigation method of the quadruped robot dog in the related art has the following defects:

[0132] First, the positioning method that relies on motor output is greatly affected by the environment, dynamic model, and noise, and usually has low accuracy and robustness;

[0133] Second, the number of sensors (such as lidar, cameras, etc.) used in the positioning method of quadruped robot dogs that rely on inertial measurement units is large and expensive, which limits the use scenarios of quadruped robot dogs;

[0134] Third, the positioning method cannot distinguish between ground types, making it difficult to achieve high-precision positioning of a quadruped robot dog in a variety of terrains.

[0135] In order to solve the technical problems similar to those described above, the disclosed embodiment provides an inertial navigation method for a quadruped robot dog, by installing an inertial measurement unit on the mechanical leg of the quadruped robot dog, collecting the inertial data of the mechanical leg, determining the landing time of the mechanical leg according to the inertial data, correcting the integrated velocity of the mechanical leg to zero at the landing time, and using the zero velocity as an observation quantity to update the navigation state variable of the quadruped robot dog, thereby applying the zero velocity update (ZUPT) algorithm to the odometer of the quadruped robot dog, and positioning the quadruped robot dog by the pedestrian navigation method. The disclosed embodiment fixes the inertial measurement unit on the mechanical leg of the quadruped robot dog, and can use the acceleration information and angular velocity information to solve the navigation information of the quadruped robot dog. By introducing the zero velocity correction algorithm, according to the characteristic that the mechanical leg of the quadruped robot dog has a zero velocity at the moment of touching the ground, the accelerometer error is periodically corrected, thereby suppressing the error accumulation problem of the inertial measurement unit. The disclosed embodiment relies on an inertial measurement unit, does not require the introduction of more sensors (such as laser radar, camera, etc.), does not rely on the external environment, performs error correction based on the motion characteristics of the quadruped robot dog, and has a wide range of application scenarios. Therefore, the disclosed embodiment provides a convenient solution for autonomous dead reckoning of a quadruped robot dog.

[0136] The inertial navigation method for a quadruped robot dog provided by an embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0137] Figure 1A flow chart of the inertial navigation method for a quadruped robot dog provided by an embodiment of the present disclosure is shown. In one possible implementation, the execution subject of the inertial navigation method for a quadruped robot dog may be an inertial navigation device for a quadruped robot dog. For example, the inertial navigation method for a quadruped robot dog may be executed by a terminal device or a server or other electronic device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device or a wearable device, etc. In some possible implementations, the inertial navigation method for a quadruped robot dog may be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the quadruped robot dog inertial navigation method includes steps S11 to S14.

[0138] In step S11, inertial measurement units installed on the mechanical legs of the quadruped robot dog are used to collect inertial data of the mechanical legs.

[0139] In step S12, the landing moment of the mechanical leg is determined according to the inertial data.

[0140] In step S13, at the landing moment, the integrated speed of the robotic leg is corrected to zero.

[0141] In step S14, the zero speed is used as an observation quantity to update the navigation state variables of the quadruped robot dog.

[0142] In the disclosed embodiment, an inertial measurement unit (IMU) may be installed on the mechanical legs of the quadruped robot dog. Figure 2 A schematic diagram showing a layout of an inertial measurement unit on the odometer of a quadruped robot dog in the inertial navigation method of a quadruped robot dog provided by an embodiment of the present disclosure is shown. Figure 2 As shown, the inertial measurement unit (i.e. Figure 2 IMU module in the .

[0143] In the disclosed embodiment, an inertial measurement unit fixed to the mechanical leg of the quadruped robot dog can be used to collect inertial data of the mechanical leg of the quadruped robot dog. Among them, inertial data can refer to data collected by the inertial measurement unit. In some application scenarios, inertial data can also be called IMU information, IMU data, etc., which is not limited here.

[0144] In the disclosed embodiment, the inertial data collected by the inertial measurement unit may include acceleration and angular velocity, that is, the acceleration and angular velocity of the mechanical legs of the quadruped robot dog when the quadruped robot dog moves may be collected by the inertial measurement unit fixed to the mechanical legs of the quadruped robot dog.

[0145] In one example, the output frequency of the acceleration and angular velocity of the inertial measurement unit may be 104 Hz, which is not limited here.

[0146] Among them, packet loss may occur in the inertial data during transmission or processing, that is, some data points are not correctly recorded or transmitted, which may be caused by hardware failure, communication interference or software error. In one possible implementation, when processing inertial data, the system can check whether there is packet loss in the inertial data stream. For example, it can be done by checking the timestamp or sequence number of the data. If the receiving time interval or sequence number of the inertial data does not match the expectation, data packet loss can be detected. Once packet loss is detected, the system needs to decide how to deal with these inertial data. For example, if the packet loss is not serious, the system can try to use other data points to interpolate or estimate the lost inertial data. If the packet loss is serious, or the integrity of the inertial data is critical to navigation, the system can decide to discard these inertial data to avoid using incomplete inertial data to cause erroneous navigation decisions. By checking the data packet loss and deciding the availability of inertial data accordingly, the system can reduce positioning errors caused by hardware problems (such as sensor failure or communication problems). This is because inaccurate or incomplete inertial data, if used for navigation calculations, may cause serious deviations in the position and heading estimates of the quadruped robot dog.

[0147] In one possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the method further includes: determining a scale factor matrix and a zero bias of the accelerometer, as well as a scale factor matrix and a zero bias of the gyroscope based on a result of turntable calibration; compensating for an acceleration output by the accelerometer based on the scale factor matrix and the zero bias of the accelerometer; and compensating for an angular velocity output by the gyroscope based on the scale factor matrix and the zero bias of the gyroscope.

[0148] Turntable calibration is a calibration process that obtains the correspondence between the IMU output data and the real motion by fixing the IMU on a turntable that rotates in a known pattern. Through this process, the error characteristics of the IMU, namely the scale factor matrix and zero bias of the accelerometer and gyroscope, can be determined.

[0149] Among them, the scale factor is a parameter that describes the proportional relationship between the sensor output and the actual physical quantity. If the scale factor is inaccurate, the output of the sensor may systematically overestimate or underestimate the true value. Zero bias is the output value of the sensor when it is not subjected to any input (i.e., static state). Ideally, this value should be zero, but in reality, due to factors such as manufacturing errors, this value may not be zero.

[0150] Based on the results of the turntable calibration, the scale factor matrix and zero bias of the accelerometer and gyroscope can be determined. Then, the scale factor matrix and zero bias can be used to compensate the output data of the accelerometer and gyroscope. This means adjusting the original data to correct these systematic errors. That is, for the accelerometer, the measured acceleration is corrected using its scale factor matrix and zero bias to obtain more accurate acceleration data; for the gyroscope, the measured angular velocity is corrected using its scale factor matrix and zero bias to obtain more accurate angular velocity data.

[0151] By adopting this implementation method, the measurement accuracy of the inertial measurement unit can be significantly improved, thereby improving the navigation accuracy of the quadruped robot dog.

[0152] In another possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the method further includes: controlling the quadruped robot dog to remain stationary for a preset time; determining the zero bias of the accelerometer based on the average value of the acceleration output by the accelerometer during the stationary stage; determining the zero bias of the gyroscope based on the average value of the angular velocity output by the gyroscope during the stationary stage; compensating for the acceleration output by the accelerometer based on the zero bias of the accelerometer; and compensating for the angular velocity output by the gyroscope based on the zero bias of the gyroscope.

[0153] In this implementation, after the quadruped robot dog is started, the quadruped robot dog can be controlled to be stationary for a preset time. For example, the preset time can be 5 seconds, that is, after the quadruped robot dog is started, the quadruped robot can be controlled to be stationary for 5 seconds. By controlling the quadruped robot dog to be stationary for a preset time, the inertial measurement unit can have time to collect data in a non-moving state.

[0154] During the period when the quadruped robot dog is stationary, the accelerometer and gyroscope in the inertial measurement unit will continue to work and output data. The accelerometer will output the acceleration value in the stationary state, and the gyroscope will output the angular velocity value in the stationary state.

[0155] In theory, when the quadruped robot dog is completely still, the acceleration value measured by the accelerometer should be zero (except for the acceleration of gravity, which is usually handled separately in the navigation algorithm). However, the actual accelerometer will also have an output when it is still, and this output value is the zero bias of the accelerometer. This zero bias value can be obtained by calculating the average value of the acceleration output by the accelerometer during the stationary stage.

[0156] Similarly, when the quadruped robot dog is stationary, the angular velocity value measured by the gyroscope should also be zero. But in fact, the gyroscope will also have output when it is not rotating, and this output value is the zero bias of the gyroscope. This zero bias value can be obtained by calculating the average value of the angular velocity output by the gyroscope during the stationary phase.

[0157] Right now:

[0158]

[0159] Once the zero bias value of the accelerometer is obtained, the data can be compensated by subtracting this zero bias value from the actual output of the accelerometer to obtain more accurate acceleration data; similarly, once the zero bias value of the gyroscope is obtained, the data can be compensated by subtracting this zero bias value from the actual output of the gyroscope to obtain more accurate angular velocity data.

[0160] This implementation is simple to operate, does not require special equipment, and only requires the quadruped robot to remain still for a short period of time. This is a practical calibration method for quadruped robot dogs that can be performed regularly to improve the accuracy of inertial data. However, the zero bias obtained by this method may not be as accurate as turntable calibration because it relies on the stability of the environment (for example, vibrations in the ground may affect the results). Nevertheless, it can still significantly improve the accuracy of inertial data.

[0161] In a possible implementation, the method further includes: performing a dot product operation on the initial acceleration of the inertial measurement unit in the carrier coordinate system and the initial heading of the inertial measurement unit in the navigation coordinate system to obtain a dot product result; determining an angle between the carrier coordinate system and the navigation coordinate system based on the dot product result; and determining the initial heading of the inertial measurement unit using the angle.

[0162] In this implementation, after the raw inertial data is processed, the initial heading of the inertial measurement unit can be determined by a dual-vector method.

[0163] In this implementation, the carrier coordinate system is a coordinate system directly related to the physical carrier (a quadruped robot dog in the disclosed embodiment) on which the inertial measurement unit resides. In the carrier coordinate system, the accelerometer measures accelerations that are directly related to the motion of the carrier. The navigation coordinate system is a coordinate system used for navigation and positioning, and is usually aligned with an earth coordinate system (such as a geographic coordinate system). In the navigation coordinate system, heading refers to the azimuth of the carrier relative to north or some reference direction.

[0164] In this implementation, when the inertial measurement unit starts, it measures an initial acceleration value in the carrier coordinate system. This value usually includes the acceleration due to the carrier's motion and the acceleration of gravity. At the same time, the inertial measurement unit also measures an initial heading value in the navigation coordinate system, which is the initial azimuth of the carrier relative to north or a reference direction.

[0165] The angle between the carrier coordinate system and the navigation coordinate system can be calculated by performing a dot product operation on the initial acceleration vector in the carrier coordinate system and the initial heading vector in the navigation coordinate system. This angle is the rotation angle between the reference directions of the two coordinate systems. Once the angle between the carrier coordinate system and the navigation coordinate system is determined, this angle can be used to determine the initial heading of the inertial measurement unit. The formula is as follows:

[0166]

[0167] This implementation can directly use the measurement data of the inertial measurement unit to determine the initial heading without relying on external reference information. This is very helpful for improving the autonomy and robustness of the navigation system. In this way, the orientation of the inertial measurement unit in the navigation coordinate system can be quickly and accurately determined when it is started, providing an accurate reference for subsequent navigation and positioning.

[0168] In a possible implementation, the method further includes: obtaining longitude and latitude information of the location of the inertial measurement unit; calculating the angular velocity of the earth's rotation at the location based on the longitude and latitude information; and determining the initial heading of the inertial measurement unit relative to the earth's fixed reference system using the angular velocity of the earth's rotation and the angular velocity measured by the inertial measurement unit.

[0169] In this implementation, first, the longitude and latitude information of the geographical location of the inertial measurement unit can be obtained. This can be obtained through the Global Navigation Satellite System (GNSS) or other positioning technologies. After obtaining the longitude and latitude information of the location of the inertial measurement unit, the angular velocity of the earth's rotation at the location of the inertial measurement unit can be calculated based on the longitude and latitude information.

[0170] The gyroscope in the IMU can measure the angular velocity of the mechanical legs of the quadruped robot dog. When the IMU is stationary, the angular velocity measured by the gyroscope is mainly caused by the rotation of the earth. By comparing the angular velocity measured by the IMU with the angular velocity of the earth's rotation, the initial heading of the IMU relative to the earth's fixed reference frame can be determined.

[0171] Specifically, if the inertial measurement unit is stationary on the ground, then its heading angle can be determined by the effect of the earth's rotation. For example, if the inertial measurement unit is located in the northern hemisphere, then the local horizontal component of the earth's rotation angular velocity vector points to the north. The gyroscope can measure this component to help determine the heading angle. A key factor in this method is latitude, because latitude determines the magnitude of the local horizontal component of the earth's rotation angular velocity vector. For example, in an area with a latitude of 30°, when the east gyro zero bias is 0.01° / hr, the resulting heading error can be calculated by correlation.

[0172] In this implementation, the initial heading of the inertial measurement unit can be determined based on the longitude and latitude information of the location of the inertial measurement unit in combination with the angular velocity of the earth's rotation.

[0173] During the movement of the quadruped robot dog, each time the mechanical legs touch the ground, there will be a short period of zero speed. In the disclosed embodiment, the landing time of the mechanical legs of the quadruped robot dog can be determined based on the inertial data output by the inertial measurement unit.

[0174] In a possible implementation, the inertial data includes acceleration and angular velocity; determining the landing moment of the mechanical leg based on the inertial data includes: for any moment, in response to the modulus of acceleration at the moment being less than an acceleration modulus threshold, and the modulus of angular velocity at the moment being less than an angular velocity modulus threshold, determining the moment as the landing moment of the mechanical leg.

[0175] In this implementation, for any moment, if the modulus of acceleration at that moment is less than the acceleration modulus threshold, and the modulus of angular velocity at that moment is less than the angular velocity modulus threshold, then the moment can be determined to be the landing moment of the mechanical leg; if the modulus of acceleration at that moment is greater than or equal to the acceleration modulus threshold, or the modulus of angular velocity at that moment is greater than or equal to the angular velocity modulus threshold, then it can be determined that the moment is not the landing moment of the mechanical leg.

[0176] The stepping frequency of a quadruped robot dog is usually higher than that of a pedestrian, and the duration of each landing is shorter, so the threshold setting can be higher than the threshold setting in the pedestrian zero-speed correction algorithm. The formula for threshold judgment is as follows:

[0177]

[0178] In one example, the angular velocity modulus threshold ω Can be set to 0.05, acceleration modulus threshold a Can be set to 0.24.

[0179] In the disclosed embodiment, when the mechanical leg lands, the zero speed can be used as an observation quantity to correct the integrated speed of the mechanical leg, thereby realizing error correction of the inertial navigation algorithm. That is, when the mechanical leg lands, the zero speed can be used as an observation quantity to update the navigation state variables of the quadruped robot dog. In the application of inertial navigation and state estimation, the navigation state variables can represent a set of parameters describing the position and motion state of a carrier (such as a quadruped robot dog) in a navigation coordinate system.

[0180] In a possible implementation manner, the navigation state variable includes at least part of the following: position, heading, and speed.

[0181] Among them, the position can refer to the specific coordinates of the quadruped robot dog in the navigation coordinate system, usually the latitude, longitude and altitude in three-dimensional space (in some applications, only two-dimensional coordinates may be used). The position information tells us where the quadruped robot dog is.

[0182] Heading, also called direction or attitude, refers to the orientation of the quadruped robot dog. In navigation, heading usually refers to the azimuth of the quadruped robot dog relative to the geographic North Pole or a reference direction. Heading information tells us where the quadruped robot dog is facing.

[0183] Speed ​​refers to the speed at which the quadruped robot dog moves in the navigation coordinate system, including the magnitude (i.e., rate) and direction of the speed. Speed ​​information can be three-dimensional, including horizontal speed (lateral and longitudinal) and vertical speed (e.g., ascent or descent speed). Speed ​​information tells us how the quadruped robot dog moves.

[0184] In a possible implementation, taking zero speed as an observation quantity and updating the navigation state variables of the quadruped robot dog includes: inputting zero speed as an observation quantity into a Kalman filter; and updating the navigation state variables of the quadruped robot dog through the Kalman filter.

[0185] In this implementation, when the mechanical legs of the quadruped robot dog touch the ground, the speed of the mechanical legs at that moment can be considered to be zero. This zero speed information is used as an observation.

[0186] This zero-speed observation is fed into a Kalman filter. The Kalman filter is an efficient recursive filter that estimates the state of a dynamic system from a series of noisy measurements. It does this in two steps: prediction and update. In the prediction step, the filter uses the state and control inputs from the previous moment to predict the state at the current moment. In the update step, the filter uses the measurements at the current moment to correct the predicted state, resulting in a more accurate estimate.

[0187] The Kalman filter uses zero-speed observations to update the navigation state variables of the quadruped robot dog. These state variables may include position, heading, speed, etc. In this way, the Kalman filter can reduce navigation errors and improve positioning accuracy.

[0188] In this implementation, the accuracy and reliability of the quadruped robot dog in the navigation process can be improved by using the Kalman filter. The Kalman filter is particularly suitable for processing noisy data and can provide real-time state estimation, which is very suitable for navigation applications in dynamic and complex environments.

[0189] Figure 3 The flowchart of the zero-speed correction algorithm in the inertial navigation method of the quadruped robot dog provided by the embodiment of the present disclosure is shown. Figure 3 As shown, the inertial measurement unit installed on the mechanical leg of the quadruped robot dog can be used to collect the inertial data of the mechanical leg. When processing the inertial data, it can be checked whether there is packet loss in the inertial data stream. Subsequently, the raw inertial data can be calibrated.

[0190] According to the results of the turntable calibration, the scale factor matrix and zero bias of the accelerometer, as well as the scale factor matrix and zero bias of the gyroscope, can be determined, so as to compensate the acceleration output by the accelerometer according to the scale factor matrix and zero bias of the accelerometer, and compensate the angular velocity output by the gyroscope according to the scale factor matrix and zero bias of the gyroscope. This processing method of raw inertial data requires advance turntable calibration, which can provide more complete compensation for the accelerometer and gyroscope.

[0191] If the turntable calibration is not possible, you can first stop the quadruped robot dog for 5 seconds before starting it, and determine the accelerometer bias based on the average value of the acceleration output by the accelerometer during the static phase, and determine the gyroscope bias based on the average value of the angular velocity output by the gyroscope during the static phase. This processing method is simpler and more convenient, but the calibration accuracy is limited.

[0192] After the raw inertial data is processed, the initial heading of the inertial measurement unit can be determined by the dual vector method. Specifically, the angle can be determined by multiplying the initial acceleration in the carrier coordinate system by the initial heading in the navigation coordinate system, thereby determining the initial heading of the inertial measurement unit.

[0193] Alternatively, the initial heading of the inertial measurement unit can be determined based on the latitude and longitude information of the location of the inertial measurement unit and the angular velocity of the earth's rotation. This method requires the inertial measurement unit to have high acceleration and angular velocity measurement accuracy.

[0194] During the movement of the quadruped robot dog, the acceleration and angular velocity output by the inertial measurement unit on the mechanical legs of the quadruped robot dog can be judged by threshold. When the modulus of acceleration is less than the acceleration modulus threshold, and the modulus of angular velocity is less than the angular velocity modulus threshold, it can be determined as the foot landing stage. Each time the foot lands, the velocity can be taken as zero, which is used as an observation value input into the Kalman filter to update the navigation state variable.

[0195] The zero-speed correction algorithm is verified on the odometer of a quadruped robot dog. Figure 4 A schematic diagram showing the positioning test results of the zero-speed correction algorithm on the odometer of the quadruped robot dog in the inertial navigation method of the quadruped robot dog provided by the embodiment of the present disclosure. It has been verified experimentally that in a closed-loop trajectory of about 15 meters, the error between the start and end point is 0.2 meters, which is 1.3% of the total distance; in a double-L trajectory of about 23 meters, the error between the start and end point is 0.09 meters, which is 0.3% of the total distance.

[0196] In a possible implementation, the method further includes: performing navigation based on four inertial measurement units arranged on the four mechanical legs of the quadruped robot dog respectively to obtain navigation state variables of the four mechanical legs; and performing kinematic constraints on the navigation state variables of the four mechanical legs.

[0197] In this implementation, four inertial measurement units can be respectively arranged on the four mechanical legs of the quadruped robot dog, and zero-speed correction algorithm navigation can be performed according to the inertial data output by the four inertial measurement units to obtain the navigation state variables of the four mechanical legs (which may include the positions and headings of the four mechanical legs).

[0198] This implementation introduces kinematic constraints into the positioning of the quadruped robot dog, and corrects the positioning errors of the four mechanical legs according to the geometric structure information of the quadruped robot dog. This implementation can correct the positioning error (such as acceleration error and / or angular velocity error) in time to avoid the accumulation of errors over time.

[0199] Figure 5 A schematic diagram showing the arrangement of inertial measurement units on the four legs of a quadruped robot dog in the inertial navigation method of a quadruped robot dog provided by an embodiment of the present disclosure is shown. Figure 5In the example shown, IMU module 1 is fixed to the left front leg of the quadruped robot dog, IMU module 2 is fixed to the right front leg of the quadruped robot dog, IMU module 3 is fixed to the left rear leg of the quadruped robot dog, and IMU module 4 is fixed to the right rear leg of the quadruped robot dog. When the quadruped robot dog is moving, the four IMU modules can perform navigation solutions through the zero-speed correction algorithm respectively, so as to obtain the positions of the four mechanical legs.

[0200] In one possible implementation, the kinematic constraints on the navigation state variables of the four mechanical legs include: dividing the four mechanical legs into two mechanical leg pairs; for any mechanical leg pair, in response to one mechanical leg in the mechanical leg pair landing, determining the distance between the other mechanical leg in the mechanical leg pair and the mechanical leg; in response to the distance being greater than or equal to a distance threshold corresponding to the mechanical leg pair, determining a reference position of the other mechanical leg according to the distance threshold; and correcting the position of the other mechanical leg to the reference position.

[0201] According to the geometric structure information of the quadruped robot dog, there is a theoretical maximum value for the distance between its four mechanical legs. During each step, two mechanical legs are stationary and two mechanical legs are moving forward. With the stationary mechanical legs as a reference, when the distance between the moving mechanical legs and the stationary mechanical legs exceeds the theoretical maximum value (i.e., the distance threshold), the navigation result can be corrected based on the reference position and the geometric structure information of the quadruped robot dog.

[0202] In a possible implementation, the method further includes: in response to two mechanical legs in the pair of mechanical legs landing at the same time, obtaining a distance between the two mechanical legs through a motor encoder to obtain a distance threshold corresponding to the pair of mechanical legs.

[0203] In this implementation, when the four-legged robot dog is moving, when the two mechanical legs land on the ground at the same time, the distance between the two mechanical legs can be obtained as an observation quantity through the motor encoder to correct the positioning error of the mechanical legs.

[0204] This implementation method realizes a method of integrating inertial technology, motor output and carrier kinematic constraints, and uses the navigation results of the four legs of the quadruped robot dog and the output of the motor encoder to correct the sensor error.

[0205] In one possible implementation, the kinematic constraints on the navigation state variables of the four mechanical legs include: in response to a heading deviation between any two of the four mechanical legs being greater than or equal to an angle threshold, performing a heading correction according to the headings of the other two of the four mechanical legs.

[0206] Figure 6A schematic diagram of a positioning error constraint algorithm for a quadruped robot dog combining kinematics and motors in an inertial navigation method for a quadruped robot dog provided in an embodiment of the present disclosure is shown.

[0207] like Figure 6 As shown, in terms of kinematic constraints, IMU module 1 and IMU module 2 can be taken as references. The two will fall to the ground alternately. When IMU module 1 falls to the ground, the moving IMU module 2 will be within a circle with IMU module 1 as the center and radius r. That is:

[0208] Δl=‖x1-x2‖ <r

[0209] When the position of IMU module 2 exceeds the range of the circle, it can be considered that an incorrect navigation position has occurred, and it can be corrected to the circle of the corresponding orientation to obtain the corrected navigation position x2′. The same is true for IMU modules 3 and 4, assuming that the navigation position x3′ is obtained. For x2′ and x3′, there are also kinematic constraints:

[0210] ‖x2′-x3′‖ <l dog

[0211] That is, the distance between the two navigation positions should be smaller than the overall length of the quadruped robot dog, which can be constrained again.

[0212] In terms of motor output constraints, according to the walking rules of the quadruped robot dog, IMU module 1 and IMU module 4 will land at the same time, and IMU module 2 and IMU module 3 will land at the same time. Taking IMU module 1 and IMU module 4 as reference, when both land at the same time, the distance between them can be obtained based on the motor encoder. Using this as an observation, the inertial navigation results of IMU module 1 and IMU module 4 can be corrected. The same is true for IMU module 2 and IMU module 3.

[0213] The heading constraint can also be achieved through the geometric position of the IMU modules on the four mechanical legs. The heading deviation of the IMU modules can be limited according to the angle threshold. For example, when the heading deviation between IMU module 1 and IMU module 2 is greater than the angle threshold, the heading deviation of IMU module 1, IMU module 3 and IMU module 4 can be used to calculate the average heading result. Solve the heading deviation of IMU module 2:

[0214]

[0215] And it can be based on the average results of the heading of IMU module 2, IMU module 3 and IMU module 4 Solve the heading deviation of IMU module 1:

[0216]

[0217] The minimum value of Δθ1 and Δθ2 can be taken to determine the heading correction result at this time to be the corresponding or

[0218] According to the mutual constraint results of the distance and heading of the four IMU modules, the odometer-corrected navigation trajectory output of the quadruped robot dog can be obtained.

[0219] In a possible implementation, the method further includes: identifying the type of ground on which the quadruped robot dog is located according to the inertial data; and determining the acceleration modulus threshold and the angular velocity modulus threshold according to the ground type.

[0220] The ground type can be smooth ground, soft ground, rocky ground, etc., which is not limited here. Based on the collected inertial data, such as specific patterns of acceleration and angular velocity, it can be determined whether the quadruped robot dog is walking on a smooth ground (such as a tile road), a soft ground (such as a grassy field), or a rocky ground (such as a gravel road). The inertial characteristics of the quadruped robot dog moving on different ground types are different, and the ground type can be inferred by analyzing the inertial data.

[0221] Once the ground type is identified, the acceleration modulus threshold and the angular velocity modulus threshold can be set or adjusted. The acceleration modulus threshold and the angular velocity modulus threshold are key parameters used in the zero-speed correction algorithm, which determine when the mechanical legs of the quadruped robot dog are considered to be in contact with the ground, so that the speed should be corrected to zero. For example, on a smooth ground, a lower threshold can be set because the ground provides less resistance, while a higher threshold can be set on a soft ground.

[0222] In this implementation, the zero-speed correction algorithm can be adjusted on different types of ground to achieve accurate positioning of four groups of quadruped robot dogs in multiple scenarios, thereby solving the environmental adaptability problem of quadruped robot dogs.

[0223] In one possible implementation, identifying the type of ground on which the quadruped robot dog is located based on the inertial data includes: inputting the inertial data into a pre-trained long short-term memory network (LSTM), and identifying the type of ground on which the quadruped robot dog is located through the LSTM.

[0224] The LSTM network is a special type of recurrent neural network (RNN) that can capture long-term dependencies in time series data. In this implementation, the LSTM network is trained to recognize different ground types, such as smooth ground, soft ground, or rocky ground.

[0225] As an example of this implementation, the input of the long short-term memory network may be the three-axis angular velocity and three-axis acceleration of a certain robotic leg.

[0226] In this implementation, the long short-term memory network can be used to detect the type of ground the quadruped robot dog touches the ground, so as to adaptively adjust the parameters in the zero-speed correction algorithm.

[0227] As an example of this implementation, six-dimensional data (three-axis angular velocity and three-axis acceleration) can be used as training data for the long short-term memory network. A large amount of inertial data can be collected on gravel roads, tiles, and grass, and a test set and a training set are randomly selected based on the data set. The training set is used to train the long short-term memory network, and the test set is used to verify the recognition accuracy of the long short-term memory network.

[0228] In a possible implementation, before inputting the inertial data into a pre-trained long short-term memory network, the method further includes: performing Gaussian filtering on the inertial data; after identifying the type of ground on which the quadruped robot dog is located through the long short-term memory network, the method further includes: performing mean filtering and continuity judgment on the output result of the long short-term memory network.

[0229] In this implementation, the inertial data can be first Gaussian filtered before being input into the pre-trained LSTM network. Gaussian filtering can reduce the noise in the data while retaining the main features of the signal. In this way, the accuracy of the LSTM network in identifying the ground type can be improved because the input data is cleaner and smoother.

[0230] After the LSTM network recognizes the ground type, the output of the LSTM network can be subjected to mean filtering and continuity judgment. Mean filtering is a simple low-pass filter that smoothes the result by taking the average of multiple output values ​​to reduce random fluctuations. Continuity judgment is to ensure the consistency of the recognition results in the time series and avoid incorrect ground type recognition due to short-term, discontinuous changes.

[0231] In this implementation, the recognition results of the long short-term memory network can be output as the final judgment results through methods such as mean filtering and continuous judgment. The parameters of the zero-speed correction are adjusted based on the judgment results to achieve targeted adjustments for different types of road surfaces.

[0232] Figure 7 Schematic diagram showing different road surfaces. Figure 7In the example shown, the following three types of road surfaces can be identified: gravel road, tile road, and grass. Inertial data of the feet of the quadruped robot dog can be collected on the above three types of road surfaces to train the long short-term memory network.

[0233] Figure 8 The schematic diagram of the structure of the long short-term memory network in the inertial navigation method of the quadruped robot dog provided by the embodiment of the present disclosure is shown. Figure 8 As shown in , the LSTM network includes a reshape layer for the input, a series of fully connected layers, a nonlinear activation function, etc. It can be based on Figure 7 The inertial data of the three types of roads shown are used to train the long short-term memory network. 85% of the data can be randomly selected from the data set as a training set to train the long short-term memory network; the remaining 15% of the data is used as a test set to test the discrimination accuracy of the long short-term memory.

[0234] Fig. 9 The confusion matrix after the long short-term memory network training in the inertial navigation method of the quadruped robot dog provided by the embodiment of the present disclosure is shown. Fig. 9 In the figure, the discrimination results of the long short-term memory network on the test set are plotted in the form of a matrix. The confusion matrix has a high degree of diagonalization and the judgment results on the test set are highly accurate.

[0235] Fig.10 The overall schematic diagram of the ground contact type detection algorithm of the quadruped robot dog in the inertial navigation method of the quadruped robot dog provided by the embodiment of the present disclosure is shown. The algorithm can be based on the long short-term memory network as the core, and combined with the post-processing method to suppress outliers. First, the inertial measurement unit installed on the mechanical leg of the quadruped robot dog outputs acceleration and angular velocity, and the inertial data is windowed, and the window size can be 100. The windowed inertial data can be Gaussian filtered to complete the processing of the original inertial data. A set of data can be taken at intervals of one row to obtain 50×6 inertial data. This data selection method can cover data for a longer period of time while reducing the amount of data processing, and can improve the accuracy of the long short-term memory network in recognizing the ground type. The 50×6 inertial data can be input into the long short-term memory network for recognition, and the output is the discrimination probability of various ground surfaces.

[0236] For the output results of the long short-term memory network, the outputs of the three groups of long short-term memory networks can be taken in time series for mean filtering, and the discrimination probability after filtering can be threshold judged. Since the ground type of the quadruped robot dog should be continuous during its movement, the result of the threshold judgment in time series is continuous in time, and only when the ground changes will the conversion occur. According to this rule, the continuity of the recognition results can be judged and the mutations that appear can be removed.

[0237] According to the ground type recognition results, the parameters of the landing judgment part in the zero-speed correction (acceleration modulus threshold and angular velocity modulus threshold) can be adjusted. For example, on a relatively rough ground such as a gravel road, the zero-speed judgment threshold should be increased; on a relatively flat ground such as a tile road, the zero-speed judgment threshold should be lowered accordingly. Combined with the detection method of the ground type that the quadruped robot dog touches the ground, a zero-speed correction navigation algorithm with adaptive parameter adjustment can be implemented. Since the three types of ground tested cannot completely cover all ground conditions, the threshold can be determined based on the algorithm's judgment probability of the ground, with the angular velocity threshold threshold ω For example:

[0238]

[0239] That is, the ground threshold at this time can be determined based on the probability of judging various ground types and the threshold result calibrated on the ground. The acceleration modulus threshold is similar.

[0240] The inertial navigation method of the quadruped robot dog provided in the embodiment of the present disclosure may relate to the field of robots, and specifically to the dead reckoning and positioning technology of the quadruped robot dog. The inertial navigation method of the quadruped robot dog provided in the embodiment of the present disclosure can provide a high-precision positioning service of the quadruped robot dog odometer by using an inertial measurement unit, and is an autonomous navigation method that does not rely on the external environment. The error of the inertial measurement unit is corrected by a zero-speed correction algorithm, the absolute position of the motor encoder, and kinematic constraints. At the same time, the type of ground on which the quadruped robot dog walks is judged, and the navigation algorithm is adaptively adjusted according to the ground type, which effectively improves the navigation accuracy of the quadruped robot dog in complex terrain.

[0241] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0242] In addition, the present disclosure also provides a quadruped robot dog inertial navigation device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any quadruped robot dog inertial navigation method provided by the present disclosure. The corresponding technical solutions and technical effects can be found in the corresponding records in the method part and will not be repeated here.

[0243] Fig.11 FIG. 2 is a block diagram of an inertial navigation device for a quadruped robot dog provided by an embodiment of the present disclosure. Fig.11 As shown, the quadruped robot dog inertial navigation device includes:

[0244] The acquisition module 21 is used to acquire inertial data of the mechanical legs through an inertial measurement unit installed on the mechanical legs of the quadruped robot dog;

[0245] A first determination module 22, configured to determine a landing moment of the mechanical leg according to the inertial data;

[0246] A correction module 23, used for correcting the integrated speed of the mechanical leg to zero at the landing moment;

[0247] The updating module 24 is used to update the navigation state variables of the quadruped robot dog by taking the zero speed as an observation quantity.

[0248] In a possible implementation, the inertial data includes acceleration and angular velocity;

[0249] The first determining module 22 is used for:

[0250] For any moment, in response to the modulus of acceleration at the moment being less than the acceleration modulus threshold, and the modulus of angular velocity at the moment being less than the angular velocity modulus threshold, the moment is determined to be the landing moment of the mechanical leg.

[0251] In a possible implementation, the updating module 24 is used to:

[0252] Input zero velocity as an observation into the Kalman filter;

[0253] The navigation state variables of the quadruped robot dog are updated through the Kalman filter.

[0254] In a possible implementation manner, the navigation state variable includes at least part of the following: position, heading, and speed.

[0255] In a possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the device further includes:

[0256] A second determination module, used to determine the scale factor matrix and zero bias of the accelerometer, and the scale factor matrix and zero bias of the gyroscope according to the result of the turntable calibration;

[0257] A first acceleration compensation module, used for compensating the acceleration output by the accelerometer according to the scale factor matrix and zero bias of the accelerometer;

[0258] The first angular velocity compensation module is used to compensate the angular velocity output by the gyroscope according to the scale factor matrix and zero bias of the gyroscope.

[0259] In a possible implementation, the inertial measurement unit includes an accelerometer and a gyroscope, and the device further includes:

[0260] A control module, used for controlling the four-legged robot dog to remain stationary for a preset period of time;

[0261] A third determination module is used to determine the zero bias of the accelerometer according to the average value of the acceleration output by the accelerometer in the static stage;

[0262] A fourth determination module, configured to determine the zero bias of the gyroscope according to an average value of the angular velocity output by the gyroscope during a stationary phase;

[0263] A second acceleration compensation module, used for compensating the acceleration output by the accelerometer according to the zero bias of the accelerometer;

[0264] The second angular velocity compensation module is used to compensate the angular velocity output by the gyroscope according to the zero bias of the gyroscope.

[0265] In a possible implementation manner, the device further includes:

[0266] A dot product module, used for performing a dot product operation on the initial acceleration of the inertial measurement unit in the carrier coordinate system and the initial heading of the inertial measurement unit in the navigation coordinate system to obtain a dot product result;

[0267] A fifth determination module, configured to determine an angle between the carrier coordinate system and the navigation coordinate system according to the dot product result;

[0268] A sixth determination module is used to determine an initial heading of the inertial measurement unit using the angle.

[0269] In a possible implementation manner, the device further includes:

[0270] A first obtaining module is used to obtain the longitude and latitude information of the location of the inertial measurement unit;

[0271] A calculation module, used for calculating the angular velocity of the earth's rotation at the position according to the latitude and longitude information;

[0272] The seventh determination module is used to determine the initial heading of the inertial measurement unit relative to the earth's fixed reference system by using the earth's rotation angular velocity and the angular velocity measured by the inertial measurement unit.

[0273] In a possible implementation manner, the device further includes:

[0274] A navigation module, used to perform navigation based on four inertial measurement units arranged on the four mechanical legs of the quadruped robot dog, respectively, to obtain navigation state variables of the four mechanical legs;

[0275] The kinematic constraint module is used to perform kinematic constraints on the navigation state variables of the four mechanical legs.

[0276] In a possible implementation, the kinematic constraint module is used to:

[0277] Dividing the four mechanical legs into two mechanical leg pairs;

[0278] For any pair of mechanical legs, in response to one mechanical leg in the pair of mechanical legs landing, determining a distance between the other mechanical leg in the pair of mechanical legs and the mechanical leg;

[0279] In response to the distance being greater than or equal to a distance threshold corresponding to the pair of mechanical legs, determining a reference position of the other mechanical leg according to the distance threshold;

[0280] The position of the other robot leg is corrected to the reference position.

[0281] In a possible implementation manner, the device further includes:

[0282] The second obtaining module is used for obtaining the distance between the two mechanical legs through the motor encoder in response to the two mechanical legs in the mechanical leg pair landing at the same time, so as to obtain the distance threshold corresponding to the mechanical leg pair.

[0283] In a possible implementation, the kinematic constraint module is used to:

[0284] In response to a heading deviation between any two of the four mechanical legs being greater than or equal to an angle threshold, a heading correction is performed according to the headings of the other two of the four mechanical legs.

[0285] In a possible implementation manner, the device further includes:

[0286] A ground type identification module, used to identify the ground type on which the quadruped robot dog is located according to the inertial data;

[0287] An eighth determination module is used to determine the acceleration modulus threshold and the angular velocity modulus threshold according to the terrain type.

[0288] In a possible implementation, the ground type identification module is used to:

[0289] The inertial data is input into a pre-trained long short-term memory network, and the type of ground on which the quadruped robot dog is located is identified through the long short-term memory network.

[0290] In a possible implementation manner, the device further includes:

[0291] A Gaussian filter module, used for performing Gaussian filter processing on the inertial data;

[0292] The outlier processing module is used to perform mean filtering and continuity judgment on the output results of the long short-term memory network.

[0293] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. Its specific implementation and technical effects can refer to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0294] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

[0295] The embodiment of the present disclosure further provides a computer program, including a computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the above method.

[0296] The embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0297] An embodiment of the present disclosure also provides an electronic device, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0298] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0299] Fig.12 1 is a block diagram of an electronic device 1900 provided in an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a terminal or a server. Fig.12 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions that can be executed by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0300] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (MacOS X TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0301] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0302] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0303] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0304] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0305] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0306] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0307] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0308] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0309] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0310] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0311] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0312] If the technical solution of the embodiments of the present disclosure involves personal information, the product using the technical solution of the embodiments of the present disclosure has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of the embodiments of the present disclosure involves sensitive personal information, the product using the technical solution of the embodiments of the present disclosure has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to collect his or her personal information; or on the device for processing personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0313] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A quadruped robot dog inertial navigation method, characterized in that: include: By installing an inertial measurement unit on a mechanical leg of the quadruped robot dog, inertial data of the mechanical leg is collected; Determining the landing moment of the mechanical leg according to the inertial data; At the landing moment, correcting the integrated speed of the mechanical leg to zero; Taking zero speed as an observation quantity, the navigation state variables of the quadruped robot dog are updated.

2. The method according to claim 1, characterized in that The inertial data includes acceleration and angular velocity; Determining the landing time of the mechanical leg according to the inertial data includes: For any moment, in response to the modulus of acceleration at the moment being less than the acceleration modulus threshold, and the modulus of angular velocity at the moment being less than the angular velocity modulus threshold, the moment is determined to be the landing moment of the mechanical leg.

3. The method according to claim 1, characterized in that The method of taking the zero speed as an observation quantity and updating the navigation state variable of the quadruped robot dog comprises: Input zero velocity as an observation into the Kalman filter; The navigation state variables of the quadruped robot dog are updated through the Kalman filter.

4. The method according to any one of claims 1 to 3, characterized in that The navigation state variables include at least part of the following: position, heading, and speed.

5. The method according to claim 1, characterized in that The inertial measurement unit includes an accelerometer and a gyroscope, and the method further includes: Determining the scale factor matrix and zero bias of the accelerometer and the scale factor matrix and zero bias of the gyroscope according to the turntable calibration result; Compensating the acceleration output by the accelerometer according to the scale factor matrix and zero bias of the accelerometer; The angular velocity output by the gyroscope is compensated according to the scale factor matrix and the zero bias of the gyroscope.

6. The method according to claim 1, characterized in that The inertial measurement unit includes an accelerometer and a gyroscope, and the method further includes: Controlling the four-legged robot dog to remain stationary for a preset period of time; Determining the zero bias of the accelerometer according to the average value of the acceleration output by the accelerometer during the static phase; Determining the zero bias of the gyroscope according to the average value of the angular velocity output by the gyroscope during the static phase; Compensating the acceleration output by the accelerometer according to the zero bias of the accelerometer; The angular velocity output by the gyroscope is compensated according to the zero bias of the gyroscope.

7. The method according to claim 1, characterized in that The method further comprises: Performing a dot product operation on the initial acceleration of the inertial measurement unit in the carrier coordinate system and the initial heading of the inertial measurement unit in the navigation coordinate system to obtain a dot product result; Determine the angle between the carrier coordinate system and the navigation coordinate system according to the dot product result; The initial heading of the inertial measurement unit is determined using the angle.

8. The method according to claim 1, characterized in that The method further comprises: Obtaining the latitude and longitude information of the location of the inertial measurement unit; Calculate the angular velocity of the earth's rotation at the location according to the latitude and longitude information; The initial heading of the inertial measurement unit relative to the earth fixed reference system is determined by using the earth rotation angular velocity and the angular velocity measured by the inertial measurement unit.

9. The method according to claim 1, characterized in that: The method further comprises: Based on four inertial measurement units arranged on the four mechanical legs of the quadruped robot dog, navigation is performed respectively to obtain navigation state variables of the four mechanical legs; Kinematic constraints are imposed on the navigation state variables of the four robotic legs.

10. The method according to claim 9, characterized in that The kinematic constraints on the navigation state variables of the four mechanical legs include: Dividing the four mechanical legs into two mechanical leg pairs; For any pair of mechanical legs, in response to one mechanical leg in the pair of mechanical legs landing, determining a distance between the other mechanical leg in the pair of mechanical legs and the mechanical leg; In response to the distance being greater than or equal to a distance threshold corresponding to the pair of mechanical legs, determining a reference position of the other mechanical leg according to the distance threshold; The position of the other robot leg is corrected to the reference position.

11. The method according to claim 10, characterized in that The method further comprises: In response to two mechanical legs in the pair of mechanical legs landing at the same time, the distance between the two mechanical legs is obtained through a motor encoder to obtain a distance threshold corresponding to the pair of mechanical legs.

12. The method according to claim 9, characterized in that The kinematic constraints on the navigation state variables of the four mechanical legs include: In response to a heading deviation between any two of the four mechanical legs being greater than or equal to an angle threshold, a heading correction is performed according to the headings of the other two of the four mechanical legs.

13. The method according to claim 2, characterized in that The method further comprises: identifying the type of ground on which the quadruped robot dog is located based on the inertial data; The acceleration modulus threshold and the angular velocity modulus threshold are determined according to the terrain type.

14. The method according to claim 13, characterized in that The step of identifying the type of ground on which the quadruped robot dog is located according to the inertial data includes: The inertial data is input into a pre-trained long short-term memory network, and the type of ground on which the quadruped robot dog is located is identified through the long short-term memory network.

15. The method according to claim 14, characterized in that Before inputting the inertial data into the pre-trained long short-term memory network, the method further comprises: performing Gaussian filtering on the inertial data; After identifying the type of ground on which the quadruped robot dog is located by using the long short-term memory network, the method further includes: performing mean filtering and continuity judgment on the output result of the long short-term memory network.

16. A quadruped robot dog inertial navigation device, characterized in that: include: A collection module, used for collecting inertial data of the mechanical legs through an inertial measurement unit installed on the mechanical legs of the quadruped robot dog; A first determination module, used to determine the landing moment of the mechanical leg according to the inertial data; A correction module, used for correcting the integrated speed of the mechanical leg to zero at the landing moment; The updating module is used to update the navigation state variables of the quadruped robot dog by taking the zero speed as an observation quantity.

17. An electronic device, characterized in that: include: one or more processors; a memory for storing executable instructions; The one or more processors are configured to call the executable instructions stored in the memory to execute the method according to any one of claims 1 to 15.

18. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 15 is implemented.

19. A computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, characterized in that: When the computer readable code is executed in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 15.

Images