Robot positioning method, device and equipment and medium

Abstract

The invention discloses a robot positioning method, device and equipment and a medium, and the method comprises the steps: obtaining a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, the inertial measurement unit and the wheel speed odometer being carried on a robot; whether the robot slips or not is judged on the basis of the difference value between the first yaw angular velocity and the second yaw angular velocity; if the robot does not slip, the position of the robot is determined based on the position of a wheel speed odometer and the position of a navigation satellite system, the position of the wheel speed odometer is the position determined based on the wheel speed odometer, and the position of the navigation satellite system is the position determined based on a navigation satellite system sensor carried by the robot; and if the robot slips, the position of the robot is determined based on data collected by a visual sensor carried by the robot. Therefore, the positioning accuracy of the robot can be improved, so that the operation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of robot positioning technology, and in particular to a robot positioning method, device, equipment and medium. Background Art

[0002] With the development of robot technology, robots are used in more and more scenarios. During the operation of the robot, the robot needs to be positioned in order to control the robot to complete the operation accurately.

[0003] At present, robot positioning technology usually relies on a single positioning method, such as navigation satellite systems, wheel speed odometers or IMU (Inertial Measurement Unit). However, in complex scenarios, inaccurate positioning often occurs, affecting operations. Therefore, how to improve the accuracy of robot positioning and thus improve operating efficiency is a problem that needs to be solved at present. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a robot positioning method, device, equipment and medium, which can improve the accuracy of robot positioning, thereby improving work efficiency. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a robot positioning method, comprising:

[0006] Acquire a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, wherein the inertial measurement unit and the wheel speed odometer are both mounted on the robot;

[0007] determining whether the robot is slipping based on a difference between the first yaw angular velocity and the second yaw angular velocity;

[0008] If the robot does not slip, determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position, wherein the wheel speed odometer position is a position determined based on the wheel speed odometer, and the navigation satellite system position is a position determined based on a navigation satellite system sensor carried by the robot;

[0009] If the robot slips, the position of the robot is determined based on data collected by a visual sensor carried by the robot.

[0010] Optionally, judging whether the robot slips based on a difference between the first yaw angular velocity and the second yaw angular velocity includes:

[0011] calculating a slip rate based on a difference between the first yaw angular velocity and the second yaw angular velocity;

[0012] The slip rate is compared with a preset threshold value. If the slip rate is lower than the preset threshold value, it is determined that the robot has not slipped. If the slip rate is greater than or equal to the preset threshold value, it is determined that the robot has slipped.

[0013] Optionally, before determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position, the method further includes:

[0014] Calculating the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer, wherein the world coordinate system is a coordinate system established based on the posture of the robot at the initial moment;

[0015] Determine a rotation matrix for converting a corresponding coordinate system of the navigation satellite system to a world coordinate system based on the angle;

[0016] Based on the rotation matrix, the coordinates in the corresponding coordinate system of the navigation satellite system determined based on the navigation satellite system sensor are converted into coordinates in the world coordinate system to obtain the position of the navigation satellite system.

[0017] Optionally, the calculating the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer includes:

[0018] N pairs of data are selected, each pair of data includes two wheel speed odometer position data and two navigation satellite system position data, wherein the wheel speed odometer position data is position data in a world coordinate system determined based on the wheel speed odometer, and the navigation satellite system position data is position data in a corresponding coordinate system of the navigation satellite system determined based on a navigation satellite system sensor;

[0019] Calculate the angle between the first vector determined by the two wheel speed odometer position data and the second vector determined by the two navigation satellite system position data in each pair of data;

[0020] The N angles corresponding to the N pairs of data are averaged to obtain the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer.

[0021] Optionally, determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position includes:

[0022] Determine the error distance between the wheel speed odometer position and the navigation satellite system position;

[0023] Determine a noise covariance matrix corresponding to the wheel speed odometer based on the error distance;

[0024] Based on the noise covariance matrix corresponding to the wheel speed odometer and using the position of the navigation satellite system, the position of the wheel speed odometer is corrected to obtain the position of the robot.

[0025] Optionally, the step of correcting the position of the wheel speed odometer based on a noise covariance matrix corresponding to the wheel speed odometer and utilizing the position of the navigation satellite system to obtain the position of the robot includes:

[0026] Determining a predicted state covariance matrix based on a noise covariance matrix corresponding to the wheel speed odometer;

[0027] Determine the Kalman gain based on the predicted state covariance matrix and the noise covariance matrix corresponding to the navigation satellite system;

[0028] Based on the Kalman gain, the wheel speed odometer position is corrected by using the navigation satellite system position to obtain the position of the robot.

[0029] Optionally, the visual sensor is a monocular camera, and before determining the position of the robot based on data collected by the visual sensor carried by the robot, the method further includes:

[0030] Based on the position of the robot corresponding to two adjacent frames of images in the reference coordinate system, the time interval between the two adjacent frames of images, the displacement increment obtained by pre-integration of the inertial measurement unit, the speed increment obtained by pre-integration of the inertial measurement unit, and the speed output by the wheel speed odometer corresponding to the two adjacent frames of images, and taking the robot speed, gravity vector and scale factor in the reference coordinate system as the parameters to be solved, a residual term between two adjacent frames of images is constructed as an objective function, wherein the reference coordinate system is a coordinate system established based on the posture of the camera;

[0031] Determine the robot speed, gravity vector and scale factor in the reference coordinate system based on the objective function;

[0032] Calculate a transformation matrix from the reference coordinate system to the world coordinate system based on the weight vector in the reference coordinate system and the gravity vector in the world coordinate system;

[0033] Restoring the physical scale of the robot's position based on the scale factor;

[0034] Based on the transformation matrix, the state variables in the reference coordinate system are converted to the world coordinate system to obtain the initial state variables of the robot, wherein the state variables include the position, posture and speed of the robot;

[0035] Accordingly, determining the position of the robot based on data collected by a visual sensor carried by the robot includes:

[0036] Based on the initial state variables and data collected by a monocular camera carried by the robot, the position and posture of the robot are output through a monocular visual odometer.

[0037] In a second aspect, the present application discloses a robot positioning device, comprising:

[0038] A yaw angular velocity acquisition module, which acquires a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, wherein the inertial measurement unit and the wheel speed odometer are both mounted on the robot;

[0039] a slip detection module, for determining whether the robot slips based on a difference between the first yaw angular velocity and the second yaw angular velocity;

[0040] a first positioning module, which determines the position of the robot based on a wheel speed odometer position and a navigation satellite system position if the robot does not slip, wherein the wheel speed odometer position is a position determined based on the wheel speed odometer, and the navigation satellite system position is a position determined based on a navigation satellite system sensor carried by the robot;

[0041] The second positioning module determines the position of the robot based on data collected by a visual sensor carried by the robot if the robot slips.

[0042] In a third aspect, a self-contained device includes a memory and a processor, wherein:

[0043] The memory is used to store the computer program;

[0044] The processor is used to execute the computer program to implement the aforementioned robot positioning method.

[0045] In a fourth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program implements the aforementioned robot positioning method when executed by a processor.

[0046] It can be seen from the above scheme that the present application provides a robot positioning method, including: obtaining a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, wherein the inertial measurement unit and the wheel speed odometer are both mounted on the robot; judging whether the robot slips based on the difference between the first yaw angular velocity and the second yaw angular velocity; if the robot does not slip, determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position, wherein the wheel speed odometer position is a position determined based on the wheel speed odometer, and the navigation satellite system position is a position determined based on the navigation satellite system sensor carried by the robot; if the robot slips, determining the position of the robot based on data collected by the visual sensor carried by the robot.

[0047] It can be seen that the beneficial effects of the present application are as follows: in the process of robot positioning, the data of the wheel speed odometer, inertial measurement unit, navigation satellite system sensor and visual sensor are integrated, and the difference between the yaw angular velocity measured by the inertial measurement unit and the yaw angular velocity measured by the wheel speed odometer is used to determine whether the robot is slipping. When the robot does not slip, positioning is performed based on the position of the wheel speed odometer and the position of the navigation satellite system, which can cope with environments with weak textures or sparse feature points. When the robot slips, positioning is performed based on the data collected by the visual sensor, which can avoid the problem of the wheel speed odometer being inaccurate in complex terrain, and can also cope with scenes with complex environments and weak navigation satellite system signals. In this way, by integrating data from multiple sensors and switching in time based on the slip judgment results, the accuracy of the robot's positioning can be improved, thereby improving operating efficiency.

[0048] Correspondingly, a robot positioning device, equipment and readable storage medium provided by the present application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0050] Figure 1 A flow chart of a robot positioning method provided in an embodiment of the present application;

[0051] Figure 2 A positioning flow chart of a lawn mowing robot provided in an embodiment of the present application;

[0052] Figure 3A schematic diagram of the structure of a robot positioning device provided in an embodiment of the present application;

[0053] Figure 4 A structural diagram of a self-moving device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] At present, the autonomous navigation technology of robots, such as outdoor lawn mowing robots, mostly relies on a single positioning method, such as the global positioning system (GPS), wheel speed odometer or inertial measurement unit. However, these methods have obvious limitations in practical applications. Although GPS can provide absolute position information, its positioning accuracy and reliability are significantly reduced in satellite signal blocking areas such as urban canyons and dense forests. The wheel speed odometer estimates the position by recording the rotation of the wheels, but it is easy to produce cumulative errors after long-term operation, especially when driving on complex terrain or slippery ground. The IMU can provide high-precision angular velocity and acceleration information, but there is also an integral drift problem. After long-term operation, the position estimate will deviate significantly from the actual. In addition, although visual positioning technology, especially the simultaneous localization and mapping (SLAM) technology based on monocular vision, has made significant progress in recent years, its positioning accuracy and robustness are still insufficient in scenes with weak textures or sparse feature points. To this end, the present application provides a robot positioning solution, which realizes high-precision, high-stability positioning and autonomous navigation of the robot in complex environments by integrating the data of wheel speed odometer, IMU, navigation satellite system and visual sensor.

[0056] See also Figure 1 As shown, the embodiment of the present application discloses a robot positioning method, including:

[0057] Step S11: obtaining a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, wherein the inertial measurement unit and the wheel speed odometer are both mounted on the robot.

[0058] Among them, the inertial measurement unit is a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object, which may include three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration signal of the object in the independent three axes of the carrier coordinate system, and the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system, measures the angular velocity and acceleration of the object in three-dimensional space, and calculates the attitude of the object. The wheel speed odometer is a device for obtaining the vehicle speed and displacement from the sensors installed on the wheels. The robot can be a robot that performs various operations, such as a lawn mowing robot that performs lawn mowing operations.

[0059] The embodiment of the present application can obtain the yaw angular velocity measured by the inertial measurement unit and the yaw angular velocity measured by the wheel speed odometer. Among them, the second yaw angular velocity can be calculated using the yaw angular velocity calculation formula. The yaw angular velocity calculation formula is:

[0060]

[0061] Among them, ω enc is the second yaw angular velocity, v L and v R They represent the linear speed of the left and right wheels of the robot respectively, d is the wheelbase, and the linear speed is obtained by the wheel speed odometer.

[0062] Step S12: determining whether the robot is slipping based on the difference between the first yaw angular velocity and the second yaw angular velocity.

[0063] In an optional implementation, the slip rate may be calculated based on the difference between the first yaw angular velocity and the second yaw angular velocity; the slip rate is compared with a preset threshold, and if the slip rate is lower than the preset threshold, it is determined that the robot has not slipped; if the slip rate is greater than or equal to the preset threshold, it is determined that the robot has slipped. The calculation formula for the slip rate may be:

[0064]

[0065] Among them, γ is the slip rate, ω imu and ω enc are the yaw rates measured by the IMU and wheel speed odometer respectively.

[0066] Faced with complex terrain, the robot may slip during operation. For example, a lawn mowing robot is very likely to slip on outdoor grass. The embodiment of the present application provides a method for detecting slippage, which calculates the slip rate based on the difference between the yaw angular velocity measured by the IMU and the wheel speed odometer. The angular velocity measurement of the IMU is more accurate, especially in a short period of time. The embodiment of the present application can set a slip rate threshold γ threshold,This threshold can be determined through experiments and experience to ensure that it will not be misjudged as skidding in normal driving and minor disturbances, while being able to accurately identify real skidding.

[0067] Step S13: If the robot does not slip, the position of the robot is determined based on the wheel speed odometer position and the navigation satellite system position, wherein the wheel speed odometer position is a position determined based on the wheel speed odometer, and the navigation satellite system position is a position determined based on a navigation satellite system sensor carried by the robot.

[0068] Among them, the navigation satellite system is the Global Navigation Satellite System (GNSS), which is an air-based radio navigation and positioning system that can provide users with all-weather 3D coordinates, speed and time information at any location on the earth's surface or in near-earth space. For example, it can be the Global Positioning System (GPS), the Beidou Satellite Navigation System (BDS), the GLONASS Satellite Navigation System (GLONASS) and the Galileo Satellite Navigation System (GALILEO).

[0069] In this embodiment, a world coordinate system can be established based on the posture of the robot in the first frame, the wheel speed odometer determines the robot speed in the robot's own coordinate system, the robot posture (i.e., direction and tilt) is obtained based on the angular velocity integration of the inertial measurement unit, and the robot speed in the robot's own coordinate system is converted to the world coordinate system based on the robot posture. The robot speed in the world coordinate system is integrated to obtain the displacement increment within the time interval, and then the calculated displacement increment is added to the position at the previous moment to obtain the position of the robot in the world coordinate system at the current moment, that is, the wheel speed odometer position, thereby realizing odometer ground push.

[0070] Furthermore, the odometer in this embodiment is recursive. For a two-wheel differential robot, the speeds of its left and right wheels are v and L and v d , the wheelbase is d. The robot's centerline speed v b It can be calculated by the following formula:

[0071]

[0072] Among them, v b It is the forward speed of the robot in the body coordinate system (i.e. the robot's own coordinate system). The body coordinate system changes with the movement of the body. It is established with the center points of the two wheels of the body as the center and describes the local movement of the body. The world coordinate system is established with the body posture of the first frame. The world coordinate system is fixed and is established with the position and posture of the body at the initial moment as the origin, serving as a global reference. The body posture at subsequent moments It is obtained by integrating the angular velocity of the IMU, and then converting the velocity in the vehicle coordinate system of the current frame into the velocity v in the world coordinate system w , the conversion formula is: After obtaining the velocity vector in the world coordinate system, the displacement increment within a short time interval Δt can be calculated by velocity integration, and then the calculated displacement increment is added to the position at the previous moment to obtain the estimated position of the robot in the world coordinate system at the current moment:

[0073]

[0074] Among them, (x cur ,y cur ) represents the updated robot position at the current moment, (x pre ,y pre ) represents the robot position at the last moment, Indicates the speed of the robot in the horizontal and vertical directions in the world coordinate system. The horizontal axis is the x-axis, and the vertical axis is the y-axis, which can be determined according to the actual scenario. For example, forward (i.e. the robot's forward direction) is the y-axis, and rightward (determined based on the robot's forward direction) is the x-axis.

[0075] In this embodiment, before determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position, the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer can also be calculated, wherein the world coordinate system is a coordinate system established based on the robot's posture at the initial moment; based on the angle, the rotation matrix of the navigation satellite system corresponding coordinate system converted to the world coordinate system is determined; based on the rotation matrix, the coordinates in the navigation satellite system corresponding coordinate system determined based on the navigation satellite system sensor are converted to the coordinates in the world coordinate system to obtain the navigation satellite system position. That is, in order to perform multi-sensor fusion positioning, the embodiment of the present application calibrates the navigation satellite system position.

[0076] Further, calculating the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer may include: selecting N pairs of data, each pair of data including two wheel speed odometer position data and two navigation satellite system position data, wherein the wheel speed odometer position data is position data in the world coordinate system determined based on the wheel speed odometer, and the navigation satellite system position data is position data in the coordinate system corresponding to the navigation satellite system determined based on the navigation satellite system sensor; calculating the angle between the first vector determined by the two wheel speed odometer position data and the second vector determined by the two navigation satellite system position data in each pair of data; performing mean processing on the N angles corresponding to the N pairs of data to obtain the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer. N is a positive integer, including 1 and a positive integer greater than or equal to 2.

[0077] In this embodiment, the wheel speed odometer position data and the navigation satellite system position data can be recorded after the robot is started. In addition, the wheel speed odometer position data and the navigation satellite system position data are recorded when the displacement is greater than a preset displacement threshold. The displacement is determined by the wheel speed odometer. The navigation satellite system position data is relatively more reliable when the robot has a certain displacement, so that the recorded data can improve the calibration accuracy. This embodiment selects the aforementioned N pairs of data from the recorded data.

[0078] In an optional embodiment, the coordinate system corresponding to the navigation satellite system can be a northeast celestial coordinate system, which is a rectangular coordinate system with the station center (such as the center of the GPS receiving antenna) as the coordinate origin O, the Z axis coincides with the ellipsoid normal, upward is positive (celestial direction), the Y axis coincides with the minor semi-axis of the ellipsoid (north), and the X axis coincides with the major semi-axis of the earth ellipsoid (east).

[0079] This embodiment can calculate the angle between the northeast celestial coordinate system and the world coordinate system corresponding to the wheel speed odometer, wherein the world coordinate system is a coordinate system established based on the posture of the robot at the initial moment; determine the rotation matrix for converting the northeast celestial coordinate system to the world coordinate system based on the angle; based on the rotation matrix, convert the coordinates in the northeast celestial coordinate system determined based on the navigation satellite system sensor into coordinates in the world coordinate system to obtain the navigation satellite system position. Further, in an optional embodiment, the calculation of the angle between the northeast celestial coordinate system and the world coordinate system corresponding to the wheel speed odometer may include: selecting N pairs of data, each pair of data including two wheel speed odometer position data and two navigation satellite system position data, wherein the wheel speed odometer position data is position data in the world coordinate system determined based on the wheel speed odometer, and the navigation satellite system position data is position data in the northeast celestial coordinate system determined based on the navigation satellite system sensor; calculating the angle between the first vector determined by the two wheel speed odometer position data and the second vector determined by the two navigation satellite system position data in each pair of data; and performing mean processing on the N angles corresponding to the N pairs of data to obtain the angle between the northeast celestial coordinate system and the world coordinate system corresponding to the wheel speed odometer.

[0080] The following uses GPS as an example to further describe the alignment process of the navigation satellite system. Other types of navigation satellite systems can refer to GPS and will not be described in detail here. In order to ensure the accurate positioning of the robot (such as a lawn mowing robot) in an outdoor environment, it is necessary to align the northeast celestial coordinate system with the world coordinate system established by the wheel speed odometer. The initial alignment of GPS may include the following steps

[0081] Get GPS origin data: In a static state, continuously obtain GPS latitude, longitude and altitude data within 5 seconds (L i,λ i ,H i ), (L i ,λ i ,H i ) represent longitude, latitude, and elevation data respectively, where i = 1, 2, ..., n, and n is the number of data points within 5 seconds. The data within these 5 seconds are averaged to reduce noise and errors. The calculated mean (L0, λ0, H0) is used as the origin of GPS. Subsequent GPS data will convert the longitude and latitude coordinates into data in the northeast sky coordinate system based on this origin.

[0082] Data recording and filtering: Start the mowing robot and simultaneously record the GPS coordinates in the northeast sky coordinate system and the coordinate data of the wheel speed odometer Where i = 1, 2, ..., n. Instead of recording all data, a displacement threshold d is set. th .

[0083]

[0084] As shown in the above formula, only when the odometer movement displacement is greater than this threshold, the corresponding GPS and odometer data are recorded, because GPS data is relatively more reliable when the robot has a certain displacement.

[0085] Data pairing and alignment angle calculation: N pairs of data are selected from the recorded data queue, each pair of data contains two GPS coordinates and two odometer coordinates. For each pair of data, the angle between the vector formed by the two GPS coordinates and the vector formed by the two odometer coordinates is calculated.

[0086]

[0087] in, and Represents two GPS coordinates in a pair of data, and Represents two odometer coordinates in a pair of data, where θ enu is the angle of the vector formed by the two GPS coordinates, θ w The angle of the vector formed by two odometry coordinates. arctan2 represents the two-parameter inverse tangent function, which is used to find the angle of the vector.

[0088] Angle filtering: average the calculated angles of all selected data.

[0089]

[0090] in, is the angle calculated for the jth pair of data. The average is the initial alignment angle between the northeast celestial coordinate system and the world coordinate system of the odometer. Use the calculated alignment angle to convert the subsequent northeast celestial coordinates output by the GPS into the coordinates in the world coordinate system of the odometer. Convert to rotation matrix Then convert the GPS coordinate output to the world coordinate system:

[0091]

[0092] in, Indicates the coordinate value of GPS in the world coordinate system. Indicates the coordinate value of GPS in the northeast celestial coordinate system.

[0093] In an optional embodiment, determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position may include: determining an error distance between the wheel speed odometer position and the navigation satellite system position; determining a noise covariance matrix corresponding to the wheel speed odometer based on the error distance; and correcting the wheel speed odometer position based on the noise covariance matrix corresponding to the wheel speed odometer and using the navigation satellite system position to obtain the position of the robot.

[0094] Further, in an optional embodiment, the method of correcting the position of the wheel speed odometer based on the noise covariance matrix corresponding to the wheel speed odometer and using the position of the navigation satellite system to obtain the position of the robot may include: determining a predicted state covariance matrix based on the noise covariance matrix corresponding to the wheel speed odometer; determining the Kalman gain based on the predicted state covariance matrix and the noise covariance matrix corresponding to the navigation satellite system; and correcting the position of the wheel speed odometer based on the Kalman gain and using the position of the navigation satellite system to obtain the position of the robot.

[0095] That is, the embodiment of the present application can correct the wheel speed odometer position based on the navigation satellite system position correction to obtain the position of the robot, thereby improving the accuracy of positioning.

[0096] The following uses GPS as an example to further describe the correction process of the wheel speed odometer position by the navigation satellite system. Other types of navigation satellite systems can refer to GPS and will not be described in detail here. In this embodiment, the GPS observation can be used to correct the error of the odometer, and the recursive position of the odometer can be used as the predicted state quantity:

[0097]

[0098] Among them, x pred is the predicted state, Indicates the position of the vehicle (i.e., the robot) in the world coordinate system, i.e., the position of the wheel speed odometer. Covariance prediction:

[0099] P pred =FP est F T +Q odom ;

[0100] Among them, P pred is the predicted state covariance matrix, P est is the current state covariance matrix, which is initially set empirically, F is the state transfer matrix (2×2 identity matrix), Q odom is the noise covariance matrix of the wheel speed odometer. The noise of the wheel speed odometer is calculated based on the error distance d between the wheel speed odometer and GPS. error And set, when the error distance d error The smaller it is, the smaller the noise covariance matrix of the wheel speed odometer is (that is, the smaller the diagonal element y is), and vice versa;

[0101]

[0102] in, and Respectively represent the positions of GPS and vehicle body in the world coordinate system, that is, the GPS position of the robot and the wheel speed odometer position, both of which are positions in the world coordinate system. The embodiment of the present application designs a function to set the noise covariance matrix of the wheel speed odometer:

[0103]

[0104] Among them, (a, b, c, d) are the parameters of the function, which are used to set the turning point and slope of the function. error As x value into the above formula, we can calculate the noise covariance matrix of the wheel speed odometer. Assuming that the covariance of the x-axis and y-axis is equal, the covariance matrix Q of the wheel speed odometer is odom as follows:

[0105]

[0106] Among them, y is the output value of the function designed above. Use the Jacobian matrix and the predicted covariance matrix to calculate the Kalman gain:

[0107] K=P pred H T (HP pred H T +R gps ) -1 ;

[0108] Among them, R gps is the noise covariance matrix of the GPS measurements, which has a fixed size, and H is the observation matrix, which is a 2 × 2 identity matrix. The Kalman gain and the measurement residual are used to update the state estimate.

[0109]

[0110] in, Indicates the position of GPS in the world coordinate system (i.e. ), that is, the observed value, x est is the corrected robot position.

[0111] Furthermore, the state covariance matrix is ​​updated to reflect the uncertainty of the new state estimate.

[0112] P est =(I-KH)P pred ;

[0113] Where I is the identity matrix.

[0114] Step S14: If the robot slips, the position of the robot is determined based on data collected by a visual sensor carried by the robot.

[0115] Among them, the visual sensor includes but is not limited to a monocular camera, a binocular camera, etc. Based on the data collected by the visual sensor, that is, the image data, the position of the robot is determined using a visual odometer. Visual odometer is a technology that estimates the travel distance by analyzing continuous camera images to determine equivalent odometer information. Relying on the image data collected by the camera and combining it with a computer vision algorithm, the displacement and posture changes of the robot equipped with a camera in the environment are inferred. In this embodiment, a visual inertial odometer (VIO) can be used in combination with visual information and inertial information of an inertial measurement unit to provide a positioning result with higher accuracy and robustness.

[0116] In an optional implementation, the visual sensor is a monocular camera, and before determining the position of the robot based on data collected by the visual sensor carried by the robot, the method further includes:

[0117] Based on the position of the robot in the reference coordinate system corresponding to two adjacent frames of images, the time interval between the two adjacent frames of images, the displacement increment obtained by the pre-integration of the inertial measurement unit, the speed increment obtained by the pre-integration of the inertial measurement unit, the speed output of the wheel speed odometer corresponding to the two adjacent frames of images, and the robot speed, gravity vector and scale factor in the reference coordinate system as the parameters to be solved, the residual term between the two adjacent frames of images is constructed as the objective function, wherein the reference coordinate system is a coordinate system established based on the posture of the camera; based on the objective function, the robot speed, gravity vector and scale factor in the reference coordinate system are determined; based on the weight vector in the reference coordinate system and the gravity vector in the world coordinate system, the transformation matrix from the reference coordinate system to the world coordinate system is calculated; based on the scale factor, the physical scale of the robot's position is restored; based on the transformation matrix, the state variables in the reference coordinate system are converted to the world coordinate system to obtain the initial state variables of the robot, wherein the state variables include the position, posture and speed of the robot. It can be understood that the scale factor is a proportional parameter that restores the data determined based on the monocular camera to the real physical scale.

[0118] Correspondingly, determining the position of the robot based on data collected by the visual sensor carried by the robot includes: outputting the position and posture of the robot through a monocular visual odometer based on the initial state variables and the data collected by the monocular camera carried by the robot.

[0119] When serious skidding occurs, the wheel speed odometer output is not very accurate. This embodiment can switch to the monocular visual odometer, but since the monocular has no scale information, it is impossible to obtain the real physical scale. Therefore, initialization operation is required first. This embodiment uses a joint initialization strategy of monocular-IMU-wheel speed meter (i.e., wheel speed odometer) to restore the gravity vector, vehicle speed and scale information. Different from the traditional initialization method, the residual term corresponding to the wheel speed meter in this embodiment is used as the objective function to improve the accuracy of the estimated value. The initialization steps are:

[0120] The first frame image c0 coordinate system is selected as the reference coordinate system. The first frame image coordinate system refers to the coordinate system established when the camera takes the first image, with the camera optical center as the origin and the camera posture as the coordinate axis orientation. The SfM (Structure from Motion) method is used to calculate the scale-free camera pose at all image moments (i.e. the time when the image was taken). and landmark position λ h .in, is the attitude, expressed as a quaternion, is the position, expressed in x-axis and y-axis coordinates, according to the rotation of the camera relative to the IMU Calculate the posture of the vehicle (i.e. the robot) in the reference coordinate system c0 Rotation refers to the rotation transformation between coordinate systems, which can be represented by quaternions. Scale refers to the conversion ratio between the real physical distance and the distance calculated by the algorithm.

[0121] Solve the IMU angular velocity bias b by minimizing the IMU pre-integration error ω The initial value of The specific formula is:

[0122]

[0123] Among them, K init is the set of image frame indices considered during initialization, is the relative rotation between the two frames obtained by IMU pre-integration, The angular velocity bias term can be included. Minimizing the above formula can obtain the initial value of the angular velocity bias. 2 represents the square of the Euclidean distance, Indicates the multiplication between quaternions. The argmin function indicates the parameter value (the value of the independent variable) that a function takes to achieve the minimum value in its domain. represents the inverse of the posture of the vehicle (i.e., the robot) in the reference coordinate system c0 at the next moment, Represents the posture of the vehicle (i.e., the robot) in the reference coordinate system c0.

[0124] The residual term is used as the objective function to improve the accuracy of the estimated value. Construct the residual term r between the kth and k+1th frame images init,k-k+1 , minimize the objective function and make it approach 0, and solve the vehicle speed Gravity vector and the scale factor s, that is, the value when the objective function is minimized. Even if the residual term is minimized, the vehicle velocity, gravity vector, and scale factor can be calculated using the gradient descent method.

[0125]

[0126] Among them, c0 represents the camera coordinate system of the first frame image (i.e., the reference coordinate system), b k represents the vehicle coordinate system at the kth frame, b k+1 represents the vehicle coordinate system at the k+1th moment, and R represents the relative rotation between the two coordinate systems. represents the relative rotation between the vehicle coordinate system at the kth frame and the camera coordinate system at the first frame. Indicates the relative rotation between the camera coordinate system of the first frame image and the vehicle coordinate system of the kth frame image. Represents the relative rotation between the camera coordinate system of the first frame image and the vehicle coordinate system at the k+1th frame image. Both the camera coordinate system and the vehicle coordinate system are local coordinate systems, which change as the robot moves. In the embodiment of the present application, the camera coordinate system of the first frame image is used as the reference coordinate system.

[0127] in, and They represent the positions of the robot in the reference coordinate system at the kth frame and the k+1th frame respectively. These two values ​​can be obtained by the following formulas; Δt k is the time interval between the kth frame and the k+1th frame image; is the displacement increment obtained by IMU pre-integration, is the velocity increment obtained by IMU pre-integration, and It is the vehicle speed corresponding to the k-th frame and the k+1-th frame image output by the wheel speed meter. Indicates the vehicle body velocity corresponding to the kth frame and the k+1th frame image to be obtained. The displacement increment and velocity increment can be calculated based on the angular velocity bias.

[0128]

[0129] in, represents the scale-free displacement between the origin of the camera coordinate system ck of the kth frame and the origin of the camera coordinate system c0 of the first frame, Indicates the relative rotation between the camera coordinate system of the first frame image and the camera coordinate system of the kth frame image. Represents the displacement between the origin of the body coordinate system established on the robot and the origin of the camera coordinate system.

[0130] Define the origin of the world coordinate system to coincide with the origin of the c0 reference coordinate system, and the gravity vector g w =[0,0,G] T , G is the value of gravity acceleration, which is 9.8m / s2. According to the gravity vector in the c0 reference coordinate system and the gravity vector g in the world coordinate system w , calculate the rotation from the c0 reference coordinate system to the world coordinate system Finally, use s to restore the scale of the vehicle body and the landmark position, and then use Convert the state variables from the c0 reference coordinate system to the world coordinate system to obtain the vehicle posture at the kth frame image moment The speed of the vehicle in the world coordinate system and landmark position λ h The initial value is the same as the original value. The landmark point is a 3D point in the real world that is back-projected from a 2D pixel point through the camera's intrinsic parameters and posture information. The state variable describes the current state of motion, which can include position, posture, and speed.

[0131] After initialization is complete and the scale information is restored, you can use monocular VIO to output odometer information:

[0132]

[0133] in and is the external parameter matrix between the camera and IMU, obtained through calibration. The camera pose calculated by the monocular visual odometer is calculated based on the information obtained by initialization, and b0 coincides with the world coordinate system w, so that the pose of the vehicle body in the world coordinate system can be output by the monocular visual odometer. b0 is the pose of the vehicle body in the first frame, and the world coordinate system established above is the world coordinate system established based on the pose of the vehicle body in the first frame.

[0134] It can be seen that in the process of positioning the robot, the embodiment of the present application integrates the data of the wheel speed odometer, the inertial measurement unit, the navigation satellite system sensor and the visual sensor, and determines whether the robot is slipping by the difference between the yaw angular velocity measured by the inertial measurement unit and the yaw angular velocity measured by the wheel speed odometer. If the robot does not slip, positioning is performed based on the position of the wheel speed odometer and the position of the navigation satellite system, which can cope with environments with weak textures or sparse feature points. If the robot slips, positioning is performed based on the data collected by the visual sensor, which can avoid the problem of the wheel speed odometer being inaccurate in complex terrain, and can also cope with scenes with complex environments and weak navigation satellite system signals. In this way, by integrating data from multiple sensors and switching in time based on the slip judgment results, the accuracy of the robot's positioning can be improved, thereby improving work efficiency.

[0135] For further information, see Figure 2 As shown, an embodiment of the present application provides a positioning flow chart of a lawn mowing robot. The robot positioning method provided by the present application can be run on the robot as an algorithm. When the algorithm is started, GPS initial alignment, slip detection and monocular-IMU-wheel speed meter joint initialization are performed synchronously. GPS initial alignment is intended to calculate the angle between the northeast sky coordinate system and the world coordinate system established by the wheel speed odometer. The purpose of the monocular-IMU-wheel speed meter joint initialization is to restore the scale information, gravity vector and vehicle body speed of the monocular. Slip detection is used to switch between the odometer recursion (i.e., wheel speed odometer recursion) and monocular visual odometer modes. When there is no slip or the slip degree is small, the current posture of the vehicle body (i.e., the robot) is output through the odometer recursion and the EKF filtering of the GPS. If the initial position alignment is unsuccessful, the current posture of the vehicle body is output through the odometer recursion, and the algorithm is started next time and re-aligned; when the slip is serious, it is switched to the monocular visual odometer to output the current posture of the vehicle body to ensure that accurate posture information can be output even in the case of slip.

[0136] This embodiment fuses multi-sensor data: first, the wheel speed odometer is used to record the rotation of the wheels and preliminarily estimate the position of the robot; the IMU provides high-precision angular velocity and acceleration information for posture estimation; the GPS provides absolute position information for correction and global positioning; the monocular vision sensor is used to extract environmental features and perform simultaneous localization and mapping (SLAM). These four sensor data complement each other, and through a reasonable fusion algorithm, accurate estimation of the position of the lawn mowing robot and stable tracking of its posture are achieved.

[0137] In addition, when the algorithm starts, the initial GPS alignment is performed to calculate the angle between the northeast celestial coordinate system and the world coordinate system established by the odometer. This ensures the consistency of the GPS data with the robot's internal coordinate system, providing an accurate benchmark for the subsequent EKF. In order to restore the scale information, gravity vector, and vehicle speed of the monocular vision, a monocular-IMU-wheel speedometer joint initialization process is designed. This process uses the pre-integration of the IMU and the speed information of the wheel speed odometer, combined with the monocular vision pose estimation, to jointly optimize the initial state and provide accurate initial parameters for the subsequent monocular vision odometer.

[0138] This embodiment performs slip detection and compensation. Considering that the lawn mowing robot is prone to slipping on outdoor grass, this application designs a slip detection and compensation mechanism. By comparing the yaw angular velocity measured by the IMU and the wheel speed encoder, the slip rate is calculated, and the need to switch to the monocular visual odometer mode is determined based on the threshold of the slip rate. This mechanism significantly improves the robot's autonomous navigation capability in complex scenarios such as slippery surfaces, and reduces positioning deviations and the risk of loss of control due to slipping. When there is no slippage or the slippage is small, the odometer recursion and GPS extended Kalman filter (EKF) are used to output the robot's current posture. The EKF algorithm can effectively integrate the observation data of the odometer and GPS, update and correct the robot's position in real time, and improve positioning accuracy.

[0139] In this way, positioning accuracy can be improved: by integrating the data of wheel speed odometer, IMU, GPS and monocular vision sensor, high-precision position estimation and posture tracking in complex outdoor environments are achieved, effectively solving the problem of low positioning accuracy in areas with weak or unstable satellite signals. In addition, stability is enhanced: the slip detection and compensation mechanism significantly improves the autonomous navigation capability of the mowing robot in complex scenarios such as slippery ground, reducing positioning deviation and risk of loss of control caused by slipping. High-precision and high-robustness autonomous navigation is achieved in complex outdoor environments, significantly improving the operating efficiency and positioning accuracy of the mowing robot in practical applications.

[0140] Furthermore, the embodiment of the present application can also depict the motion trajectory of the robot based on each position point obtained by the robot positioning to achieve mapping. For example, the mowing robot is made to circle around the fence boundary, and then the trajectory is drawn to build a map. That is, the embodiment of the present application provides a positioning and mapping solution for an outdoor mowing robot based on monocular vision, which integrates the data of the wheel speed odometer, inertial measurement unit, global positioning system and monocular vision sensor. By integrating the data of the wheel speed odometer, inertial measurement unit, GPS and monocular vision sensor, the problem of low positioning accuracy and poor stability of the existing outdoor mowing robot in complex environments is solved. It can improve the autonomous navigation capability of the mowing robot in complex scenarios such as areas with weak or unstable satellite signals and slipping caused by wet and slippery ground, and achieve high-precision position estimation and posture tracking. Through this algorithm, the mowing robot can plan the operation path more accurately, reduce repetitive work, improve work efficiency, and at the same time reduce the need for manual intervention and improve the level of intelligence.

[0141] See also Figure 3 As shown, the embodiment of the present application discloses a robot positioning device, comprising:

[0142] A yaw angular velocity acquisition module 11 is used to acquire a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, wherein the inertial measurement unit and the wheel speed odometer are both mounted on the robot;

[0143] a slip detection module 12, configured to determine whether the robot slips based on a difference between the first yaw angular velocity and the second yaw angular velocity;

[0144] a first positioning module 13, configured to determine the position of the robot based on a wheel speed odometer position and a navigation satellite system position if the robot does not slip, wherein the wheel speed odometer position is a position determined based on the wheel speed odometer, and the navigation satellite system position is a position determined based on a navigation satellite system sensor carried by the robot;

[0145] The second positioning module 14 is used to determine the position of the robot based on data collected by the visual sensor carried by the robot if the robot slips.

[0146] The skidding detection module 12 may include:

[0147] a slip rate calculation submodule, configured to calculate a slip rate based on a difference between the first yaw angular velocity and the second yaw angular velocity;

[0148] The slip judgment submodule is used to compare the slip rate with a preset threshold value. If the slip rate is lower than the preset threshold value, it is determined that the robot has not slipped; if the slip rate is greater than or equal to the preset threshold value, it is determined that the robot has slipped.

[0149] The device also includes a navigation satellite system alignment module, which may specifically include:

[0150] An angle calculation submodule, used to calculate the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer, wherein the world coordinate system is a coordinate system established based on the posture of the robot at the initial moment;

[0151] A rotation matrix calculation submodule, used to determine the rotation matrix of the navigation satellite system corresponding coordinate system converted to the world coordinate system based on the angle;

[0152] The coordinate conversion submodule is used to convert the coordinates in the corresponding coordinate system of the navigation satellite system determined based on the navigation satellite system sensor into coordinates in the world coordinate system based on the rotation matrix to obtain the position of the navigation satellite system.

[0153] Among them, the angle calculation submodule can be specifically used to select N pairs of data, each pair of data includes two wheel speed odometer position data and two navigation satellite system position data, wherein the wheel speed odometer position data is position data in a world coordinate system determined based on the wheel speed odometer, and the navigation satellite system position data is position data in a corresponding coordinate system of the navigation satellite system determined based on a navigation satellite system sensor; calculate the angle between a first vector determined by the two wheel speed odometer position data and a second vector determined by the two navigation satellite system position data in each pair of data; perform mean processing on the N angles corresponding to the N pairs of data to obtain the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer.

[0154] The first positioning module 13 may include:

[0155] The error distance determination submodule is used to determine the error distance between the wheel speed odometer position and the navigation satellite system position;

[0156] A noise determination submodule, configured to determine a noise covariance matrix corresponding to the wheel speed odometer based on the error distance;

[0157] The position correction submodule is used to correct the position of the wheel speed odometer based on the noise covariance matrix corresponding to the wheel speed odometer and using the position of the navigation satellite system to obtain the position of the robot.

[0158] Among them, the position correction submodule is specifically used to determine the predicted state covariance matrix based on the noise covariance matrix corresponding to the wheel speed odometer; determine the Kalman gain based on the predicted state covariance matrix and the noise covariance matrix corresponding to the navigation satellite system; based on the Kalman gain and using the position of the navigation satellite system, correct the position of the wheel speed odometer to obtain the position of the robot.

[0159] Furthermore, the device also includes a monocular visual odometer initialization module, which is used for, when the visual sensor is a monocular camera, before determining the position of the robot based on the data collected by the visual sensor carried by the robot, based on the position of the robot in the reference coordinate system corresponding to two adjacent frames of images, the time interval between the two adjacent frames of images, the displacement increment obtained by pre-integration of the inertial measurement unit, the speed increment obtained by pre-integration of the inertial measurement unit, the speed output of the wheel speed odometer corresponding to the two adjacent frames of images, and constructing the robot speed, gravity vector and scale factor in the reference coordinate system as the parameters to be solved. The residual term between the images is used as the objective function, wherein the reference coordinate system is a coordinate system established based on the posture of the camera; based on the objective function, the robot speed, gravity vector and scale factor in the reference coordinate system are determined; based on the weight vector in the reference coordinate system and the gravity vector in the world coordinate system, the transformation matrix from the reference coordinate system to the world coordinate system is calculated; based on the scale factor, the physical scale of the robot's position is restored; based on the transformation matrix, the state variables in the reference coordinate system are converted to the world coordinate system to obtain the initial state variables of the robot, wherein the state variables include the position, posture and speed of the robot;

[0160] Correspondingly, the second positioning module is specifically used to: output the position and posture of the robot through a monocular visual odometer based on the initial state variables and the data collected by the monocular camera carried by the robot.

[0161] It can be seen that in the process of positioning the robot, the embodiment of the present application integrates the data of the wheel speed odometer, the inertial measurement unit, the navigation satellite system sensor and the visual sensor, and determines whether the robot is slipping by the difference between the yaw angular velocity measured by the inertial measurement unit and the yaw angular velocity measured by the wheel speed odometer. If the robot does not slip, positioning is performed based on the position of the wheel speed odometer and the position of the navigation satellite system, which can cope with environments with weak textures or sparse feature points. If the robot slips, positioning is performed based on the data collected by the visual sensor, which can avoid the problem of the wheel speed odometer being inaccurate in complex terrain, and can also cope with scenes with complex environments and weak navigation satellite system signals. In this way, by integrating data from multiple sensors and switching in time based on the slip judgment results, the accuracy of the robot's positioning can be improved, thereby improving work efficiency.

[0162] See also Figure 4 As shown, an embodiment of the present application discloses a self-moving device 20, including a processor 21 and a memory 22; wherein the memory 22 is used to store a computer program; the processor 21 is used to execute the computer program, the robot positioning method disclosed in the above embodiment.

[0163] For the specific process of the above-mentioned robot positioning method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0164] Among them, autonomous devices are devices that can move automatically, and are devices that plan trajectories based on navigation intelligence to complete transportation or other functions. According to different functions and application scenarios, autonomous devices can be divided into many types, including but not limited to lawn mower robots, cleaning robots, etc.

[0165] Furthermore, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.

[0166] In addition, the self-mobile device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26; wherein the power supply 23 is used to provide working voltage for each hardware device on the self-mobile device 20; the communication interface 24 can create a data transmission channel between the self-mobile device 20 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input / output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0167] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the robot positioning method disclosed in the aforementioned embodiment.

[0168] For the specific process of the above-mentioned robot positioning method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0169] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0170] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0171] The above is a detailed introduction to a robot positioning method, device, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A robot positioning method, characterized in that: include: Acquire a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, wherein the inertial measurement unit and the wheel speed odometer are both mounted on the robot; determining whether the robot is slipping based on a difference between the first yaw angular velocity and the second yaw angular velocity; If the robot does not slip, determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position, wherein the wheel speed odometer position is a position determined based on the wheel speed odometer, and the navigation satellite system position is a position determined based on a navigation satellite system sensor carried by the robot; If the robot slips, the position of the robot is determined based on data collected by a visual sensor carried by the robot.

2. The robot positioning method according to claim 1, characterized in that: The determining whether the robot is slipping based on the difference between the first yaw angular velocity and the second yaw angular velocity comprises: calculating a slip rate based on a difference between the first yaw angular velocity and the second yaw angular velocity; The slip rate is compared with a preset threshold value. If the slip rate is lower than the preset threshold value, it is determined that the robot has not slipped. If the slip rate is greater than or equal to the preset threshold value, it is determined that the robot has slipped.

3. The robot positioning method according to claim 1, characterized in that: Before determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position, the method further includes: Calculating the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer, wherein the world coordinate system is a coordinate system established based on the posture of the robot at the initial moment; Determine a rotation matrix for converting a corresponding coordinate system of the navigation satellite system to a world coordinate system based on the angle; Based on the rotation matrix, the coordinates in the corresponding coordinate system of the navigation satellite system determined based on the navigation satellite system sensor are converted into coordinates in the world coordinate system to obtain the position of the navigation satellite system.

4. The robot positioning method according to claim 3, characterized in that: The calculating the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer includes: N pairs of data are selected, each pair of data includes two wheel speed odometer position data and two navigation satellite system position data, wherein the wheel speed odometer position data is position data in a world coordinate system determined based on the wheel speed odometer, and the navigation satellite system position data is position data in a corresponding coordinate system of the navigation satellite system determined based on a navigation satellite system sensor; Calculate the angle between the first vector determined by the two wheel speed odometer position data and the second vector determined by the two navigation satellite system position data in each pair of data; The N angles corresponding to the N pairs of data are averaged to obtain the angle between the coordinate system corresponding to the navigation satellite system and the world coordinate system corresponding to the wheel speed odometer.

5. The robot positioning method according to claim 1, characterized in that: The determining the position of the robot based on the wheel speed odometer position and the navigation satellite system position comprises: Determine the error distance between the wheel speed odometer position and the navigation satellite system position; Determine a noise covariance matrix corresponding to the wheel speed odometer based on the error distance; Based on the noise covariance matrix corresponding to the wheel speed odometer and using the position of the navigation satellite system, the position of the wheel speed odometer is corrected to obtain the position of the robot.

6. The robot positioning method according to claim 5, characterized in that: The method of correcting the position of the wheel speed odometer based on the noise covariance matrix corresponding to the wheel speed odometer and using the position of the navigation satellite system to obtain the position of the robot includes: Determining a predicted state covariance matrix based on a noise covariance matrix corresponding to the wheel speed odometer; Determine the Kalman gain based on the predicted state covariance matrix and the noise covariance matrix corresponding to the navigation satellite system; Based on the Kalman gain, the wheel speed odometer position is corrected by using the navigation satellite system position to obtain the position of the robot.

7. The robot positioning method according to any one of claims 1 to 6, characterized in that: The visual sensor is a monocular camera, and before determining the position of the robot based on the data collected by the visual sensor carried by the robot, the method further includes: Based on the position of the robot corresponding to two adjacent frames of images in the reference coordinate system, the time interval between the two adjacent frames of images, the displacement increment obtained by pre-integration of the inertial measurement unit, the speed increment obtained by pre-integration of the inertial measurement unit, and the speed output by the wheel speed odometer corresponding to the two adjacent frames of images, and taking the robot speed, gravity vector and scale factor in the reference coordinate system as the parameters to be solved, a residual term between two adjacent frames of images is constructed as an objective function, wherein the reference coordinate system is a coordinate system established based on the posture of the camera; Determine the robot speed, gravity vector and scale factor in the reference coordinate system based on the objective function; Calculate a transformation matrix from the reference coordinate system to the world coordinate system based on the weight vector in the reference coordinate system and the gravity vector in the world coordinate system; Restoring the physical scale of the robot's position based on the scale factor; Based on the transformation matrix, the state variables in the reference coordinate system are converted to the world coordinate system to obtain the initial state variables of the robot, wherein the state variables include the position, posture and speed of the robot; Accordingly, determining the position of the robot based on data collected by a visual sensor carried by the robot includes: Based on the initial state variables and data collected by a monocular camera carried by the robot, the position and posture of the robot are output through a monocular visual odometer.

8. A robot positioning device, characterized in that: include: A yaw angular velocity acquisition module, which acquires a first yaw angular velocity measured by an inertial measurement unit and a second yaw angular velocity measured by a wheel speed odometer, wherein the inertial measurement unit and the wheel speed odometer are both mounted on the robot; a slip detection module, for determining whether the robot slips based on a difference between the first yaw angular velocity and the second yaw angular velocity; a first positioning module, which determines the position of the robot based on a wheel speed odometer position and a navigation satellite system position if the robot does not slip, wherein the wheel speed odometer position is a position determined based on the wheel speed odometer, and the navigation satellite system position is a position determined based on a navigation satellite system sensor carried by the robot; The second positioning module determines the position of the robot based on data collected by a visual sensor carried by the robot if the robot slips.

9. A self-propelled device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store the computer program; The processor is used to execute the computer program to implement the robot positioning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the robot positioning method according to any one of claims 1 to 7 is implemented.

Images