Multi-source fusion positioning method of mobile robot and mowing robot

Abstract

The embodiment of the invention provides a multi-source fusion positioning method of a mobile robot and a mowing robot. The method comprises the following steps: acquiring an inertial measurement state measured at a first moment; according to the inertial measurement state, predicting to obtain a motion state of the movable robot at a second moment, and taking the motion state as an inertial prediction state; acquiring a visual observation state measured at a second moment; acquiring a satellite navigation observation state measured at a second moment; if the obtained satellite navigation observation state meets a preset available condition, fusing the inertial prediction state, the visual observation state and the satellite navigation observation state to obtain a fused state; and if the obtained satellite navigation observation state does not meet the preset available condition, fusing the inertial prediction state, the visual observation state and the wheel speed meter observation state to obtain a fused state. By applying the technical scheme provided by the embodiment of the invention, the positioning precision of the mobile robot can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to a multi-source fusion positioning method for a mobile robot and a lawn mowing robot. Background Art

[0002] For mobile robots with insufficient computing power such as lawn mowers and cleaning robots, in the process of realizing autonomous positioning, a multi-sensor fusion positioning scheme mainly based on visual sensors is usually adopted, and a fusion method based on a filtering framework is used. However, in related technologies, the utilization of sensor data is limited, and it is difficult to accurately realize the positioning of mobile robots. Therefore, how to improve the positioning accuracy of mobile robots has become an urgent problem to be solved. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a multi-source fusion positioning method for a mobile robot and a lawn mowing robot to improve the positioning accuracy of the mobile robot. The specific technical solutions are as follows:

[0004] In a first aspect, the embodiments of the present application provide a multi-source fusion positioning method for a mobile robot, and the method includes:

[0005] Obtain the inertial measurement state measured by an inertial sensor disposed on the mobile robot at a first moment;

[0006] According to the inertial measurement state, predict the motion state of the mobile robot at a second moment as an inertial prediction state;

[0007] Obtain the motion state measured by a visual sensor disposed on the mobile robot at the second moment as a visual observation state; and obtain the motion state measured by a satellite navigation sensor disposed on the mobile robot at the second moment as a satellite navigation observation state;

[0008] If the obtained satellite navigation observation state meets a preset available condition, fuse the inertial prediction state, the visual observation state, and the satellite navigation observation state to obtain a fusion state as the actual motion state of the mobile robot at the second moment;

[0009] If the obtained satellite navigation observation state does not meet the preset available condition, fuse the inertial prediction state, the visual observation state, and the wheel speedometer observation state to obtain the fusion state as the actual motion state of the mobile robot at the second moment, where the wheel speedometer observation state is the motion state measured by a wheel speedometer disposed on the mobile robot at the second moment.

[0010] In some embodiments, the states are fused in the following manner, including:

[0011] Based on the accuracy of each state to be fused respectively, determine the weight of each state, where the weight of the state is positively correlated with the accuracy of the state;

[0012] Calculate the residuals between each observed state to be fused and the inertial prediction state respectively, as the residuals corresponding to each observed state;

[0013] Perform a weighted sum of the residuals corresponding to each observed state and the inertial prediction state according to the weight of each observed state, to obtain the fused state.

[0014] In some embodiments, the accuracy of the inertial prediction state is determined in the following manner:

[0015] Obtain the inertial historical state of the inertial sensor at the first moment, where the first moment is the second moment and multiple historical moments before the second moment;

[0016] Determine the prediction error of the inertial prediction state according to the inertial prediction state and each of the inertial historical states; the accuracy of the inertial prediction is negatively correlated with the prediction error.

[0017] In some embodiments, the prediction error is calculated by the following formula:

[0018] P k+1|k =Φ(t k+1 ,t k )P k|k Φ(t k+1 ,t k ) T +Q k

[0019] where P k+1|k represents the prediction error of the inertial prediction state, P k|k represents the covariance matrix of the inertial measurement state, Φ(t k+1 ,t k ) represents the state transition matrix, and Q k represents the noise covariance matrix.

[0020] In some embodiments, the residual corresponding to the visual observation state is determined in the following manner:

[0021] Obtain a first image and a second image, where the first image is the image measured by the visual sensor at the first moment, and the second image is the image measured by the visual sensor at the second moment;

[0022] Determine a first projection point in the first image, and determine a second projection point in the second image that feature-matches the first projection point;

[0023] When projecting the first projection point and the second projection point onto the same coordinate system, calculate the distance between the first projection point and the second projection point; the residual corresponding to the visual observation state is positively correlated with the distance between the first projection point and the second projection point.

[0024] In some embodiments, determine the residual corresponding to the satellite navigation observation state by the following method, including:

[0025] Obtain first position data, second position data, and theoretical offset data; wherein, the first position data is the position data measured by the satellite navigation sensor at the second moment, the second position data is the position data measured by the inertial sensor at the second moment, and the theoretical offset data is the relative offset between the position data measured by the satellite navigation sensor and the position data measured by the inertial sensor under theoretical circumstances;

[0026] Determine the degree of deviation of the first position relative to the second position according to the first position data, the second position data, and the theoretical offset data; the residual corresponding to the satellite navigation observation state is positively correlated with the degree of deviation of the first position relative to the second position.

[0027] In some embodiments, calculate the degree of deviation of the first position relative to the second position by the following formula:

[0028]

[0029] Wherein, represents the degree of deviation of the first position relative to the second position, represents the first position data in the visual and inertial fusion coordinate system, represents the second position data in the visual and inertial fusion coordinate system, represents the conversion relationship between the visual and inertial fusion coordinate system and the inertial coordinate system, I p G represents the theoretical offset data between the visual sensor and the inertial sensor in the inertial coordinate system.

[0030] In some embodiments, determine the residual corresponding to the wheel speedometer observation state by the following method:

[0031] Calculate the difference degree between the first speed and the second speed; wherein, the first speed is the speed of the mobile robot measured by the wheel speed sensor at the second moment, and the second speed is the speed of the mobile robot measured by the inertial sensor at the second moment; the residual corresponding to the wheel speedometer observation state is positively correlated with the difference degree between the first speed and the second speed.

[0032] In some embodiments, the difference degree between the first speed and the second speed is calculated by the following formula:

[0033]

[0034] wherein, represents the difference degree between the first speed and the second speed, represents the conversion relationship between the inertial coordinate system and the wheel speedometer coordinate system, represents the first speed in the wheel speedometer coordinate system; represents the second speed in the inertial coordinate system.

[0035] In some embodiments, the weights of each state are determined by the following formula,

[0036]

[0037] wherein, K represents the weight of the observation state to be fused, and H k represents the observation matrix, represents the covariance matrix of the inertial measurement state, and R k represents the covariance matrix of the observation state to be fused.

[0038] In some embodiments, the H k is a 3×15 matrix;

[0039] When the fused state includes attitude, the sub-matrix formed by the 1st to 3rd columns of the H k is the identity matrix;

[0040] When the fused state includes position, the sub-matrix formed by the 4th to 6th columns of the H k is the identity matrix;

[0041] When the fused state includes speed, the sub-matrix formed by the 7th to 9th columns of the H k is the identity matrix.

[0042] In some embodiments, the satellite navigation sensor is a single-antenna based sensor, and the method further includes:

[0043] Obtain the position data measured by the satellite navigation sensor at the second moment as the third position data; and obtain the position data at at least one moment before the second moment as the fourth position data;

[0044] Determine the orientation of the mobile robot at the second moment according to the third position data and each of the fourth position data.

[0045] In some embodiments, the determining the orientation of the mobile robot at the second moment according to the third position data and each of the fourth position data includes:

[0046] For the third position data and each of the fourth position data, determine the orientation of each adjacent position as the initial orientation;

[0047] Perform a weighted sum of each of the initial orientations according to the weights of the initial orientations to obtain the orientation of the mobile robot at the second moment; wherein, the weight of the initial orientation is positively correlated with the distance between the adjacent positions to which the initial orientation belongs.

[0048] In some embodiments, the inertial measurement state measured by the inertial sensor includes any one or more of the following states: position, attitude, velocity;

[0049] The visual measurement state measured by the visual sensor includes any one or more of the following states: position, attitude, velocity;

[0050] The satellite navigation observation state measured by the satellite navigation sensor includes any one or more of the following states: position, attitude, velocity;

[0051] The wheel speedometer observation state measured by the wheel speedometer sensor includes: velocity.

[0052] In a second aspect, an embodiment of the present application provides a multi-source fusion positioning device for a mobile robot, and the method includes:

[0053] A first acquisition module, configured to acquire the inertial measurement state measured by the inertial sensor disposed on the mobile robot at the first moment;

[0054] A prediction module, configured to predict the motion state of the mobile robot at the second moment according to the inertial measurement state as the inertial prediction state;

[0055] A second acquisition module, configured to acquire the motion state measured by the visual sensor disposed on the mobile robot at the second moment as the visual observation state; and acquire the motion state measured by the satellite navigation sensor disposed on the mobile robot at the second moment as the satellite navigation observation state;

[0056] A fusion module, configured to, if the obtained satellite navigation observation state meets a preset available condition, fuse the inertial prediction state, the visual observation state, and the satellite navigation observation state to obtain a fusion state as the actual motion state of the mobile robot at the second moment; if the obtained satellite navigation observation state does not meet the preset available condition, fuse the inertial prediction state, the visual observation state, and the wheel speedometer observation state to obtain the fusion state as the actual motion state of the mobile robot at the second moment, where the wheel speedometer observation state is the motion state measured by a wheel speedometer disposed on the mobile robot at the second moment.

[0057] In some embodiments, the states are fused in the following manner, including:

[0058] Respectively determine the weights of the states to be fused based on the accuracy of each state to be fused, where the weight of a state is positively correlated with the accuracy of the state;

[0059] Respectively calculate the residuals between each observation state to be fused and the inertial prediction state as the residuals corresponding to each observation state;

[0060] Perform a weighted sum of the residuals corresponding to each observation state and the inertial prediction state according to the weights of each observation state to obtain the fusion state.

[0061] In some embodiments, the accuracy of the inertial prediction state is determined in the following manner:

[0062] Obtain the inertial historical state of the inertial sensor at the first moment, where the first moment is the second moment and multiple historical moments before the second moment;

[0063] Determine the prediction error of the inertial prediction state according to the inertial prediction state and each of the inertial historical states; the accuracy of the inertial prediction is negatively correlated with the prediction error.

[0064] In some embodiments, the prediction error is calculated by the following formula:

[0065] P k+1|k =Φ(t k+1 ,t k )P k|k Φ(t k+1 ,t k ) T +Q k

[0066] where P k+1|k represents the prediction error of the inertial prediction state, Pk|k represents the covariance matrix of the inertial measurement state, Φ(t k+1 , t k ) represents the state transition matrix, Q k represents the noise covariance matrix.

[0067] In some embodiments, the residual corresponding to the visual observation state is determined by the following method:

[0068] Obtain a first image and a second image, where the first image is the image measured by the visual sensor at the first moment, and the second image is the image measured by the visual sensor at the second moment;

[0069] Determine a first projection point in the first image, and determine a second projection point in the second image that is feature-matched with the first projection point;

[0070] When the first projection point and the second projection point are projected onto the same coordinate system, calculate the distance between the first projection point and the second projection point; the residual corresponding to the visual observation state is positively correlated with the distance between the first projection point and the second projection point.

[0071] In some embodiments, the residual corresponding to the satellite navigation observation state is determined by the following method, including:

[0072] Obtain first position data, second position data, and theoretical offset data; wherein, the first position data is the position data measured by the satellite navigation sensor at the second moment, the second position data is the position data measured by the inertial sensor at the second moment, and the theoretical offset data is the relative offset between the position data measured by the satellite navigation sensor and the position data measured by the inertial sensor under theoretical circumstances;

[0073] According to the first position data, the second position data, and the theoretical offset data, determine the degree of deviation of the first position relative to the second position; the residual corresponding to the satellite navigation observation state is positively correlated with the degree of deviation of the first position relative to the second position.

[0074] In some embodiments, the degree of deviation of the first position relative to the second position is calculated by the following formula:

[0075]

[0076] wherein, represents the degree of deviation of the first position relative to the second position, represents the first position data in the visual and inertial fusion coordinate system, Represents the second position data in the visual and inertial fusion coordinate system, Represents the conversion relationship between the visual and inertial fusion coordinate system and the inertial coordinate system, I p G Represents the theoretical offset data between the visual sensor and the inertial sensor in the inertial coordinate system.

[0077] In some embodiments, the residual corresponding to the wheel speedometer observation state is determined by the following method:

[0078] Calculate the difference degree between the first speed and the second speed; wherein, the first speed is the speed of the mobile robot measured by the wheel speedometer sensor at the second moment, and the second speed is the speed of the mobile robot measured by the inertial sensor at the second moment; the residual corresponding to the wheel speedometer observation state is positively correlated with the difference degree between the first speed and the second speed.

[0079] In some embodiments, the difference degree between the first speed and the second speed is calculated by the following formula:

[0080]

[0081] Wherein, Represents the difference degree between the first speed and the second speed, Represents the conversion relationship between the inertial coordinate system and the wheel speedometer coordinate system, Represents the first speed in the wheel speedometer coordinate system; Represents the second speed in the inertial coordinate system.

[0082] In some embodiments, the weights of each state are determined by the following formula,

[0083]

[0084] Wherein, K represents the weight of the observation state to be fused, H k Represents the observation matrix, Represents the covariance matrix of the inertial measurement state, R k Represents the covariance matrix of the observation state to be fused.

[0085] In some embodiments, the H k Is a 3×15 matrix;

[0086] When the fusion state includes attitude, the sub-matrix formed by the 1st to 3rd columns of the H k Is an identity matrix;

[0087] When the fusion state includes position, the H kThe sub-matrix formed by the 4th to 6th columns is an identity matrix;

[0088] When the fusion state includes speed, the H k The sub-matrix formed by the 7th to 9th columns is an identity matrix.

[0089] In some embodiments, the satellite navigation sensor is a single-antenna-based sensor, and the device further includes:

[0090] A third acquisition module, configured to acquire the position data measured by the satellite navigation sensor at the second moment as the third position data; and acquire the position data at at least one moment before the second moment as the fourth position data;

[0091] An orientation determination module, configured to determine the orientation of the mobile robot at the second moment according to the third position data and each of the fourth position data.

[0092] In some embodiments, the orientation determination module is specifically configured to:

[0093] For the third position data and each of the fourth position data, determine the orientation of each adjacent position as the initial orientation;

[0094] Perform a weighted sum of each of the initial orientations according to the weights of the initial orientations to obtain the orientation of the mobile robot at the second moment; wherein, the weight of the initial orientation is positively correlated with the distance between the adjacent positions to which the initial orientation belongs.

[0095] In some embodiments, the inertial measurement state measured by the inertial sensor includes any one or more of the following states: position, attitude, speed;

[0096] The visual measurement state measured by the visual sensor includes any one or more of the following states: position, attitude, speed;

[0097] The satellite navigation observation state measured by the satellite navigation sensor includes any one or more of the following states: position, attitude, speed;

[0098] The wheel speedometer observation state measured by the wheel speedometer sensor includes: speed.

[0099] In a third aspect, an embodiment of the present application further provides an outdoor robot, where an inertial sensor, a satellite navigation sensor, a visual sensor, and a wheel speedometer sensor are arranged on the outdoor mobile robot; the outdoor robot is configured to execute the multi-source fusion positioning method of any one of the above-mentioned mobile robots.

[0100] Fourthly, an embodiment of the present application further provides a lawn mowing robot, which is provided with an inertial sensor, a satellite navigation sensor, a vision sensor, and a wheel speed sensor; the lawn mowing robot is used for the multi-source fusion positioning method of any one of the above-mentioned mobile robots.

[0101] Fifthly, an embodiment of the present application further provides an electronic device, including:

[0102] a memory for storing a computer program;

[0103] a processor for implementing the multi-source fusion positioning method of any one of the above-mentioned mobile robots when executing the program stored in the memory.

[0104] Sixthly, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program realizes the multi-source fusion positioning method of any one of the above-mentioned mobile robots when executed by a processor.

[0105] Seventhly, an embodiment of the present application further provides a computer program product containing instructions, which when running on a computer, enables the computer to execute the multi-source fusion positioning method of any one of the above-mentioned mobile robots.

[0106] Eighthly, an embodiment of the present application further provides a computer program product containing instructions, which when running on a computer, enables the computer to execute the multi-source fusion positioning method of any one of the above-mentioned mobile robots.

[0107] Advantageous effects of the embodiments of the present application:

[0108] In the technical solution provided by the embodiment of the present application, the motion state of the mobile robot measured by the inertial sensor at the first moment, that is, the inertial measurement state, is obtained; furthermore, the motion state of the mobile robot at the second moment is predicted according to the inertial measurement state as the inertial prediction state; then, the motion state of the mobile robot measured by the vision sensor at the second moment is obtained as the vision observation state; and the motion state of the mobile robot measured by the satellite navigation sensor at the second moment is obtained as the satellite navigation observation state; in this process, if the obtained satellite navigation observation state meets the preset available conditions, the inertial prediction state, the vision observation state, and the satellite navigation observation state are fused to obtain the fusion state; if the obtained satellite navigation observation state does not meet the preset available conditions, the inertial prediction state, the vision observation state, and the wheel speed sensor observation state are fused to obtain the fusion state. Thus, the motion states measured by each sensor are fully utilized to improve the accuracy of the fusion positioning of the mobile robot.

[0109] In addition, it can be understood that when the acquired satellite navigation observation state does not meet the preset available conditions, due to the problems of speed divergence and scale drift in the vision sensor and the inertial sensor, the positioning accuracy of the mobile robot may be relatively low. Therefore, in the technical solution provided by the embodiments of the present application, when the acquired satellite navigation observation state does not meet the preset available conditions, the inertial prediction state, the vision observation state, and the wheel speedometer observation state are fused to obtain a fusion state, avoiding the problem of relatively low positioning accuracy caused by the speed divergence and scale drift in the vision sensor and the inertial sensor, and improving the positioning accuracy of the mobile robot.

[0110] Of course, implementing any product or method of the present application does not necessarily require achieving all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0112] Figure 1 A schematic diagram of each coordinate system involved in the positioning process of the mobile robot provided by the embodiments of the present application;

[0113] Figure 2 A first flow schematic diagram of the multi-source fusion positioning method of the mobile robot provided by the embodiments of the present application;

[0114] Figure 3 A refinement schematic diagram for the above-mentioned step S24 or step S25 provided by the embodiments of the present application;

[0115] Figure 4 A flow schematic diagram of the system initialization provided by the embodiments of the present application;

[0116] Figure 5 A flow schematic diagram of determining the attitude by the satellite navigation sensor in the case of a single antenna provided by the embodiments of the present application;

[0117] Figure 6 A second flow schematic diagram of the multi-source fusion positioning method of the mobile robot provided by the embodiments of the present application;

[0118] Figure 7 A third flow schematic diagram of the multi-source fusion positioning method of the mobile robot provided by the embodiments of the present application;

[0119] Figure 8A schematic structural diagram of a multi-source fusion positioning device for a mobile robot provided by an embodiment of the present application;

[0120] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0121] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0122] For a mobile robot to achieve autonomous positioning, it usually relies on vision sensors and radar sensors as the main sensors, supplemented by inertial sensors, wheel speed sensors, and satellite navigation sensors for fusion positioning to cope with complex working scenarios. In a few scenarios, technologies such as UWB (Ultra Wide Band) and artificial markers are used for fusion positioning. For multi-sensor fusion positioning, some sensor combination strategies have a high cost but good reliability; while some sensor combination strategies have a low cost but poor reliability. For example, for a multi-sensor fusion positioning scheme mainly based on radar sensors, its sensor combination strategy has a high cost but good reliability; while for a multi-sensor fusion positioning scheme mainly based on vision sensors, its sensor combination strategy has a low cost, but its reliability is lower than that of the multi-sensor fusion positioning scheme mainly based on radar.

[0123] In terms of the fusion method, there are two mainstream solutions: the optimization framework-based and the filtering framework-based. The former has high accuracy but a large computational cost, and the latter is the opposite. Usually, for devices equipped with sufficient computing power, the optimization framework can be adopted, for example, autonomous vehicles in the field of autonomous driving; while for devices with limited computing power, the filtering-based framework is more suitable, for example, lawn mowing robots and cleaning robots, etc.

[0124] Considering the above two factors, for mobile robots with insufficient computing power such as lawn mowers and cleaning robots, it is more suitable to adopt a multi-sensor fusion positioning scheme mainly based on vision sensors and a fusion method based on the filtering framework. However, in related technologies, the utilization of sensor data is limited, and there is a lack of a unified multi-sensor fusion positioning scheme, especially in the use of satellite navigation sensors and wheel speed sensors.

[0125] Specifically, for the convenience of understanding the solution of the present application, hereinafter, taking the following several related technologies as examples, the differences between the present application and the related technologies will be described:

[0126] Solution 1: A positioning method, device, terminal, medium, and mowing robot for a mowing robot. This solution is for the mowing machine scenario and uses a vision sensor, satellite navigation sensor, inertial sensor, and target image for fusion positioning. However, it mainly focuses on the introduction and use of target information and does not disclose a detailed fusion positioning method.

[0127] Compared with the above Solution 1, the multi-source fusion positioning method of the mobile robot provided in the embodiments of the present application can perform fusion positioning based on multiple sensors such as a vision sensor, inertial sensor, wheel speed sensor, and satellite navigation sensor. It supports a richer set of sensors for fusion, can fully realize the complementary advantages of each sensor, and these sensors are commonly used in mobile robots.

[0128] Solution 2: A positioning method and device. This solution is for the autonomous driving scenario and uses an inertial sensor, satellite navigation sensor, and vision sensor for fusion positioning. Specifically, when the satellite condition is good, the inertial sensor and satellite navigation sensor are used for fusion positioning; when the satellite condition is poor, the inertial sensor and vision sensor are used for fusion positioning. And in this solution, the extended Kalman filter is used for fusion. Among them, the judgment of the satellite condition takes into account the dilution of precision, the number of satellites, signal strength, and signal-to-noise ratio.

[0129] For the above Solution 2, it performs fusion or state switching based on some sensors, with limited utilization of data, which is not conducive to dealing with the complex working conditions of outdoor mowing machines. And the wheel speed sensor is not used in the above Solution 2.

[0130] Solution 3: A positioning method, device, equipment, and storage medium for a mobile terminal. This solution is for the autonomous driving scenario and uses an inertial sensor, satellite navigation sensor, wheel speed sensor, and vision sensor for fusion positioning. The fusion method is to optimize based on constructing a factor graph using an optimization framework.

[0131] Compared with the above Solution 3, the fusion method adopted in Solution 3 is factor graph optimization, while the fusion method adopted in the present application is Kalman filtering.

[0132] Solution 4: A positioning method, electronic device, and computer-readable storage medium for a mowing robot. This solution is for the mowing machine scenario and uses an inertial sensor, wheel speed sensor, or satellite navigation sensor for fusion positioning, respectively obtaining the covariance of the inertial sensor and the wheel speed sensor or satellite navigation sensor for fusion filtering.

[0133] However, the above-mentioned Solution 4 does not consider visual sensors. Since there will be large cumulative errors in inertial sensors and wheel speedometers, and satellite navigation sensors are easily restricted, when the satellite navigation sensor is restricted, the accuracy of the positioning result obtained based on the positioning solution provided by the above-mentioned Solution 4 is relatively low.

[0134] Solution 5: A vision positioning method, device, equipment and storage medium for a lawn mower. This solution aims at the lawn mower scenario and uses inertial, vision and signs for integrated positioning, and jointly optimizes the poses obtained based on signs and the poses obtained by visual sensors and inertial sensors.

[0135] Solution 6: A relocalization method, system and readable storage medium for a lawn mower robot. This solution aims at the lawn mower scenario and uses navigation data and magnetic induction data for relocalization.

[0136] Solution 7: A lawn mower robot and a lawn mower robot system having the same. This solution aims at the lawn mower scenario and uses laser and UWB technologies for integrated positioning.

[0137] Based on the above content, to make full use of the motion states provided by each sensor and improve the positioning accuracy of the mobile robot, the embodiments of the present application provide a multi-source fusion positioning method for a mobile robot.

[0138] The multi-source fusion positioning method for a mobile robot provided by the embodiments of the present application can be applied to the mobile robot itself. For example, the mobile robot itself is equipped with a processor, through which the motion states measured by each sensor can be obtained, and the motion states can be fused according to the multi-source fusion positioning method provided by the present application to achieve fusion positioning. Or, the technical solution provided by the embodiments of the present application can also be applied to an electronic device disposed outside the mobile robot. Through this electronic device, the motion states measured by each sensor can be obtained, and the motion states can be fused according to the multi-source fusion positioning method provided by the present application to achieve fusion positioning.

[0139] Among them, the mobile robot can be a lawn mower robot, a cleaning robot and other robots; the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, etc. In the following content, for the convenience of description, the electronic device is used as the execution subject for explanation.

[0140] Next, for the convenience of explaining the multi-source fusion positioning method for a mobile robot provided by the present application, each sensor used in the present application and each coordinate system involved in the positioning process of the mobile robot will be explained first.

[0141] Specifically, the motion states that can be directly or indirectly obtained by various common sensors are described in Table 1 below. Table 1: Motion states that can be directly or indirectly obtained by common sensors

[0142]

[0143] It can be understood that the motion state in this article can be any one or more of the position, attitude, and speed of the mobile robot. Based on Table 1 above, it can be seen that vision sensors, lidar sensors, inertial sensors, and satellite navigation sensors can all measure the position, attitude, and speed of the mobile robot; the wheel speed sensor can only measure the speed of the mobile robot; the magnetic sensor can only measure the attitude of the mobile robot.

[0144] Taking the vision sensor as a camera as an example, through the following Figure 1 , the various coordinate systems involved in the positioning process of the mobile robot are described. As shown in Figure 1 , an ENU (East-North-Up) coordinate system, that is, the coordinate system {E} in Figure 1 , is constructed on the mobile robot; a VIO (visual-inertial odometry) coordinate system, that is, the coordinate system {V} in Figure 1 ; a wheel speedometer coordinate system, that is, the coordinate system {O} in Figure 1 ; a satellite navigation coordinate system, that is, the coordinate system {G} in Figure 1 ; a camera coordinate system, that is, the coordinate system {C} in Figure 1 ; an inertial coordinate system, that is, the coordinate system {I} in Figure 1 .

[0145] It can be understood that there are multiple variables involved in the following content of this application, and each variable has multiple subscripts. For the convenience of understanding the meaning of the variables in the following content of this application, here, taking the variable x as an example, the meaning represented by the subscripts carried by the variable is described. For example, A x k , where x represents the variable, usually a multi-dimensional vector or matrix; the superscript A indicates that the coordinate system where the variable x is located is A; A x k represents the value of the variable x at the k-th moment in the coordinate system A; for another example, where x represents the variable, usually a coordinate transformation matrix or quaternion q, T represents the matrix transpose, represents the transpose of the transformation from coordinate system A to coordinate system B.

[0146] Next, the multi-source fusion positioning method for the mobile robot provided by this application is described through specific embodiments.

[0147] See Figure 2 , Figure 2 which is the first process schematic diagram of the multi-source fusion positioning method for a mobile robot provided by an embodiment of this application, and may include the following steps:

[0148] Step S21: Obtain the inertial measurement state measured by the inertial sensor disposed on the mobile robot at the first moment;

[0149] Step S22: According to the inertial measurement state, predict the motion state of the mobile robot at the second moment as the inertial prediction state;

[0150] Step S23: Obtain the motion state measured by the visual sensor disposed on the mobile robot at the second moment as the visual observation state; and obtain the motion state measured by the satellite navigation sensor disposed on the mobile robot at the second moment as the satellite navigation observation state;

[0151] Step S24: If the obtained satellite navigation observation state meets the preset available condition, fuse the inertial prediction state, the visual observation state, and the satellite navigation observation state to obtain a fusion state as the actual motion state of the mobile robot at the second moment;

[0152] Step S25: If the obtained satellite navigation observation state does not meet the preset available condition, fuse the inertial prediction state, the visual observation state, and the wheel speedometer observation state to obtain a fusion state as the actual motion state of the mobile robot at the second moment, where the wheel speedometer observation state is the motion state measured by the wheel speedometer disposed on the mobile robot at the second moment.

[0153] In the technical solution provided by the embodiment of this application, obtain the motion state of the mobile robot measured by the inertial sensor at the first moment, that is, the inertial measurement state; furthermore, predict the motion state of the mobile robot at the second moment according to the inertial measurement state as the inertial prediction state; then, obtain the motion state of the mobile robot measured by the visual sensor at the second moment as the visual observation state; and obtain the motion state of the mobile robot measured by the satellite navigation sensor at the second moment as the satellite navigation observation state; during this process, if the obtained satellite navigation observation state meets the preset available condition, fuse the inertial prediction state, the visual observation state, and the satellite navigation observation state to obtain a fusion state; if the obtained satellite navigation observation state does not meet the preset available condition, fuse the inertial prediction state, the visual observation state, and the wheel speedometer observation state to obtain a fusion state. Thus, make full use of the motion states measured by each sensor to improve the accuracy of the fusion positioning of the mobile robot.

[0154] In addition, it can be understood that when the acquired satellite navigation observation state does not meet the preset available conditions, due to the problems of speed divergence and scale drift in the vision sensor and the inertial sensor, the positioning accuracy of the mobile robot may be relatively low. Therefore, in the technical solution provided in the embodiments of the present application, when the acquired satellite navigation observation state does not meet the preset available conditions, the inertial prediction state, the vision observation state, and the wheel speedometer observation state are fused to obtain a fused state, avoiding the problem of relatively low positioning accuracy caused by the speed divergence and scale drift in the vision sensor and the inertial sensor, and improving the positioning accuracy of the mobile robot.

[0155] In the above step S21: The inertial sensor is disposed on the mobile robot and can measure the motion state of the mobile robot in real time. The inertial measurement state is the motion state of the mobile robot measured by the inertial sensor at the first moment. It can be understood that based on the above Table 1, the inertial sensor can measure the position, speed, and attitude of the mobile robot. Therefore, the inertial measurement state measured by the inertial sensor can include any one or more of the position, speed, and attitude, specifically depending on the inertial prediction state to be obtained in the subsequent step S22. That is, the inertial measurement state measured by the inertial sensor can be the same motion state as the inertial prediction state, or the inertial measurement state measured by the inertial sensor can also be a motion state that can be used to predict the inertial prediction state. For example, if the inertial prediction state is speed, the inertial measurement state can also be speed, or the inertial measurement state can also include other motion states related to the inertial prediction state in addition to speed. For example, when the inertial prediction state is speed, the motion states related to the inertial prediction state in addition to speed also include position and attitude. Then, when the inertial prediction state is speed, the inertial measurement state can be speed, position, and attitude. The description of the inertial prediction state can be referred to the content in the following step S22.

[0156] The first moment in the embodiments of the present application can be the second moment and multiple historical moments before the second moment, or can be only multiple historical moments before the second moment. Among them, the second moment is the current moment.

[0157] Taking the first moment as the second moment and n historical moments before the second moment as an example, the inertial measurement state acquired by the electronic device at the first moment can be expressed in the form shown in the following formula (1):

[0158]

[0159] Among them, V x k represents the inertial measurement state acquired by the electronic device at the first moment; Represents the motion state measured by the inertial sensor at the second moment; Represents the motion states measured by the inertial sensor at n historical moments before the second moment.

[0160] Specifically, among them, Can be recorded in the form shown in the following formula (2),

[0161]

[0162] Among them, Is a quaternion, representing the attitude measured by the inertial sensor at the second moment; Represents the three-dimensional position coordinates measured by the inertial sensor at the second moment; Represents the three-dimensional velocity measured by the inertial sensor at the second moment; Represents the angular velocity bias measured by the inertial sensor at the second moment; Represents the acceleration bias measured by the inertial sensor at the second moment.

[0163] Can be recorded in the form shown in the following formula (3),

[0164]

[0165] Among them, Represents the attitude measured by the inertial sensor at the (k - 1)th moment; Represents the three-dimensional position coordinates measured by the inertial sensor at the (k - 1)th moment; Represents the attitude measured by the inertial sensor at the (k - n)th moment, Represents the three-dimensional position coordinates measured by the inertial sensor at the (k - n)th moment. Among them, k - n represents n historical moments before the second moment. For example, k - 1 represents the first historical moment before the second moment.

[0166] In step S22, the inertial prediction state is the motion state of the mobile robot at the second moment predicted by the electronic device according to the inertial measurement state. Among them, the second moment is the current moment. And the specific motion states included in the inertial prediction state depend on which motion states the fusion state specifically includes. That is, the inertial prediction state can be the same motion states as the fusion state, or the inertial prediction state can be the motion states related to determining the fusion state. Specifically, if the fusion state is velocity, the inertial prediction state can also be velocity, or the inertial prediction state can also include other motion states related to the fusion state except velocity. For example, when the fusion state is velocity, the motion states related to the fusion state include position and attitude in addition to velocity. Then, when the fusion state is velocity, the inertial prediction state can be velocity, position, and attitude.

[0167] Next, taking the inertial prediction states as position, velocity, and attitude as an example, the process of predicting the inertial prediction states based on the inertial measurement states will be described through specific examples. If any of position, velocity, and attitude is not included in the inertial prediction states during the actual application process, no calculation needs to be performed for that one.

[0168] It can be understood that the inertial measurement sensor can output direct measurements of acceleration and angular velocity, and indirect measurements of position, attitude, and velocity can be achieved through integration. Among them, the modeling of the acceleration measured by the inertial sensor is shown in the following formula (4), and the modeling of the angular velocity is shown in the following formula (5):

[0169]

[0170] Among them, α m represents the actually measured acceleration; α represents the acceleration in the theoretical case; represents the gravitational acceleration converted from the I coordinate system to the V coordinate system; b α represents the acceleration bias; n α represents the Gaussian noise for the acceleration.

[0171] w m = w + b w + n w (5)

[0172] Among them, w m represents the actually measured angular velocity; w represents the angular velocity in the theoretical case; b w represents the angular velocity bias; n w represents the Gaussian noise for the angular velocity.

[0173] During the prediction process, the motion state of the mobile robot at the second moment, that is, the inertial prediction state, can be predicted based on the inertial measurement state measured by the inertial sensor at the first moment, as shown in the following formula (6):

[0174]

[0175] Among them, represents the motion state at the second moment in the V coordinate system, that is, the predicted inertial prediction state; represents the motion state at the first moment in the V coordinate system; represents the acceleration measured by the inertial sensor at the first moment, represents the angular velocity measured by the inertial sensor at the first moment.

[0176] In the above step S23, the vision sensor is disposed on the mobile robot and can measure the motion state of the mobile robot in real time. The vision observation state is the motion state of the mobile robot measured by the vision sensor at the second moment. It can be understood that based on Table 1 above, the vision sensor can measure the position, speed, and attitude of the mobile robot. Therefore, the vision observation state measured by the vision sensor can include any one or more of the position, speed, and attitude, specifically depending on the fusion state required in the subsequent step S24 or step S25. That is, the vision observation state can be the same motion state as the fusion state, or the inertial prediction state can also be the motion state related to determining the fusion state. For example, if the fusion state is speed, the vision observation state can also be speed, or the vision observation state can also include other motion states related to the fusion state in addition to speed. For example, when the fusion state is speed, the motion states related to the fusion state include position and attitude in addition to speed. Then, when the fusion state is speed, the vision observation state can be speed, position, and attitude. The description of the fusion state can be found in the content of step S23 or step S24 below.

[0177] Similarly, the satellite navigation sensor is disposed on the mobile robot and can measure the motion state of the mobile robot in real time. The satellite navigation observation state is the motion state of the mobile robot measured by the satellite navigation sensor at the second moment. It can be understood that based on Table 1 above, the satellite navigation sensor can measure the position, speed, and attitude of the mobile robot. Therefore, the vision observation state measured by the vision sensor can include any one or more of the position, speed, and attitude. The principle of determining the motion state included in the satellite navigation observation state is the same as that of the motion state included in the vision observation state. Just replace the vision observation state with the satellite navigation observation state and the vision sensor with the satellite navigation sensor. Therefore, it will not be elaborated here. Among them, the satellite navigation sensor can also be called a GNSS sensor. The GNSS sensor is a positioning device based on GNSS (Global Navigation Satellite System) technology and can receive signals from multiple satellite systems.

[0178] Next, a specific example will be used to illustrate the process of obtaining the vision observation state and the satellite navigation observation state:

[0179] For vision sensors, they cannot directly measure position, attitude, and velocity. Therefore, position, attitude, and velocity are usually measured indirectly by extracting visual features from images. There are mainly two ways of visual feature extraction, one is ORB feature and the other is KLT optical flow. Constraints between visual frames are established through feature matching. Since there is a corresponding relationship between feature points and map points, the observation of feature points can be achieved through the reprojection of map points, as shown in the following formula (7):

[0180]

[0181] Among them, z i represents the motion state measured by the vision sensor, and Π(.) represents the reprojection function. The normalized feature pixel coordinates can be obtained according to the internal parameters of the camera; n i represents noise; is calculated as shown in the following formula (8),

[0182]

[0183] Among them, and represent the rotation and translation transformation from the camera coordinate system {C} to the inertial coordinate system {I}.

[0184] For satellite navigation sensors, the position can be directly observed through GNSS, and the velocity can be indirectly observed. In a dual-antenna GNSS configuration, the direction angle can be additionally observed, that is, the attitude can be observed. The GNSS position measurement value at the second moment is defined as shown in the following formula (9):

[0185]

[0186] Among them, is calculated through the following formula (10):

[0187]

[0188] Among them, represents the satellite navigation observation state measured by the satellite navigation sensor; and E p V are the external rotation and translation parameters between the {V} coordinate system and the {E} coordinate system, I p G is the external parameter from GNSS to IMU, is a Gaussian white noise with zero mean, is the attitude of the IMU at the second moment in the {V} coordinate system.

[0189] In step S24 above, if the satellite navigation observation state obtained by the electronic device meets the preset available conditions, the inertial prediction state, the visual observation state, and the satellite navigation observation state are fused to obtain the actual motion state of the mobile robot at the second moment.

[0190] Specifically, in some embodiments, the weights can be determined according to the accuracy of the inertial prediction state, the visual observation state, and the satellite navigation observation state respectively, and then weighted summation is performed according to their respective weights. For specific details, please refer to the description below. Figure 3 in.

[0191] Among them, the preset available conditions are the conditions used to determine whether the satellite navigation state can be used for positioning the mobile robot. For example, the preset available conditions can be that the confidence level of the satellite navigation observation state is greater than the preset threshold.

[0192] In step S25 above, if the satellite navigation observation state obtained by the electronic device does not meet the preset available conditions, the inertial prediction state, the visual observation state, and the wheel speedometer observation state are fused to obtain the actual motion state of the mobile robot at the second moment.

[0193] Specifically, in some embodiments, the weights can be determined according to the accuracy of the inertial prediction state, the visual observation state, and the wheel speedometer observation state respectively, and then weighted summation is performed according to their respective weights. For specific details, please refer to the description below. Figure 3 in.

[0194] Among them, the wheel speedometer observation state is the motion state measured by the wheel speedometer sensor provided on the mobile robot at the second moment. Specifically, the wheel speedometer sensor can directly measure the rotation speed of the wheel. For a carrier based on differential steering of two wheels, taking the wheel radius as r as an example, the rotation speed ω of the left wheel can be obtained through the encoder. R , the rotation speed ω of the right wheel L , and the linear velocity v of the carrier in the x-axis direction is obtained. x , the linear velocity v of the carrier w =[v x , 0, 0]. Thus, the speed of the left wheel can be calculated by the following formula (11), and the speed of the right wheel can be in the form shown by the following formula (12):

[0195] v R =ω R r (11)

[0196] Among them, v R represents the speed of the left wheel, ω R represents the rotation speed of the left wheel, and r represents the radius of the wheel.

[0197] v L =ωL r (12)

[0198] Among them, v L represents the speed of the right wheel, ω L represents the rotational speed of the right wheel, and r represents the radius of the wheel.

[0199] Thus, the linear velocity in the x-axis direction (i.e., the speed measured by the wheel speedometer) can be calculated by the following formula (13),

[0200]

[0201] Among them, v x represents the speed measured by the wheel speedometer, v R represents the speed of the left wheel, v L represents the speed of the right wheel.

[0202] It can be understood that both step S24 and step S25 above involve fusing the inertial prediction state with each observation state to obtain the actual motion state of the mobile robot at the current moment. More specifically, it should be based on the inertial prediction state at the current moment and each observation state at the current moment for fusion to obtain the actual motion state of the mobile robot at the current moment. Among them, the inertial prediction state is already the predicted motion state of the mobile robot at the current moment. However, for each observation state, due to the inconsistent data acquisition frequencies of each sensor, it may cause some sensors to fail to collect the observation state at the current moment. For such a situation, the observation state at the moment that the sensor can collect can be used to predict the observation state that the sensor should collect at the current moment. For example, the observation state at the previous moment and the observation state at the next moment collected by the sensor at the current moment are used to predict the observation state that the sensor should collect at the current moment, so as to achieve obtaining the observation states of each sensor at the current moment.

[0203] See Figure 3 , Figure 3 which is a refined schematic diagram provided by the embodiment of the present application for the above step S24 or step S25, and may include the following steps:

[0204] Step S31: Based on the accuracy of each state to be fused respectively, determine the weight of each state, where the weight of the state is positively correlated with the accuracy of the state;

[0205] Step S32: Calculate the residuals between each observation state to be fused and the inertial prediction state respectively, as the residuals corresponding to each observation state;

[0206] Step S33: According to the weights of each observation state respectively, perform weighted summation of the residuals corresponding to each observation state and the inertial prediction state to obtain the fused state.

[0207] In the technical solution provided by the embodiment of the present application, after obtaining the inertial prediction state and various observation states, the weights of the inertial prediction state and each observation state are determined according to the obtained inertial prediction state and the accuracy of each observation state; furthermore, the residuals between each observation state to be fused and the inertial prediction state are calculated respectively as the residuals corresponding to each observation state; then, according to the weights of each observation state, the residuals corresponding to each observation state are weighted and summed with the inertial prediction state to obtain the fusion state.

[0208] In the above step S31, if the obtained satellite navigation observation state meets the preset available conditions, the states to be fused are the inertial prediction state, the visual observation state, and the satellite navigation observation state; if the obtained satellite navigation observation state does not meet the preset available conditions, the states to be fused are the inertial prediction state, the visual observation state, and the wheel speedometer observation state.

[0209] In some embodiments, the electronic device can determine the weights of each observation state through the following formula (14),

[0210]

[0211] where K represents the weight of the observation state to be fused, and H k represents the observation matrix; represents the covariance matrix of the inertial measurement state, and R k represents the covariance matrix of the observation state to be fused.

[0212] And, in some embodiments, the observation matrix H in the above formula (14) k can be represented as a 3×15 matrix. For example, the observation matrix H k can be denoted as When the fusion state includes attitude, the sub-matrix formed by the first to the third columns of H k is the identity matrix; when the fusion state includes position, the sub-matrix formed by the fourth to the sixth columns of the H k is the identity matrix; when the fusion state includes velocity, the sub-matrix formed by the seventh to the ninth columns of the H k [[ID={35]]is the identity matrix.

[0213] In addition, in some embodiments, when the mobile robot is in an application scenario with a lot of slipping, if the wheel speedometer observation state needs to be used when determining the fusion state, the wheel speedometer observation state measured by the wheel speedometer sensor can also be subjected to slipping detection, and the result of the slipping detection is involved in the determination of the weight of the wheel speedometer observation state.

[0214] Moreover, in some embodiments, a wheel speed sensor may also be used to detect whether the mobile robot is in a stationary state. When it is detected that the mobile robot is in a stationary state, zero-speed correction may be performed to further improve the positioning accuracy of the mobile robot, that is, to improve the accuracy of the calculated fusion state. It can be understood that when the wheel speed sensor detects that the speed of the mobile robot is 0, the mobile robot is very likely to be in a stationary state. At this time, if other sensors, such as vision sensors, satellite navigation sensors, and inertial sensors, detect that the mobile robot has a speed, the speed detected by other sensors can be calibrated to suppress error accumulation.

[0215] In some embodiments, the electronic device may determine the accuracy of the inertial prediction state in the following manner. Specifically: The electronic device may obtain the inertial historical state of the inertial sensor at a first moment, where the first moment is the second moment and multiple historical moments before the second moment; then, according to the inertial prediction state and each inertial historical state, the error of the inertial prediction state is determined, where the accuracy of the inertial prediction state is negatively correlated with the prediction error of the inertial prediction state.

[0216] Specifically, in some embodiments, the electronic device may calculate the prediction error of the inertial prediction state through the following formula (15),

[0217] P k+1|k =Φ(t k+1 ,t<* k )P k|k Φ(t k+1 ,t k ) T +Q k (15)

[0218] where P k+1|k represents the prediction error of the inertial prediction state, P k|k represents the covariance matrix of the inertial measurement state, Φ(t k+1 ,t k ) represents the state transition matrix, and Q k represents the noise covariance matrix.

[0219] In the embodiments of the present application, according to the covariance of each state to be fused, the degree of use of the motion state measured by each sensor is determined, that is, the weight of the motion state measured by each sensor, so that the sensor data can be fully utilized.

[0220] In the above step S32, the electronic device calculates the residuals between each observation state to be fused and the inertial prediction state respectively, as the residuals corresponding to each observation state.

[0221] Specifically, for the visual observation state, in some embodiments, the electronic device may acquire a first image and a second image, where the first image is an image measured by the visual sensor at a first moment, and the second image is an image measured by the visual sensor at a second moment; thereafter, a first projection point is determined in the first image, and a second projection point that matches the feature of the first projection point is determined in the second image, where the second projection point that matches the feature of the first projection point is the projection point that is the same point as the first projection point in the actual space, that is, the first projection point and the second projection point are the same point in the actual space; when the first projection point and the second projection point are projected onto the same coordinate system, the distance between the first projection point and the second projection point is calculated; the residual corresponding to the visual observation state is positively correlated with the distance between the first projection point and the second projection point.

[0222] It can be understood that, in some embodiments, the residual corresponding to the visual observation state can guide the optimization direction of the state quantity x and the feature p. Specifically, it can be implemented through the following formula (16):

[0223]

[0224] where represents the residual corresponding to the visual observation state, represents the Jacobian matrix of the state quantity x, represents the Jacobian matrix of the feature p.

[0225] In some embodiments, in order to avoid the computational burden problem caused by too many features, it is also possible to only perform observation updates on the state quantity x. At this time, the above formula (16) can be deformed into the form shown in the following formula (17):

[0226]

[0227] For the satellite navigation observation state, the electronic device may acquire first position data, second position data, and theoretical offset data; where the first position data is the position data measured by the satellite navigation sensor at the second moment, the second position data is the position data measured by the inertial sensor at the second moment, and the theoretical offset data is the relative offset between the position data measured by the satellite navigation sensor and the position data measured by the inertial sensor under theoretical conditions; furthermore, according to the first position data, the second position data, and the theoretical offset data, the deviation degree of the first position relative to the second position is determined; the residual corresponding to the satellite navigation observation state is positively correlated with the deviation degree of the first position relative to the second position.

[0228] where the electronic device can calculate the deviation degree of the first position relative to the second position through the following formula (18):

[0229]

[0230] Among them, represents the deviation degree of the first position relative to the second position, represents the first position data in the visual and inertial fusion coordinate system, represents the second position data in the visual and inertial fusion coordinate system, represents the conversion relationship between the visual and inertial fusion coordinate system and the inertial coordinate system, I p G represents the theoretical offset data between the visual sensor and the inertial sensor in the inertial coordinate system.

[0231] For the wheel speedometer observation state, the electronic device can calculate the difference degree between the first speed and the second speed; among them, the first speed is the speed of the mobile robot measured by the wheel speedometer sensor at the second moment, and the second speed is the speed of the mobile robot measured by the inertial sensor at the second moment; the residual corresponding to the wheel speedometer observation state is positively correlated with the difference degree between the first speed and the second speed.

[0232] Among them, the electronic device can calculate the difference degree between the first speed and the second speed through the following formula (19):

[0233]

[0234] Among them, represents the difference degree between the first speed and the second speed, represents the conversion relationship between the inertial coordinate system and the wheel speedometer coordinate system, represents the first speed in the wheel speedometer coordinate system; represents the second speed in the inertial coordinate system.

[0235] In the above step S33, the electronic device can perform a weighted sum of the residuals corresponding to each observation state and the inertial prediction state according to the weights of each observation state to obtain a fusion state.

[0236] For example, taking as the inertial prediction state, r k as the residuals corresponding to each observation state to be fused, and K as the weights corresponding to each observation state to be fused, then after fusing this observation state with the inertial prediction state, the obtained fusion state can be denoted as And

[0237] In some embodiments, the electronic device may also calculate the covariance of the fusion state to determine the credibility of the fusion state, where the covariance of the fusion state is negatively correlated with the credibility of the fusion state. It is understandable that the negative correlation between the covariance of the fusion state and the credibility of the fusion state described here means that the value of the covariance of the fusion state is negatively correlated with the credibility of the fusion state. That is, the larger the value of the covariance of the fusion state, the less credible the fusion state; the smaller the value of the covariance of the fusion state, the more credible the fusion state. The correlation between the value of the covariance of the fusion state and the value of the credibility of the fusion state depends on the representation form of the value of the credibility of the fusion state. For example, when the credibility of the fusion state is represented by the degree of credibility, the value of the covariance of the fusion state is negatively correlated with the value of the credibility of the fusion state; when the credibility of the fusion state is represented by the degree of incredibility, the value of the covariance of the fusion state is positively correlated with the value of the credibility of the fusion state.

[0238] Specifically, the electronic device may determine the covariance of the fusion state through the following formula (20),

[0239]

[0240] where, represents the covariance of the fusion state.

[0241] It can be understood that in this application, before using the inertial measurement state to predict the inertial prediction state, the electronic device may also perform system initialization. Specifically, the initialization stage mainly completes the initialization of the visual inertial system, determines the initial attitude of the system, estimates the gravity direction and the initial bias of the IMU. This step is only performed for a period of time after the system starts. After the initialization is completed, subsequent positioning can be performed. In addition, the initialization of GNSS will also be performed. For details, please refer to Figure 4 the description in Figure 4 FIG.

[0242] Step S41: Convert the longitude and latitude positioning information in GGA format into the ENU coordinate system and determine the coordinate origin; in this coordinate system, the positive direction of the x-axis is consistent with the true east direction of geography.

[0243] Step S42: Estimate the GNSS speed and attitude.

[0244] Among them, the speed can be calculated based on the position and time difference between the previous and subsequent frames and smoothed through mean filtering. In the dual-antenna scheme, the machine yaw angle can be directly obtained. In the single-antenna scheme, the attitude cannot be directly observed, but the attitude can be obtained by processing multiple position data. For details, please refer to the subsequent Figure 5 description in

[0245] Step S43: Estimate the extrinsic parameters between the {E} system and the {V} system, and align the GNSS pose in the {E} system to the {V} system.

[0246] Among them, the Umeyama method can be used for extrinsic parameter estimation. The selected alignment data length is related to the GNSS positioning quality. For example, in the RTK mode, when obtaining the fixed solution of E4, the alignment data length is 1m, and when obtaining the floating-point solution of E5, the alignment data length is 1.5m.

[0247] Step S44: Align the GNSS data in the {G} system and the IMU pose in the {I} system.

[0248] There is a translational transformation between the two. Since the accuracy of the GNSS estimated pose depends on the motion state of the machine, when the velocity in the x-axis direction is too small, the pose calculation is not stable enough. Therefore, before completing Step S43, the GNSS estimated pose is used, and after initialization, the IMU pose is used.

[0249] Taking the satellite navigation sensor as a single-antenna-based sensor and the mobile robot only having the ability to move in the x-axis direction as an example, refer to Figure 5 , Figure 5 which is a schematic flow diagram for determining the pose of a satellite navigation sensor in the case of a single antenna provided by an embodiment of the present application, and may include the following steps:

[0250] Step S51: Obtain the position data measured by the satellite navigation sensor at the second moment as the third position data; and obtain the position data at at least one moment before the second moment as the fourth position data;

[0251] Step S52: Determine the orientation of the mobile robot at the second moment according to the third position data and each fourth position data.

[0252] In the technical solution provided by the embodiment of the present application, the position data at the second moment and at least one moment before the second moment are obtained. According to the offset between the obtained position data, the motion direction of the mobile robot can be determined, and this motion direction is used as the orientation. Furthermore, according to the position and orientation, the pose of the mobile robot can be determined.

[0253] In the above Step S51, the third position data is the position data of the mobile robot measured by the satellite navigation sensor at the second moment; the fourth position data is the position data of the mobile robot measured by the satellite navigation sensor at at least one moment before the second moment. It can be understood that if the position data at two moments before the second moment is obtained, there are two fourth position data; if the position data at three moments before the second moment is obtained, there are three fourth position data.

[0254] The above step S52: The electronic device can determine the orientation of the movable robot at the second moment according to the third position data and each fourth position data.

[0255] Specifically, in some embodiments, for the third position data and each fourth position data, the electronic device can determine the orientation between adjacent positions as the initial orientation. Then, the initial orientations are weighted and summed according to the weights of the initial orientations, and the orientation of the movable robot at the second moment is obtained; wherein, the weight of the initial orientation is positively correlated with the distance between the adjacent positions to which the initial orientation belongs. That is, an initial orientation can be determined based on two position data, and the greater the distance between these two position data, the greater the weight of the initial orientation determined by these two position data.

[0256] Illustrated with an example, for example, the third position data is position 1, and the fourth position data are position 2 and position 3. Among them, in the movement trajectory of the movable robot, the movable robot passes through position 3, position 2, and position 1 in sequence; then the electronic device can calculate an initial orientation based on position 3 and position 2 as the initial orientation 1, and calculate an initial orientation based on position 2 and position 1 as the initial orientation 2. At the same time, calculate the distance between position 3 and position 2, and the distance between position 2 and position 1. The greater the distance, the greater the corresponding weight.

[0257] Alternatively, in another embodiment, when there are only three position data, the electronic device can use the movement direction determined by the position data at the second moment and the position data at the moment adjacent to the second moment as the orientation of the movable robot at the second moment.

[0258] Specifically, taking an example, for example, the poses at three moments are taken on the GNSS trajectory, such as position A at time t1, position B at time t2, and position C at time t3, where t1 < t2 < t3. At this time, the yaw angle can be calculated according to the direction of the vector BC, that is, the orientation of the movable robot at the second moment. Here, a window width of 3 is selected to calculate the weighted average of the yaw angle, and the weight is the modulus length of the vector BC. It can be understood that in the actual application process, the window width can be adjusted according to the frame rate of the GNSS.

[0259] In some embodiments, the electronic device can also determine the accuracy of the obtained orientation by calculating the collinearity degree of each position data, where the accuracy of the orientation is positively correlated with the collinearity degree of the position data.

[0260] Specifically, when there are three position data (such as the above position A, position B, and position C), the collinearity degree of these three positions can be determined by the following formula (21):

[0261] error = (Cy -A y )(B x -A x )-(B y -A y )(C x -A x ) (21)

[0262] where error represents the collinearity degree; (A x , A y ) represents the position data of position A; (B x , B y ) represents the position data of position B; (C x , C y ) represents the position data of position C.

[0263] Next, through Figure 6 and Figure 7 the specific examples shown, the multi-source fusion positioning method of the mobile robot provided by the embodiments of the present application will be described again.

[0264] Figure 6 is the second flow diagram of the multi-source fusion positioning method of the mobile robot provided by the embodiments of the present application. It can be seen from Figure 6 that the technical solution provided by the embodiments of the present application first needs to perform system initialization. After the initialization is completed, based on the inertial data (i.e., inertial measurement state) measured by the inertial sensor (Inertial Measurement Unit, IMU), the state of the mobile robot at the next moment is predicted by inertia, that is, the inertial prediction state of the mobile robot is obtained; furthermore, based on the states such as position, speed, and attitude measured by other sensors (such as vision sensors, laser sensors, GNSS sensors, odometer sensors, magnetic sensors), the predicted inertial prediction state is observed and updated, and finally the actual motion state of the mobile robot at the next moment is obtained, that is Figure 6 the state quantity output in

[0265] Taking the mobile robot as an intelligent lawn mower as an example, see Figure 7 , Figure 7 is the third flow diagram of the multi-source fusion positioning method of the mobile robot provided by the embodiments of the present application. It can be seen from Figure 7It can be known that after the system initialization, the inertial data measured by the inertial sensor (i.e., the inertial measurement state) is first used to predict the motion state of the mobile robot at the next moment, and the inertial prediction state is obtained. Furthermore, visual measurement, GNSS measurement, and wheel speed measurement are respectively performed through the visual sensor, GNSS sensor, and wheel speed sensor to obtain the observation state. Then, based on the observation state, the inertial prediction state is updated by observation to obtain the actual motion state of the mobile robot at the next moment, that is, the output state quantity is obtained.

[0266] Corresponding to the multi-source fusion positioning method of the above-mentioned mobile robot, the embodiment of the present application further provides a multi-source fusion positioning device for a mobile robot, as Figure 8 shown, the device includes:

[0267] The first acquisition module 81 is used to acquire the inertial measurement state measured by the inertial sensor disposed on the mobile robot at the first moment;

[0268] The prediction module 82 is used to predict the motion state of the mobile robot at the second moment according to the inertial measurement state as the inertial prediction state;

[0269] The second acquisition module 83 is used to acquire the motion state measured by the visual sensor disposed on the mobile robot at the second moment as the visual observation state; and acquire the motion state measured by the satellite navigation sensor disposed on the mobile robot at the second moment as the satellite navigation observation state;

[0270] The fusion module 84 is used to fuse the inertial prediction state, the visual observation state, and the satellite navigation observation state to obtain the fusion state as the actual motion state of the mobile robot at the second moment if the acquired satellite navigation observation state meets the preset available conditions; if the acquired satellite navigation observation state does not meet the preset available conditions, the inertial prediction state, the visual observation state, and the wheel speed observation state are fused to obtain the fusion state as the actual motion state of the mobile robot at the second moment, where the wheel speed observation state is the motion state measured by the wheel speed meter disposed on the mobile robot at the second moment.

[0271] In some embodiments, the states are fused in the following manner, including:

[0272] Based on the accuracy of each state to be fused, the weights of each state are determined, where the weight of the state is positively correlated with the accuracy of the state;

[0273] The residuals between each observation state to be fused and the inertial prediction state are respectively calculated as the residuals corresponding to each observation state;

[0274] The residuals corresponding to each observation state are weighted and summed with the inertial prediction state according to the weights of the respective observation states to obtain a fused state.

[0275] In some embodiments, the accuracy of the inertial prediction state is determined by the following method:

[0276] Obtain the inertial historical state of the inertial sensor at a first moment, where the first moment is a second moment and a plurality of historical moments before the second moment;

[0277] Determine the prediction error of the inertial prediction state according to the inertial prediction state and each inertial historical state; the accuracy of the inertial prediction is negatively correlated with the prediction error.

[0278] In some embodiments, the prediction error is calculated by the following formula:

[0279] P k+1|k =Φ(t k+1 ,t k )P k|k Φ(t k+1 ,t k ) T +Q k

[0280] Wherein, P k+1|k represents the prediction error of the inertial prediction state, P k|k represents the covariance matrix of the inertial measurement state, Φ(t k+1 ,t k ) represents the state transition matrix, and Q k represents the noise covariance matrix.

[0281] In some embodiments, the residual corresponding to the visual observation state is determined by the following method:

[0282] Obtain a first image and a second image, where the first image is an image measured by the visual sensor at a first moment, and the second image is an image measured by the visual sensor at a second moment;

[0283] Determine a first projection point in the first image, and determine a second projection point in the second image that is feature-matched with the first projection point;

[0284] When the first projection point and the second projection point are projected onto the same coordinate system, calculate the distance between the first projection point and the second projection point; the residual corresponding to the visual observation state is positively correlated with the distance between the first projection point and the second projection point.

[0285] In some embodiments, the residual corresponding to the satellite navigation observation state is determined by the following method, including:

[0286] Obtain the first position data, the second position data, and the theoretical offset data; wherein, the first position data is the position data measured by the satellite navigation sensor at the second moment, the second position data is the position data measured by the inertial sensor at the second moment, and the theoretical offset data is the relative offset between the position data measured by the satellite navigation sensor and the position data measured by the inertial sensor under theoretical circumstances;

[0287] Determine the deviation degree of the first position relative to the second position according to the first position data, the second position data, and the theoretical offset data; the residual corresponding to the satellite navigation observation state is positively correlated with the deviation degree of the first position relative to the second position.

[0288] In some embodiments, the deviation degree of the first position relative to the second position is calculated by the following formula:

[0289]

[0290] Wherein, represents the deviation degree of the first position relative to the second position, represents the first position data in the visual and inertial fusion coordinate system, represents the second position data in the visual and inertial fusion coordinate system, represents the conversion relationship between the visual and inertial fusion coordinate system and the inertial coordinate system, I p G represents the theoretical offset data between the visual sensor and the inertial sensor in the inertial coordinate system.

[0291] In some embodiments, the residual corresponding to the wheel speedometer observation state is determined by the following method:

[0292] Calculate the difference degree between the first speed and the second speed; wherein, the first speed is the speed of the mobile robot measured by the wheel speedometer sensor at the second moment, and the second speed is the speed of the mobile robot measured by the inertial sensor at the second moment; the residual corresponding to the wheel speedometer observation state is positively correlated with the difference degree between the first speed and the second speed.

[0293] In some embodiments, the difference degree between the first speed and the second speed is calculated by the following formula:

[0294]

[0295] Wherein, represents the difference degree between the first speed and the second speed, represents the conversion relationship between the inertial coordinate system and the wheel speedometer coordinate system, represents the first speed in the wheel speedometer coordinate system; represents the second speed in the inertial coordinate system.

[0296] In some embodiments, the weights of each state are determined by the following formula:

[0297]

[0298] where K represents the weight of the observation state to be fused, and H k represents the observation matrix, represents the covariance matrix of the inertial measurement state to be determined, and R k represents the covariance matrix of the observation state to be fused.

[0299] In some embodiments, H k is a 3×15 matrix; when the fused state includes attitude, the sub-matrix formed by the first to third columns of H k is the identity matrix; when the fused state includes position, the sub-matrix formed by the fourth to sixth columns of H k is the identity matrix; when the fused state includes velocity, the sub-matrix formed by the seventh to ninth columns of H k is the identity matrix.

[0300] In some embodiments, the satellite navigation sensor is a single-antenna-based sensor, and the device further includes:

[0301] A third acquisition module, configured to acquire the position data measured by the satellite navigation sensor at the second moment as the third position data; and acquire the position data at at least one moment before the second moment as the fourth position data;

[0302] An orientation determination module, configured to determine the orientation of the mobile robot at the second moment according to the third position data and each of the fourth position data.

[0303] In some embodiments, the orientation determination module is specifically configured to:

[0304] For the third position data and each of the fourth position data, determine the orientation of each adjacent position as the initial orientation;

[0305] Perform a weighted sum of each initial orientation according to the weight of each initial orientation to obtain the orientation of the mobile robot at the second moment; wherein, the weight of the initial orientation is positively correlated with the distance between the adjacent positions to which the initial orientation belongs.

[0306] In some embodiments, the inertial measurement state measured by the inertial sensor includes any one or more of the following states: position, attitude, velocity;

[0307] The visual measurement state measured by the visual sensor includes any one or more of the following states: position, attitude, velocity;

[0308] The satellite navigation observation states measured by the satellite navigation sensor include any one or more of the following states: position, attitude, and velocity;

[0309] The wheel speed sensor measurement of the wheel speedometer observation state includes: speed.

[0310] Corresponding to the above multi-source fusion positioning method of the mobile robot, an embodiment of the present application further provides an outdoor robot, which is provided with an inertial sensor, a satellite navigation sensor, a vision sensor, and a wheel speed sensor; the outdoor robot is used to implement any one of the above multi-source fusion positioning methods of the mobile robot.

[0311] And, corresponding to the above multi-source fusion positioning method of the mobile robot, an embodiment of the present application further provides a lawn mowing robot, on which an inertial sensor, a satellite navigation sensor, a vision sensor, and a wheel speed sensor are provided; the lawn mowing robot is used to implement any one of the above multi-source fusion positioning methods of the mobile robot.

[0312] An embodiment of the present application further provides an electronic device, as Figure 9 shown, including:

[0313] A memory 91 for storing a computer program;

[0314] A processor 92, when executing the program stored on the memory 91, implements any one of the above multi-source fusion positioning methods of the mobile robot.

[0315] And the above electronic device may further include a communication bus and / or a communication interface, and the processor 92, the communication interface, and the memory 91 complete communication with each other through the communication bus.

[0316] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0317] The communication interface is used for communication between the above electronic device and other devices.

[0318] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0319] The aforementioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0320] In another embodiment provided by the present application, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above multi-source fusion positioning methods for a mobile robot are implemented.

[0321] In another embodiment provided by the present application, a computer program product containing instructions is also provided. When it runs on a computer, it causes the computer to execute any of the multi-source fusion positioning methods for a mobile robot in the above embodiments.

[0322] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a solid state disk (SSD), etc.

[0323] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device including the element.

[0324] Each embodiment in this specification is described in a related manner. The same or similar parts among the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the device, mobile robot, lawn mower, electronic device, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0325] The above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. A multi-source fusion positioning method for a mobile robot, characterized in that The method includes: Obtaining an inertial measurement state measured by an inertial sensor disposed on a mobile robot at a first moment; Predicting, according to the inertial measurement state, a motion state of the mobile robot at a second moment as an inertial prediction state; Obtaining a motion state measured by a vision sensor disposed on the mobile robot at the second moment as a vision observation state; and obtaining a motion state measured by a satellite navigation sensor disposed on the mobile robot at the second moment as a satellite navigation observation state; If the obtained satellite navigation observation state meets a preset available condition, fusing the inertial prediction state, the vision observation state, and the satellite navigation observation state to obtain a fusion state as an actual motion state of the mobile robot at the second moment; If the obtained satellite navigation observation state does not meet the preset available condition, fusing the inertial prediction state, the vision observation state, and a wheel speedometer observation state to obtain the fusion state as an actual motion state of the mobile robot at the second moment, where the wheel speedometer observation state is a motion state measured by a wheel speedometer disposed on the mobile robot at the second moment.

2. The method according to claim 1, wherein The states are fused in the following manner, including: Respectively determining weights of the states based on accuracies of the states to be fused, where the weight of a state is positively correlated with the accuracy of the state; Respectively calculating residuals between each observation state to be fused and the inertial prediction state as residuals corresponding to each observation state; Performing weighted summation of the residuals corresponding to each observation state and the inertial prediction state according to the weights of each observation state to obtain the fusion state.

3. The method according to claim 2, wherein The accuracy of the inertial prediction state is determined in the following manner: Obtaining an inertial historical state of the inertial sensor at the first moment, where the first moment is the second moment and multiple historical moments before the second moment; Determining a prediction error of the inertial prediction state according to the inertial prediction state and each of the inertial historical states; the accuracy of the inertial prediction is negatively correlated with the prediction error.

4. The method according to claim 3, characterized in that, The prediction error is calculated by the following formula: P k+1|k = Φ(t k+1 , t k )P k|k Φ(t k+1 , t k ) T + Q k where, P k+1|k represents the prediction error of the inertial prediction state, P k|k represents the covariance matrix of the inertial measurement state, Φ(t k+1 , t k ) represents the state transition matrix, Q k represents the noise covariance matrix.

5. The method according to claim 2, wherein The residual corresponding to the vision observation state is determined in the following manner: Obtaining a first image and a second image, where the first image is an image measured by the vision sensor at the first moment, and the second image is an image measured by the vision sensor at the second moment; Determining a first projection point in the first image, and determining a second projection point that is feature-matched with the first projection point in the second image; Calculating a distance between the first projection point and the second projection point in a case where the first projection point and the second projection point are projected onto the same coordinate system; the residual corresponding to the vision observation state is positively correlated with the distance between the first projection point and the second projection point.

6. The method according to claim 2, wherein The residual corresponding to the satellite navigation observation state is determined in the following manner, including: Obtain the first position data, the second position data, and the theoretical offset data; wherein, the first position data is the position data measured by the satellite navigation sensor at the second moment, the second position data is the position data measured by the inertial sensor at the second moment, and the theoretical offset data is the relative offset between the position data measured by the satellite navigation sensor and the position data measured by the inertial sensor under theoretical circumstances; Determine the deviation degree of the first position relative to the second position according to the first position data, the second position data, and the theoretical offset data; the residual corresponding to the satellite navigation observation state is positively correlated with the deviation degree of the first position relative to the second position.

7. The method according to claim 6, characterized in that, Calculate the deviation degree of the first position relative to the second position through the following formula: Among them, represents the deviation degree of the first position relative to the second position, represents the first position data in the visual and inertial fusion coordinate system, represents the second position data in the visual and inertial fusion coordinate system, represents the conversion relationship between the visual and inertial fusion coordinate system and the inertial coordinate system, I p G represents the theoretical offset data between the visual sensor and the inertial sensor in the inertial coordinate system.

8. The method according to claim 2, characterized in that Determine the residual corresponding to the wheel speedometer observation state in the following manner: Calculate the difference degree between the first speed and the second speed; wherein, the first speed is the speed of the mobile robot measured by the wheel speedometer sensor at the second moment, and the second speed is the speed of the mobile robot measured by the inertial sensor at the second moment; the residual corresponding to the wheel speedometer observation state is positively correlated with the difference degree between the first speed and the second speed.

9. The method according to claim 8, characterized in that, Calculate the difference degree between the first speed and the second speed through the following formula: wherein, represents the degree of difference between the first speed and the second speed, represents the conversion relationship between the inertial coordinate system and the wheel speedometer coordinate system, represents the first speed in the wheel speedometer coordinate system; represents the second speed in the inertial coordinate system.

10. The method according to claim 2, wherein Determine the weights of each state through the following formula, where K represents the weight of the observation state to be fused, and H k represents the observation matrix, represents the covariance matrix of the inertial measurement state, and R k represents the covariance matrix of the observation state to be fused.

11. The method according to claim 10, wherein The said H k is a 3×15 matrix; When the fusion state includes attitude, the sub-matrix formed by the 1st to 3rd columns of H k is an identity matrix; When the fusion state includes a position, the H k sub-matrix formed by columns 4 to 6 is an identity matrix; When the fusion state includes speed, the H k sub-matrix formed by the 7th to 9th columns is an identity matrix.

12. The method according to claim 1, wherein The satellite navigation sensor is a single-antenna-based sensor, and the method further includes: Obtain the position data measured by the satellite navigation sensor at the second moment as the third position data; and obtain the position data at at least one moment before the second moment as the fourth position data; Determine the orientation of the mobile robot at the second moment according to the third position data and each of the fourth position data.

13. The method according to claim 12, wherein The determining the orientation of the mobile robot at the second moment according to the third position data and each of the fourth position data includes: For the third position data and each of the fourth position data, determine the orientation of each adjacent position as the initial orientation; Perform weighted summation on each of the initial orientations according to the weights of the initial orientations to obtain the orientation of the mobile robot at the second moment; wherein, the weight of the initial orientation is positively correlated with the distance between the adjacent positions to which the initial orientation belongs.

14. The method according to claim 1, characterized in that, The inertial measurement state measured by the inertial sensor includes any one or more of the following states: position, attitude, speed; The visual measurement state measured by the visual sensor includes any one or more of the following states: position, attitude, speed; The satellite navigation observation state measured by the satellite navigation sensor includes any one or more of the following states: position, attitude, speed; The wheel speedometer observation state measured by the wheel speedometer sensor includes: speed.

15. An outdoor robot, characterized in that, An inertial sensor, a satellite navigation sensor, a visual sensor, and a wheel speedometer sensor are provided on the outdoor robot; the outdoor robot is used to execute the method according to any one of claims 1-14 above.

16. A lawn mowing robot, characterized in that, An inertial sensor, a satellite navigation sensor, a vision sensor, and a wheel speed sensor are provided on the lawn mower; the lawn mowing robot is used to execute the method according to any one of claims 1-14 above.

Images