Pose adjustment method of mobile robot and mobile robot

Abstract

The embodiment of the present application provides a pose adjustment method of a mobile robot and the mobile robot. In the embodiment, an image acquisition component on the mobile robot acquires an image, such as a monocular camera sensor acquiring a monocular image, to obtain a key frame (denoted as the ith key frame). Then, a pose estimation function (which is related to velocity, angle and position) is established based on a visual pose deviation and an inertial navigation pose deviation corresponding to the ith key frame. Finally, a fusion pose at each subsequent time is predicted through the pose estimation function, and parameters related to the pose of the mobile robot, such as bias of an IMU, PI compensation coefficient and a sensor external parameter matrix such as T c , T b , are adjusted based on the fusion pose, so as to eliminate the cumulative error caused by long-time integration of an inertial navigation principle such as a gyroscope, and improve the accuracy of the pose of the robot.

Description

Technical Field

[0001] This application relates to mobile robots, and more particularly to a method for adjusting the pose of a mobile robot and the mobile robot itself. Background Technology

[0002] Pose, in this context, refers to the combined effects of position and orientation. Currently, pose estimation is often based on inertial measurement units (IMUs) such as wheel speedometers and gyroscopes. However, the biggest technical challenge in pose estimation using inertial sensors is the accumulation of error. This accumulation is caused by the principles of inertial navigation. For example, inertial navigation requires that position calculations not only use acceleration but also attitude. This process couples the errors from acceleration components (such as accelerometers, specifically one type of inertial sensor) and attitude components (such as gyroscopes, specifically another type of inertial sensor), which accumulate sequentially, resulting in a significant cumulative error.

[0003] The aforementioned cumulative errors often prevent mobile robots from accurately adjusting their pose during movement. Summary of the Invention

[0004] This application provides a method for adjusting the pose of a mobile robot and a mobile robot, so as to adjust the pose of the mobile robot in a timely manner during the movement of the mobile robot and eliminate the cumulative error caused by inertial navigation estimation.

[0005] This application provides a method for adjusting the pose of a mobile robot, the method comprising:

[0006] Obtain the visual pose deviation and inertial navigation pose deviation corresponding to the i-th keyframe; wherein, the i-th keyframe is generated based on at least two adjacent images acquired by the image acquisition component on the mobile robot during the robot's movement; the visual pose deviation refers to the deviation between the fused pose and the visual pose corresponding to the i-th keyframe; the visual pose is obtained by pose estimation based on the at least two adjacent images; the fused pose is obtained by fusing the visual pose and the inertial navigation pose corresponding to the i-th keyframe; the inertial navigation pose deviation refers to the deviation between the fused pose and the inertial navigation pose.

[0007] Based on the visual pose deviation corresponding to each keyframe, the inertial navigation pose deviation corresponding to each keyframe, and the determined visual pose weight coefficients and inertial navigation pose weight coefficients, a pose prediction function is determined; the pose prediction function is used to predict the fused pose at subsequent time steps, and the current pose of the mobile robot is adjusted based on the predicted fused pose.

[0008] This application embodiment also provides a mobile robot, which includes:

[0009] Logic processing components, image acquisition components, and inertial measurement units (IMUs);

[0010] The image acquisition component is used to acquire images during the operation of the mobile robot; at least two adjacent images acquired by the image acquisition component are used to create corresponding keyframes; the at least two adjacent images are also used to estimate visual pose.

[0011] An IMU is used to measure IMU data during the operation of a mobile robot; the IMU data is used to estimate the inertial navigation pose.

[0012] The logic operation component is used to perform the steps in the method described above.

[0013] As can be seen from the above technical solutions, the embodiments of this application acquire images through image acquisition components on the mobile robot, such as monocular camera sensors, to obtain a key frame (denoted as the i-th key frame). Then, based on the visual pose deviation and inertial navigation pose deviation corresponding to the i-th key frame, a pose prediction function (which is related to velocity, angle, and position) is established. Finally, the pose prediction function is used to predict the fused pose at subsequent times, and the local pose-related parameters of the mobile robot, such as the IMU bias, PI compensation coefficient, and sensor extrinsic parameter matrix, are adjusted based on the fused pose to eliminate the cumulative error caused by long-term integration of inertial navigation principles such as gyroscopes, thereby improving the accuracy of robot pose. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0015] Figure 1 This is a structural diagram of the mobile robot provided in an embodiment of this application;

[0016] Figure 2 This is an application structure diagram of the mobile robot provided in the embodiments of this application;

[0017] Figure 3 A flowchart illustrating the method provided in this application embodiment;

[0018] Figure 4 A flowchart illustrating the implementation of step 301 provided in this application embodiment;

[0019] Figure 5 The flowchart for implementing step 302 provided in the embodiments of this application is shown. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0022] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0023] The mobile robot provided in the embodiments of this application will be described below.

[0024] See Figure 1 , Figure 1 This is a structural diagram of the mobile robot provided in an embodiment of this application. Figure 1 As shown, the mobile robot may include: an image acquisition component, an IMU (Inertial Measurement Unit), a linear Hall sensor, and a logic operation component.

[0025] As an example, the image acquisition component described above may be a monocular camera sensor, which is positioned between the two wheels of the mobile robot (e.g., in the middle) and is substantially parallel to the horizontal plane.

[0026] As one embodiment, the aforementioned IMU sensor can be set at a designated location on the mobile robot, such as the center of the entire mobile robot, or the left rear position of the center of the entire mobile robot, etc. This embodiment is not specifically limited.

[0027] As an example, the aforementioned linear Hall sensor can be positioned at the center of the left and right wheels of the mobile robot, and is used to measure the displacement information (such as mileage) of the two wheels in the mobile robot.

[0028] As one embodiment, the above-mentioned logic operation component can be set in the processor of the mobile robot, or it can be set in other specified locations. This embodiment is not specifically limited.

[0029] Based on the above description, Figure 2An example is shown in the specific structural diagram of a mobile robot.

[0030] The method provided in the embodiments of this application is described below:

[0031] See Figure 3 , Figure 3 This is a flowchart illustrating a method provided in an embodiment of this application. This process can be applied to the aforementioned mobile robot. Figure 3 As shown, the process may include the following steps:

[0032] Step 301: Obtain the visual pose deviation and inertial navigation pose deviation corresponding to the i-th keyframe.

[0033] Here, i is a positive integer greater than or equal to 1. This is just a name used for ease of description and is not intended to be a limitation.

[0034] In this embodiment, images (e.g., monocular images) can be acquired at different time points using a camera sensor, such as the aforementioned monocular camera sensor. Keyframes are created according to the monitoring point order and based on images acquired at at least two adjacent time points (referred to as at least two adjacent images). Taking the creation of a keyframe using two adjacent images as an example, initially, the images are sorted according to the acquisition order. Then, the first two images in the sequence (i.e., two adjacent images) are selected to generate a keyframe, which can be recorded as the first keyframe (the selected image can be referred to as the image involved in the keyframe). Then, the third and fourth images in the sequence (i.e., two adjacent images) are selected to generate a keyframe, which can be recorded as the second keyframe, and so on, ultimately resulting in multiple keyframes. Here, a keyframe refers to an image containing a specified key location.

[0035] In this embodiment, the visual pose deviation corresponding to the i-th keyframe will be described in detail below, and will not be repeated here. Similarly, in this embodiment, the inertial navigation pose deviation corresponding to the i-th keyframe will also be described with examples below, and will not be repeated here.

[0036] Step 302: Determine the pose prediction function based on the visual pose deviation corresponding to each key frame, the inertial navigation pose deviation corresponding to each key frame, and the determined visual pose weight coefficient and inertial navigation pose weight coefficient; the pose prediction function is used to predict the fused pose at subsequent time steps, and adjust the current pose of the mobile robot based on the predicted fused pose.

[0037] In this embodiment, the visual pose weight coefficient and the inertial navigation pose weight coefficient can be preset or dynamically determined. The dynamic determination will be described below and will not be repeated here.

[0038] In this embodiment, the pose prediction function is related to velocity, position, and angle. It can adjust the pose of the mobile robot from three dimensions: velocity, position, and angle, respectively, to correct the deviation of the mobile robot's pose. This will be described below and will not be repeated here.

[0039] pass Figure 3 As shown in the flowchart, this embodiment acquires images using an image acquisition component on the mobile robot, such as a monocular camera sensor, to obtain a keyframe (denoted as the i-th keyframe). Then, based on the visual pose deviation and inertial navigation pose deviation corresponding to the i-th keyframe, a pose prediction function (related to velocity, angle, and position) is established. Finally, the pose prediction function is used to predict the fused pose at subsequent time points, and based on the fused pose, the local pose-related parameters of the mobile robot, such as the IMU bias, PI compensation coefficient, and sensor extrinsic matrices, such as T, are adjusted. c T b This is to eliminate the accumulated errors caused by long-term integration in inertial navigation principles, such as gyroscopes, and improve the accuracy of robot pose. The extrinsic parameter matrix T here... c The extrinsic parameter matrix T is used to transform between the coordinate system of the image acquisition component on the mobile robot and the coordinate system of the mobile robot. b Used to convert between the coordinate system corresponding to the IMU on the mobile robot and the coordinate system corresponding to the mobile robot.

[0040] Furthermore, this embodiment is relatively simple to implement and can be quickly applied to various indoor mobile robot products. It has the characteristics of strong adaptability, low cost, small amount of code, and fast deployment, and has a wide range of application scenarios.

[0041] The following example illustrates how to obtain the visual pose error and inertial navigation pose error corresponding to the i-th keyframe in step 301:

[0042] See Figure 4 , Figure 4 This is a flowchart illustrating the implementation of step 301 in an embodiment of this application. Figure 4 As shown, the process may include:

[0043] Step 401: Determine the visual pose corresponding to the i-th keyframe based on the images of each frame involved in determining the i-th keyframe.

[0044] As described above, the frames involved in determining the i-th keyframe can be at least two adjacent images used in determining the i-th keyframe.

[0045] Optionally, in this embodiment, the pose of the camera sensor, such as a monocular sensor (also known as the visual pose, denoted as T), can be estimated by performing feature matching on each frame of images involved in the i-th keyframe. ic For specific implementation, please refer to the pose estimation algorithm based on feature matching. This embodiment does not provide a specific and limited description.

[0046] Step 402: Determine the inertial navigation pose corresponding to the i-th keyframe based on the displacement of the mobile robot and the measurement data of the IMU.

[0047] As described above, this embodiment deploys linear Hall sensors at designated locations on the mobile robot, such as the left and right wheels, to estimate the robot's displacement (e.g., the displacement of the left and right wheels) based on the data collected by the linear Hall sensors. Then, the robot's displacement is synchronized with the IMU's measurement data, and complementary filtering is used to perform inertial navigation estimation to obtain the inertial navigation pose corresponding to the i-th keyframe. As for how to obtain the inertial navigation pose, conventional inertial navigation estimation algorithms can be referenced, and this embodiment is not specifically limited to them.

[0048] Optionally, the displacement of the mobile robot can be the displacement of the mobile robot within a time period. This time period consists of the time point when the image frame involved in the i-th keyframe is acquired. Similarly, the measurement data of the IMU can also be the data measured by the IMU within the aforementioned time period.

[0049] As an example, the inertial navigation pose in this embodiment can be decomposed into position (denoted as p). b ) and attitude (denoted as q) b As an example, position p b and posture q b It can be expressed by the following formula:

[0050] q b (t+Δt)=q b (t)+h*f(t,q b (t)) (Formula 1)

[0051] p b =p b_init +R*Δp (Formula 2)

[0052] In Formula 1, q b (t+Δt) represents the attitude at time point t+Δt, q b f(t) represents the attitude at time t, h represents the preset differential coefficient, and f(t,q) represents the attitude at time t. b (t) represents the change. In Equation 2, Δp represents the displacement change (which can be indicated by one of the inertial sensors, such as the wheel speedometer), and R represents the change in the aforementioned attitude.

[0053] Step 403: The above visual pose and the above inertial pose are fused to obtain the fused pose corresponding to the i-th keyframe.

[0054] Optionally, in this embodiment, before fusing the above-mentioned visual pose and the above-mentioned inertial navigation pose, the extrinsic parameters of the above-mentioned camera sensor and IMU are first determined.

[0055] In practical implementation, there are many ways to determine the extrinsic parameters of the IMU and the aforementioned camera sensor. For example, in this embodiment, the robot coordinate system O is established with the structural center position O of the entire mobile robot as the reference point. r Initially, the robot's coordinate system and the world coordinate system O... w They coincide. Additionally, a camera coordinate system O is established with the center of the camera sensor as the reference point. c Establish the IMU coordinate system O with the center position of the IMU. b Assume the camera sensor and IMU are respectively connected to the robot's center O. r The measurement extrinsic matrix is ​​T. c T b Then the extrinsic parameter transformation of the camera sensor and IMU can be obtained as follows:

[0056] O r =T c *O c =T b *O b ;

[0057] Based on the extrinsic parameter transformation of the camera sensor and IMU, and utilizing the principle of Local Bundle Adjustment (BA), the optimization nodes of the visual pose and inertial navigation pose construction graph can be fused and optimized to obtain the aforementioned fused pose. Here, the fused pose can be expressed by the following formula: T ir =G(T) ic ,T ib ). (Formula 3)

[0058] It should be noted that, in this embodiment, the fused pose in Formula 3 may specifically involve updating the angular velocity offset and proportional-integral (PI) compensation method of the mobile robot as follows:

[0059]

[0060] In Formula 4, gyro′ represents the compensated angular velocity, gyro represents the measured original angular velocity, and gyro bias and a bias These represent the biases for angular velocity and linear acceleration, respectively. PI1 and PI2 are the PI compensation coefficients. initRepresents the initial state quantity, typically 0, accel error This represents the compensated linear acceleration. For the measured linear acceleration, This is the theoretical acceleration after the gravitational acceleration is transformed into the robot coordinate system.

[0061] Step 404: Based on the deviation between the fused pose and the visual pose, determine the visual pose deviation corresponding to the i-th keyframe, and based on the deviation between the fused pose and the inertial navigation pose, determine the inertial navigation pose deviation corresponding to the i-th keyframe.

[0062] Based on the above-mentioned fused pose, the visual pose deviation and the inertial navigation pose deviation can be expressed by the following formula:

[0063]

[0064] Where, ΔT ic Indicates visual pose deviation, ΔT ib Indicates the inertial navigation posture deviation.

[0065] This concludes the process. Figure 4 The description.

[0066] pass Figure 4 The process shown achieves the acquisition of the visual pose deviation and inertial navigation pose deviation corresponding to the i-th keyframe in step 301 above.

[0067] The following example illustrates how, in step 302, the pose prediction function is determined based on the visual pose deviation corresponding to each keyframe, the inertial navigation pose deviation corresponding to each keyframe, and the determined visual pose weight coefficients and inertial navigation pose weight coefficients.

[0068] See Figure 5 , Figure 5 This is a flowchart illustrating the implementation of step 302 in an embodiment of this application. Figure 5 As shown, the process may include the following steps:

[0069] Step 501: Based on the visual pose deviation corresponding to each key frame, determine the visual pose velocity change of adjacent key frames, and determine the visual pose error function based on the visual pose velocity change of adjacent key frames.

[0070] As an example, the visual pose velocity change (SP) of any adjacent keyframe (e.g., the i-th keyframe and the (i-1)-th keyframe) ic This can be expressed by the following formula:

[0071]

[0072] Among them, v icw represents the change in linear velocity between adjacent keyframes in the camera coordinate system, such as the i-th keyframe and the (i-1)-th keyframe. ic It represents the change in angular velocity between adjacent keyframes in the camera coordinate system, such as the i-th keyframe and the (i-1)-th keyframe.

[0073] After determining the visual pose velocity changes of each adjacent keyframe, the visual pose error function (H(t,ΔT)) can be determined based on these changes. tc )).

[0074] Step 502: Based on the inertial navigation pose deviation corresponding to each key frame, determine the inertial navigation pose velocity change of adjacent key frames, and determine the inertial navigation pose error function based on the inertial navigation pose velocity change of adjacent key frames.

[0075] As an example, the inertial navigation pose velocity change (SP) of any adjacent keyframe (e.g., the i-th keyframe and the (i-1)-th keyframe) ib This can be expressed by the following formula:

[0076]

[0077] Among them, v ib This represents the change in linear velocity between adjacent keyframes in the IMU coordinate system, such as the i-th keyframe and the (i-1)-th keyframe. ib This indicates the change in angular velocity between adjacent keyframes in the IMU coordinate system, such as the i-th keyframe and the (i-1)-th keyframe.

[0078] After determining the inertial navigation pose velocity changes in each adjacent keyframe, the inertial navigation pose error function (H(t,ΔT)) can be determined based on these changes. tb )).

[0079] Step 503: Determine the pose prediction function based on the visual pose error function, the inertial navigation pose error function, and the determined visual pose weight coefficients and inertial navigation pose weight coefficients.

[0080] As an example, the pose prediction function is expressed by the following formula:

[0081]

[0082] Among them, T′ (t+1)r G(T) represents the fused pose that needs to be predicted at the next time step t+1. tc ,T tb H(t,ΔT) represents the known fused pose at the most recent time t. tc F(t,ΔT) represents the visual pose error function.tb W(t,T) represents the inertial navigation pose error function. tc ,T tb The symbols () represent the weighting coefficients of the visual pose and the inertial navigation pose. In this embodiment, the weighting coefficient of the visual pose represents the contribution of the visual pose to the fused pose, and the weighting coefficient of the inertial navigation pose represents the contribution of the inertial navigation pose to the fused pose. The weighting coefficients of the visual pose and the inertial navigation pose can be preset or dynamically determined based on the obtained fused poses and the proportions of the visual pose and the inertial navigation pose in the fused pose. This embodiment is not specifically limited.

[0083] It should be noted that, in this embodiment, the velocity change SP of the fusion pose of adjacent keyframes can also be determined based on the fusion pose corresponding to the i-th keyframe and the fusion pose corresponding to the (i-1)-th keyframe. ir :

[0084]

[0085] In Formula 9, ΔT ir This represents the change in fused pose between adjacent keyframes (e.g., the change in fused pose between the i-th keyframe and the (i-1)-th keyframe). Where v ir w represents the change in linear velocity between adjacent keyframes in the robot coordinate system, such as the i-th keyframe and the (i-1)-th keyframe. rr This indicates the change in angular velocity between adjacent keyframes in the robot coordinate system, such as the i-th keyframe and the (i-1)-th keyframe.

[0086] In specific implementation, the pose prediction function can also be determined based on the fusion pose velocity change determined by each adjacent keyframe. This embodiment is not specifically limited to this.

[0087] This concludes the process. Figure 5 The process is shown below.

[0088] pass Figure 5 The process shown realizes how to determine the pose prediction function in step 302 above, based on the visual pose deviation corresponding to each key frame, the inertial navigation pose deviation corresponding to each key frame, and the determined visual pose weight coefficient and inertial navigation pose weight coefficient.

[0089] As described in Formula 4 above, in this embodiment, any fused pose and the mobile robot's inertial sensor IMU bias, compensation coefficient PI, and extrinsic parameter matrix T c T bAccordingly, in this embodiment, after predicting the fused pose, the current pose of the mobile robot can be adjusted based on the predicted fused pose, such as adjusting the mobile robot's IMU bias, compensation coefficient PI, and extrinsic parameter matrix T. c T b At least one of them, to correct the current pose of the mobile robot.

[0090] As described above, the aforementioned parameter matrix T c The extrinsic parameter matrix T is used to transform the coordinate system between the coordinate system of an image acquisition component on a mobile robot, such as a monocular camera sensor, and the coordinate system of the mobile robot. b This is used to convert between the coordinate system corresponding to the IMU on the mobile robot and the coordinate system corresponding to the mobile robot.

[0091] As can be seen from the above description, this embodiment constructs a pose prediction function (also known as a local error model) by fusing individual poses in the early stage (such as visual pose estimated based on data collected by a monocular camera sensor, inertial navigation pose estimated based on data measured by an IMU and data measured by a linear Hall sensor) and comparing the fused pose with each individual pose. Based on the pose prediction function (also known as the local error model), the fused pose is predicted, and the local pose-related parameters of the mobile robot, such as the IMU bias, PI compensation coefficient, and sensor extrinsic matrix, are dynamically adjusted based on the predicted fused pose to obtain locally optimal parameters, thereby eliminating the cumulative error generated during inertial navigation pose estimation and improving the fused robot pose.

[0092] The methods provided in the embodiments of this application have been described above. The machine-readable storage medium provided in the embodiments of this application is described below:

[0093] Correspondingly, in this embodiment, the present application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed, can implement the methods disclosed in the above examples of the present application.

[0094] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0095] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0096] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0097] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0099] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0100] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0101] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for adjusting the pose of a mobile robot, characterized in that, The method includes: Obtain the visual pose deviation and inertial navigation pose deviation corresponding to the i-th keyframe; wherein, the i-th keyframe is generated based on at least two adjacent images acquired by the image acquisition component on the mobile robot during the robot's movement; the visual pose deviation refers to the deviation between the fused pose and the visual pose corresponding to the i-th keyframe; the visual pose is obtained by pose estimation based on the at least two adjacent images; the fused pose is obtained by fusing the visual pose and the inertial navigation pose corresponding to the i-th keyframe; the inertial navigation pose deviation refers to the deviation between the fused pose and the inertial navigation pose. Based on the visual pose deviation corresponding to each keyframe, the inertial navigation pose deviation corresponding to each keyframe, and the determined visual pose weight coefficients and inertial navigation pose weight coefficients, a pose prediction function is determined; the pose prediction function is used to predict the fused pose at subsequent time steps, and the current pose of the mobile robot is adjusted based on the predicted fused pose.

2. The method according to claim 1, characterized in that, The inertial navigation pose is determined based on the displacement of the mobile robot and the measurement data from the inertial sensor IMU. A linear Hall sensor is deployed at the center of each of the left and right wheels of the mobile robot; The displacement of the mobile robot is estimated by data measured by a linear Hall sensor over a time period, which is composed of the acquisition time points of at least two adjacent images acquired by the image acquisition component; the measurement data of the inertial sensor (IMU) refers to the data measured by the IMU within the time period.

3. The method according to claim 1, characterized in that, The determination of the pose prediction function based on the visual pose deviation corresponding to each keyframe, the inertial navigation pose deviation corresponding to each keyframe, and the determined visual pose weight coefficients and inertial navigation pose weight coefficients includes: Based on the visual pose deviation corresponding to each key frame, determine the visual pose velocity change of adjacent key frames; based on the visual pose velocity change of adjacent key frames, determine the visual pose error function. Based on the inertial navigation pose deviation corresponding to each key frame, determine the inertial navigation pose velocity change corresponding to adjacent key frames; based on the inertial navigation pose velocity change of adjacent key frames, determine the inertial navigation pose error function. Based on the visual pose error function, the inertial navigation pose error function, and the determined visual pose weight coefficients and inertial navigation pose weight coefficients, the pose prediction function is determined.

4. The method according to claim 3, characterized in that, The pose prediction function is expressed by the following formula: in, This represents the visual pose error function. This represents the inertial navigation system pose error function. The weighting coefficients represent the visual pose and the inertial navigation pose, respectively. The weighting coefficient of the visual pose represents the contribution of the visual pose to the fused pose, and the weighting coefficient of the inertial navigation pose represents the contribution of the inertial navigation pose to the fused pose. The fused pose is obtained by fusing the visual pose and the inertial navigation pose. Let represent the fused pose at time t. This represents the predicted fused pose at the next time step t+1.

5. The method according to any one of claims 1 to 4, characterized in that, Any fused pose and the bias and compensation coefficient of the mobile robot's inertial sensor (IMU) and extrinsic parameter matrix , Related to; the extrinsic matrix The extrinsic parameter matrix is ​​used to transform between the coordinate system corresponding to the image acquisition component on the mobile robot and the coordinate system corresponding to the mobile robot. Used to realize the transformation between the coordinate system corresponding to the IMU on the mobile robot and the coordinate system corresponding to the mobile robot; The prediction-based fusion pose adjustment of the mobile robot's current pose includes at least: adjusting the mobile robot's IMU bias and compensation coefficients. and extrinsic parameter matrix , At least one of them.

6. The method according to claim 5, characterized in that, The image acquisition component is a monocular camera sensor.

7. A mobile robot, characterized in that, The mobile robot includes: a logic operation component, an image acquisition component, and an inertial measurement unit (IMU); The image acquisition component is used to acquire images during the operation of the mobile robot; at least two adjacent images acquired by the image acquisition component are used to create corresponding keyframes; the at least two adjacent images are also used to estimate visual pose. An IMU is used to measure IMU data during the operation of a mobile robot; the IMU data is used to estimate the inertial navigation pose. The logic operation component is used to perform the steps in the method as described in any one of claims 1 to 6.

8. The mobile robot according to claim 7, characterized in that, A linear Hall sensor is deployed at the center of each of the left and right wheels of the mobile robot; the data measured by the linear Hall sensor over a time period is used to estimate the displacement of the mobile robot; the displacement of the mobile robot is used in conjunction with the measurement data of the IMU to determine the inertial navigation pose; the time period consists of the acquisition time points of at least two adjacent images acquired by the image acquisition component.

9. The mobile robot according to claim 7, characterized in that, The image acquisition component is a monocular camera sensor, which is positioned between the two wheels of the mobile robot and is substantially parallel to the horizontal plane.

10. The mobile robot according to claim 8, characterized in that, The IMU is positioned at the center of the mobile robot, or at a designated location near the center.

Images