Multi-sensor fusion VSLAM mapping method and device

Abstract

The invention relates to the technical field of machine vision, in particular to a multi-sensor fused VSLAM mapping method and device, the initial pose of a camera is selected as the initial pose of a wheel speedometer, and encoder data and inertial measurement unit data are fused by adopting an extended Kalman filtering algorithm to estimate the pose of the wheel speedometer. Camera poses under different timestamps are estimated through a linear interpolation method, a G2O optimization algorithm is used for constraint, based on multi-source data, the speedometer estimated pose of an obstacle point is calculated and stored, when the current wheel speed meter pose and the obstacle point pose are close to historical records, loopback detection is triggered, and after loopback detection succeeds, the speedometer estimated pose of the obstacle point is determined. According to the method, the pose of the wheel speedometer and the pose of the obstacle point are corrected, the repeated obstacle point is removed, the corrected pose of the wheel speedometer and the pose of the obstacle point are utilized to construct the grid map, and the method can ensure that the grid map is constructed under the environment of lack of features. The method disclosed by the invention has excellent performance in two aspects of precision and robustness.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a multi-sensor fusion VSLAM mapping method and device. Background Art

[0002] Visual Simultaneous Localization and Mapping (VSLAM) technology is widely used in robot navigation, mobile computing and many advanced industrial fields. It can analyze sensor data of unknown environments in real time, and complete positioning and environmental map construction at the same time, thus providing basic navigation and perception functions for intelligent systems such as autonomous driving and robot navigation. It relies on visual information obtained by the camera to achieve real-time perception and position tracking of the environment, and has the advantages of low cost and high flexibility. However, in practical applications, VSLAM still faces some significant limitations, especially in the lack of features, cumulative drift and sparse point cloud performance, which greatly affects the performance of VSLAM systems in positioning and mapping.

[0003] First, feature-scarce environments are a major challenge for VSLAM. VSLAM systems rely on visual features extracted from the environment (such as corners and edges) for accurate positioning and map construction. However, in some scenes with sparse features, such as long corridors, smooth walls, or open areas, it is difficult for the camera to obtain enough visual features, resulting in the system being unable to effectively track changes in the environment. In this case, the positioning accuracy will be significantly reduced, and may even lead to the loss of pose estimation, which in turn affects the construction of the map. Secondly, the cumulative drift problem is another major bottleneck in VSLAM. In VSLAM, the pose estimation of each frame depends on the estimation result of the previous frame, so even a small error will gradually accumulate over time, causing the final positioning result to gradually deviate from the true position. This cumulative drift is particularly obvious in long-term operation or large-scale environments, especially without sufficient loop detection or other sensors (such as IMU) to assist, the system is difficult to correct the drift error, and ultimately the map appears significantly deformed or misaligned, which seriously affects the robustness and accuracy of the system. Finally, the sparse point cloud map generated by VSLAM also has obvious limitations. Sparse point clouds are usually unable to fully express the detailed information of the environment, especially in the representation of complex geometric structures. Due to the sparsity of point cloud data, the map may have blind spots at the edges of objects and subtle structures, resulting in insufficient map integrity and accuracy. This sparse representation not only affects the robot's environmental perception, but also produces inaccurate navigation information in subsequent raster map generation and path planning, increasing the robot's operational risks in actual scenarios.

[0004] Multi-sensor fusion has become a research hotspot in recent years, especially in the field of sensor information integration. Multi-source fusion strategy is regarded as an important solution to deal with complex environments. Among them, Visual-Inertial Odometry (VIO) shows great application potential by combining visual sensors with inertial measurement units (IMUs). IMUs can provide high-frequency position and direction information, which makes up for the shortcomings of visual sensors in fast motion or poor lighting conditions, while visual sensors can effectively suppress the accumulated drift of IMUs. As early as 2005, Titterton et al. emphasized the importance of accurate state estimation in VIO. Since then, research has gradually developed two mainstream methods: tight coupling and loose coupling. Tightly coupled VIO methods, such as MSCKF proposed by Mourikis et al., tightly integrate camera image information with IMU measurement data in a unified optimization framework, which can ensure high accuracy and robustness, but its computational complexity is high. Other similar methods include OKVIS, VINS-MONO, and ICE-BA. In contrast, loosely coupled VIO adopts a more flexible strategy, processing visual and IMU data separately, and then selectively fusing or optimizing the estimation results of the two. The work of Konolige et al., Tardif et al., and Weiss et al. is representative of this approach. Although the loosely coupled method reduces the computational complexity and improves the flexibility of the system, it may compromise accuracy. In addition, with the increase of environmental complexity (such as the presence of dense obstacles) or the structural degradation of visual sensor functions (such as the lack of obvious visual features or textures), even with the assistance of IMU, system performance may still be affected.

[0005] Wheel speed sensors have significant advantages in providing high-frequency body state measurements, especially in certain special motion scenarios, where they can avoid common sensor degradation problems. As demonstrated by Wu et al., by fusing wheel speed data, the problem of unobservable scale can be effectively solved, thereby significantly improving positioning accuracy. Zhang et al. proposed a vision-assisted ground robot positioning method that improves positioning accuracy in complex environments by fusing multiple sensors, but the system relies on visual sensors and may still face the problem of reduced positioning accuracy when lighting conditions are extremely poor. Liu et al. proposed a tightly coupled visual-inertial-wheel odometer fusion method that enhances positioning stability in harsh environments and realizes online external parameter calibration, but this method may be limited in terms of computational complexity and real-time performance, especially in high dynamic environments where it has high requirements for computing resources. Lee et al. proposed a joint algorithm for visual-inertial-wheel odometers that improves the impact of sensor errors on positioning accuracy through online calibration, but the online calibration process may take a long time, which will increase delays in the initialization phase when the task starts. Summary of the invention

[0006] In view of this, the purpose of the present invention is to propose a multi-sensor fusion VSLAM mapping method and device to solve the problem of insufficient mapping accuracy and robustness of the VSLAM system in complex environments.

[0007] Based on the above purpose, the present invention provides a multi-sensor fusion VSLAM mapping method, comprising:

[0008] Select the initial pose of the camera as the starting pose of the tachometer to ensure that the camera and the tachometer are in the same world coordinate system;

[0009] The extended Kalman filter algorithm is used to fuse encoder data and inertial measurement unit data to estimate the wheel speed meter posture;

[0010] The camera pose at different timestamps is estimated by linear interpolation method, and the wheel speed meter pose at the same timestamp is constrained by G2O optimization algorithm.

[0011] Based on multi-source data including wall-side, omnidirectional and collision sensors, the estimated odometer position of the obstacle point is calculated and stored. When the current wheel speedometer position and the obstacle point position are close to the historical record, loop detection is triggered. After the loop detection is successful, the wheel speedometer position and the obstacle point position will be corrected, and duplicate obstacle points will be removed.

[0012] The grid map is constructed using the corrected wheel speedometer pose and obstacle point pose.

[0013] Preferably, constructing a grid map using the corrected wheel speedometer position and obstacle point position includes:

[0014] First, a blank map is created, and then the pose information is converted into grid coordinates with a resolution of 5 cm. The idle area of ​​the sweeper, obstacles along the wall, collision points and the current position of the sweeper are drawn in combination with the ranging data of the sensor, thus completing the map construction process.

[0015] Preferably, estimating the wheel speed meter posture by fusing encoder data and inertial measurement unit data using an extended Kalman filter algorithm includes:

[0016] Set the robot position information at time t to (x t ,y t ,θ t ), taking the odometer distance increment Δs and angle increment Δθ between time t+1 and time t as input, the posture at time t+1 is expressed as follows:

[0017]

[0018] Among them, the motion increment u(Δs,Δθ) is the encoder measurement value, which is used as the control vector input. The prediction equation is nonlinear, and its Jacobian matrix for the state vector is as shown below:

[0019]

[0020] When using the extended Kalman filter algorithm to filter encoder data and inertial measurement unit data, it includes two processes: prediction and update.

[0021] Preferably, the prediction process includes calculating the predicted state variables and the predicted covariance, as shown in the following formula:

[0022]

[0023] P k|k-1 =F k-1 P k-1|k-1 F k-1 T +Q k-1

[0024] in, is the predicted state value at time k; f(·) is the state transfer function of the system, is the predicted state value at time k-1, u k is the system input at time k, P k|k-1 is the prior state covariance matrix at time k, indicating the uncertainty of the predicted state, P k-1|k-1 is the posterior state covariance matrix at time k-1, F k-1 is the Jacobian matrix of the state function, F k-1 T is the transposed matrix of the Jacobian matrix, Q k-1 is the normal distribution error matrix at time k-1.

[0025] The update process includes updating the Kalman gain, state estimation variables and error covariance:

[0026] K k =P k|k-1 H k T (H k P k|k-1 H k T +R k ) -1

[0027]

[0028] P k|k =(IK k H k )k|k-1

[0029] Among them, K k is the Kalman gain, P k|k-1 is the error covariance matrix between the measured value and the true value, H k is the Jacobian matrix of the observation function, H k T H k The transposed matrix, R k is the normal distribution error matrix at time k, the superscript -1 represents the inverse matrix of the matrix in brackets, x k|k is the state estimate at time k, x k|k-1 is the state estimate at time k-1, Z k is the measured value at time k, h(x k|k-1 ) is the observation model, P k|k is the updated error covariance matrix, and I is the identity matrix.

[0030] Preferably, during the prediction process, when the robot detects a collision, the impact of sensor degradation on pose estimation is reduced by reducing the confidence of the extended Kalman filter on the encoder input.

[0031] The present invention also provides a multi-sensor fusion VSLAM mapping device, which is used to execute the multi-sensor fusion VSLAM mapping method.

[0032] Beneficial effects of the present invention: The method of the present invention can accurately construct an occupancy grid map in different indoor scenes, and has high accuracy and stability in most scenes. Experimental results show that the method can obtain high-quality maps, and the average relative error of the map is within 3%. Compared with other VSLAM mapping methods, this method can also ensure the construction of a grid map in a feature-deficient environment. The method of the present invention has achieved excellent performance in both accuracy and robustness, and is therefore more practical in life. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 is an overall architecture diagram of an embodiment of the present invention;

[0035] Figure 2 Schematic diagram of the extended Kalman filter process according to an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of constructing a grid map according to an embodiment of the present invention;

[0037] Figure 4 The grid map is a grid map constructed by measuring the environment and constructing the grid map on the sweeper by the algorithm of the embodiment of the present invention;

[0038] Figure 5 This is a grid map obstacle point map generated by the algorithm of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0040] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0041] The overall framework of the method proposed in the present invention is as follows Figure 1 As shown in the figure, it is developed based on ORB-SLAM3. First, the extended Kalman filter (EKF) method is used to fuse the data of IMU and wheel odometer to obtain more accurate and robust positioning results. Then, multi-sensor data such as along the wall and collision are integrated to estimate the pose of the obstacle point, and a loop detection and correction method based on the similarity of wheel speedometer pose and obstacle point is proposed. Finally, the corrected wheel speedometer pose and obstacle point data are used to construct the grid map, which further improves the accuracy of the map.

[0042] 1. Extended Kalman filter fusion IMU and wheel speed meter

[0043] During the movement of the robot, due to the uneven road surface, the bumps of the wheels will cause the encoder data to be distorted, making the estimated posture of the robot far from the actual situation, and ultimately resulting in poor map effect of the robot. In order to solve this problem, the present invention adopts the extended Kalman filter (EKF) algorithm to fuse the encoder data and the inertial measurement unit (IMU) data to reduce the cumulative error of the odometer and provide more accurate position information. EKF is a commonly used multi-sensor fusion scheme that recursively determines the estimated value of the fused data in a statistically optimal way by utilizing the statistical characteristics of the measurement model. The process of EKF fusing the two sensor data is as follows: Figure 2 As shown. The state quantity at the previous moment is used to predict the state quantity at the current moment, and then the data of the wheel speed meter at the current moment is used to update the state quantity at the current moment. The state quantity at this moment is added to the data of the IMU at the current moment as a new predicted quantity and updated. Finally, the output result of the extended Kalman filter that integrates the data of the two sensors is obtained, that is, the current state quantity.

[0044] Set the robot position information at time t to (x t ,y t ,θ t ), taking the odometer distance increment Δs and angle increment Δθ between time t+1 and time t as input, the posture at time t+1 can be expressed as follows:

[0045]

[0046] Among them, the motion increment u(Δs,Δθ) can be measured by the encoder, which can be used as the control vector input. The prediction equation is nonlinear, and its Jacobian matrix for the state vector is as shown in the formula:

[0047]

[0048] When using the extended Kalman filter (EKF) algorithm to filter odometer and IMU data, there are two main steps: prediction and update. First, the prediction process requires the calculation of the predicted state variables and the predicted covariance, as shown in the formula:

[0049]

[0050] in, is the predicted state value at time k; f(·) is the state transfer function of the system; is the predicted state value at time k-1; u k is the system input at time k; P k|k-1 is the prior state covariance matrix at time k, indicating the uncertainty of the predicted state; P k-1|k-1 is the posterior state covariance matrix at time k-1; Fk-1 is the Jacobian matrix of the state function, F k-1 T is the transposed matrix of the Jacobian matrix; Q k-1 is the normal distribution error matrix at time k-1.

[0051] When the robot detects a collision, the effect of sensor degradation on pose estimation can be reduced by reducing the confidence of the extended Kalman filter on the encoder input. This is done by increasing the covariance matrix Q k The noise value of the encoder part is reduced to reduce its weight, making the extended Kalman filter more dependent on the data of other sensors, such as IMU. This can reduce the filter's dependence on the disturbed sensor and ensure a more stable pose estimation result.

[0052] Then, the system enters the update process, which requires updating the Kalman gain, state estimation variables and error covariance:

[0053]

[0054] Among them, K k is the Kalman gain; P k|k-1 is the error covariance matrix between the measured value and the true value; H k is the Jacobian matrix of the observation function; H k T H k The transposed matrix of k is the normal distribution error matrix at time k; the superscript -1 indicates the inverse matrix of the matrix in brackets; x k|k is the state estimate at time k; x k|k-1 is the state estimate at time k-1; Z k is the measured value at time k; h(x k|k-1 ) is the observation model; P k|k is the updated error covariance matrix; I is the identity matrix.

[0055] 2. Build a raster map

[0056] A grid map is used to construct the environment, and the actual environment is divided into grids, each of which has three states: occupied, idle, and unknown. When constructing the map, the distance between the robot and the target can be accurately measured by the wall sensor. The wall sensor usually consists of an infrared transmitter and a receiver. The infrared transmitter transmits an infrared signal, and the receiver receives the infrared signal reflected by the wall or obstacle. By measuring the intensity of the received infrared signal, it can be determined whether there are walls or obstacles around, and their distance and direction can be calculated. The present invention adopts a multi-sensor data fusion method, combining the information of the camera and the wheel speed meter, as well as the sensor data such as wall, omnidirectional and collision, to construct a high-precision grid map.

[0057] The pseudo code description of the raster map construction process is as follows.

[0058] Input: initial camera pose, wheel speed meter data, multi-sensor data (along the wall, omnidirectional, collision, etc.)

[0059] Output: Raster map.

[0060] 1. Initialize the camera and wheel speedometer pose

[0061] 2. Use extended Kalman filter to estimate wheel speedometer pose

[0062] T odom ←EKF(Δs,Δθ)

[0063] 3. Use camera pose to constrain wheel speed meter pose: Use linear interpolation method to estimate camera pose Tcam at different timestamps, and use G2O optimization method to constrain wheel speed meter pose at the same timestamp.

[0064]

[0065] is the linear interpolation coefficient used for o and t n Interpolation calculation between i The position of the moment.

[0066] 4. Calculate the obstacle point pose: Use the wheel speed meter pose Todom and multi-sensor data sensor_data to estimate the obstacle point pose P obstacle and store it.

[0067] P obstacle =f sensor (T odom ,sensor_data)

[0068] 5. Loop detection: using t i and t jThe distance between the two frames of odometer pose at time d(Todom(t i ),Todom(t j )) and the distance d(Pobstacle(t i ),Pobstacle(t j )) to determine whether the sum exceeds the threshold ε, and the optimized odometer posture is obtained through the posture optimization algorithm. and obstacle point location information

[0069] if(d(T odom (t i ),T odom (t j ))+d(P obstacle (t i ),P obstace (t j ))<ε)

[0070]

[0071] 6. Build a grid map: Use the corrected wheel speedometer position and obstacle points to build a grid map.

[0072]

[0073] The construction process of the grid map can be summarized as follows: First, the initial pose of the camera is selected as the starting pose of the wheel speed meter to ensure that the camera and the wheel speed meter are in the same world coordinate system. Then, the extended Kalman filter (EKF) algorithm is used to fuse multi-sensor data to estimate the wheel speed meter pose. Next, the linear interpolation method is used to estimate the camera pose at different timestamps, and the G2O optimization algorithm is used to constrain the wheel speed meter pose at the same timestamp.

[0074] Then, based on multi-source data such as wall-side, omnidirectional and collision sensors, the estimated odometer position of the obstacle point is calculated and stored. When the current wheel speedometer position and the obstacle point position are close to the historical record, loop detection is triggered. After the loop detection is successful, the wheel speedometer position and obstacle point position will be corrected, and duplicate obstacle points will be removed.

[0075] Finally, the grid map is constructed using the corrected wheel speed meter posture and obstacle point posture. The specific steps include: first creating a blank map, then converting the posture information into grid coordinates with a resolution of 5 cm, and combining the sensor's ranging data to draw the robot's idle area, obstacles along the wall, collision points, and the current position of the robot, thereby completing the map construction process. Through this process, an accurate grid map can be generated to improve the robot's environmental perception and navigation capabilities.

[0076] The method of the present invention was then experimentally evaluated.

[0077] The experiment of the present invention uses a sweeping robot equipped with a monocular camera to verify the indoor positioning and mapping performance. The device is equipped with an Allwinner MR133 chip, a 1.8GHzx4-core CPU and 512MB of memory, and runs the Linux operating system. In order to meet the real-time operation requirements of low-cost processors, this embodiment sets the frequency of the 680X480 grayscale image provided by the device from 20Hz to 2Hz, and the measurement frequency of the inertial measurement unit (IMU), wheel encoder and left and right wall sensors is 50Hz. The experimental platform uses Ubuntu 18.04, and the C++ compiler version is 7.5. In addition, due to the limited hardware resources of the sweeping robot, all tests were performed in a non-ROS environment. Finally, the sweeping robot can process image data at a frequency of 2Hz to verify the performance of the method of the present invention in practical applications.

[0078] In order to analyze the experimental results, the algorithm of the present invention uses mean absolute error (MAE), mean relative error (MRE) and root mean square error (RMSE) as evaluation indicators. The root mean square error is the square root of the ratio of the square of the deviation between the predicted value and the true value to the number of observations n, which is used to measure the deviation between the observed value and the true value.

[0079]

[0080] The formula It represents the estimated distance of the grid map, P i It represents the actual distance in the real environment.

[0081] 1. Grid map construction

[0082] For example, the starting position of the robot vacuum cleaner is recorded, and the robot vacuum cleaner is controlled by a remote controller to draw a map in the room. The effect of building a map based on a single wheel speed meter and the effect of building a map based on an odometer fused with EKF are compared, such as Figure 3 As shown:

[0083] By analyzing and comparing Figure 3 a and Figure 3 b shows the mapping effect. It can be seen that after using EKF to fuse the wheel speed meter and IMU data, the actual environment can be recorded more accurately. As shown by the red line, when relying only on a single odometer, the vertical error of the measurement is large. After introducing the EKF fused odometer, the error is significantly reduced. Moreover, the two-dimensional grid map created by the multi-sensor fusion method used in the present invention is smoother than the traditional method and describes the environment more accurately.

[0084] 2. Grid map length and width accuracy test

[0085] Exemplarily, in order to verify the accuracy of generating an occupancy grid map using the method of the present invention, the method selects three different indoor scenes and constructs an occupancy grid map, such as Figure 4 Then, this embodiment performs multiple tests in three different indoor scenes, and compares and analyzes the estimated length and width of the grid map with the length and width of the actual environment. The test results are shown in Table 1, in cm.

[0086] Table 1 Grid map length and width accuracy

[0087]

[0088] The experimental results show that the adopted grid map construction algorithm has high accuracy and stability in most scenarios. As can be seen from Table 2, the algorithm of the present invention can obtain high-quality maps, and the average relative error of the map is within 3%. Especially in the accuracy measurement in simple scenes, the multi-sensor fusion mapping algorithm effectively improves the accuracy of map construction and significantly reduces the error of the grid map. However, in scene 3, the root mean square error increases, indicating that the accuracy of the algorithm decreases in complex or larger scenes, showing that the multi-sensor fusion effect needs to be further optimized in more complex environments.

[0089] 3. Grid map obstacle point accuracy test

[0090] Combined with the collision sensor, the sweeping robot can add obstacle points to the map when it collides with objects that are visually limited or cannot be recognized (such as glass doors, low obstacles, charging stations, etc.). These obstacle points introduced by the collision sensor will be updated along with the map correction brought by the closed-loop detection during the mapping process, thereby ensuring the correctness of the map while making up for the lack of visual failure areas. Exemplarily, in order to evaluate the accuracy of obstacle points in the grid map, this embodiment first controls the sweeping robot to move along the wall, and then turns and collides with obstacles in the environment. By measuring the actual distance from the obstacle to the wall, it is compared with the distance output by the algorithm, as shown in Table 2.

[0091] Table 2. Grid map obstacle point accuracy

[0092]

[0093] Through experimental tests, it can be seen that the method of the present invention can effectively detect obstacles in the environment, such as Figure 5 The average relative error of obstacles is 4.25%. The small error shows that the algorithm can accurately reflect the obstacle location information in the grid map, which is consistent with the actual environment. The root mean square error is 7.9cm, which shows that although the average absolute error is low, there are some large error points in the data, resulting in a higher root mean square error.

[0094] Those skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present invention is limited to these examples; under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-sensor fusion VSLAM mapping method, characterized in that: include: Select the initial pose of the camera as the starting pose of the tachometer to ensure that the camera and the tachometer are in the same world coordinate system; The extended Kalman filter algorithm is used to fuse encoder data and inertial measurement unit data to estimate the wheel speed meter posture; The camera pose at different timestamps is estimated by linear interpolation method, and the wheel speed meter pose at the same timestamp is constrained by G2O optimization algorithm. Based on multi-source data including wall-side, omnidirectional and collision sensors, the estimated odometer position of the obstacle point is calculated and stored. When the current wheel speedometer position and the obstacle point position are close to the historical record, loop detection is triggered. After the loop detection is successful, the wheel speedometer position and the obstacle point position will be corrected, and duplicate obstacle points will be removed. The grid map is constructed using the corrected wheel speedometer pose and obstacle point pose.

2. The VSLAM mapping method of multi-sensor fusion according to claim 1, characterized in that: The method of constructing a grid map using the corrected wheel speedometer position and obstacle point position comprises: First, a blank map is created, and then the pose information is converted into grid coordinates with a resolution of 5 cm. The idle area of ​​the sweeper, obstacles along the wall, collision points and the current position of the sweeper are drawn in combination with the ranging data of the sensor, thus completing the map construction process.

3. The VSLAM mapping method of multi-sensor fusion according to claim 1, characterized in that, The method of estimating the wheel speed meter posture by fusing encoder data and inertial measurement unit data using an extended Kalman filter algorithm includes: Set the robot position information at time t to (x t ,y t ,θ t ), taking the odometer distance increment Δs and angle increment Δθ between time t+1 and time t as input, the posture at time t+1 is expressed as follows: Among them, the motion increment u(Δs,Δθ) is the encoder measurement value, which is used as the control vector input. The prediction equation is nonlinear, and its Jacobian matrix for the state vector is as shown below: When using the extended Kalman filter algorithm to filter encoder data and inertial measurement unit data, it includes two processes: prediction and update.

4. The VSLAM mapping method of multi-sensor fusion according to claim 3, characterized in that, The prediction process includes calculating the predicted state variables and the predicted covariance, as shown in the following formula: P k|k-1 =F k-1 P k-1|k-1 F k-1 T +Q k-1 in, is the predicted state value at time k; f(·) is the state transfer function of the system, is the predicted state value at time k-1, u k is the system input at time k, P k|k-1 is the prior state covariance matrix at time k, indicating the uncertainty of the predicted state, P k-1|k-1 is the posterior state covariance matrix at time k-1, F k-1 is the Jacobian matrix of the state function, F k-1 T is the transposed matrix of the Jacobian matrix, Q k-1 is the normal distribution error matrix at time k-1; The update process includes updating the Kalman gain, state estimation variables and error covariance: K k =P k|k-1 H k T (H k P k|k-1 H k T +R k ) -1 P k|k =(I-K k H k )P k|k-1 Among them, K k is the Kalman gain, P k|k-1 is the error covariance matrix between the measured value and the true value, H k is the Jacobian matrix of the observation function, H k T H k The transposed matrix, R k is the normal distribution error matrix at time k, the superscript -1 represents the inverse matrix of the matrix in brackets, x k|k is the state estimate at time k, x k|k-1 is the state estimate at time k-1, Z k is the measured value at time k, h(x k|k-1 ) is the observation model, P k|k is the updated error covariance matrix, and I is the identity matrix.

5. The VSLAM mapping method of multi-sensor fusion according to claim 4, characterized in that, During the prediction process, when the robot detects a collision, the effect of sensor degradation on the pose estimate is reduced by lowering the confidence of the extended Kalman filter on the encoder input.

6. A multi-sensor fusion VSLAM mapping device, characterized in that: A VSLAM mapping method for executing multi-sensor fusion as described in any one of claims 1-5.

Images