An Adaptive Multi-Fusion SLAM Method for Different Sensor Data

Abstract

The present invention discloses an adaptive multi-fusion SLAM method for different sensor data, belonging to the field of autonomous navigation of mobile robots. First, the sensor data of the mobile robot is acquired and processed, and at the same time, the features of the images obtained by the depth camera are extracted to generate global depth visual information and local depth visual information. Then, the improved IMLS-ICP algorithm is used to perform point cloud matching on the lidar data, global depth visual information, and local depth visual information to generate a point cloud map, and the improved fuzzy adaptive UKF algorithm is used for data fusion to obtain the pose of the mobile robot. Finally, the motion trajectory of the mobile robot is subjected to loop detection, and the robot pose, point cloud map, and loop detection results are used as constraints for joint optimization to obtain the path map of the mobile robot. The present invention reduces the pose estimation error of the mobile robot, the map built by the mobile robot is more accurate, and the robustness of the mobile robot to the surrounding environment is improved.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous navigation of mobile robots, and specifically relates to an adaptive multi-fusion SLAM method that can make full use of different sensor data in an unknown indoor environment. Background Technique

[0002] With the progress of science and technology, robots have developed rapidly. Robot technology has gradually spread from initial industrial production to fields such as logistics, conferences, and medical care. Mobile robots can navigate autonomously by obtaining environmental data and their own positions in the environment, and thus are widely used in multiple fields. With the development of sensor and artificial intelligence technologies, the simultaneous localization and mapping (SLAM) method for mobile robots has become a research hotspot in the field of autonomous navigation of mobile robots in recent years.

[0003] 2D lidar has been widely used in SLAM due to its low cost, high precision, strong anti-interference ability, and good real-time performance. Currently, 2D lidar SLAM has problems such as large computational complexity in laser point cloud registration, large cumulative errors in pose estimation, poor loop detection effect, easy generation of noise and leakage, and insufficient utilization of sensor data.

[0004] Currently, there are many methods to solve the 2D lidar SLAM problem of mobile robots, such as the Gmapping algorithm, Karto algorithm, and Hector algorithm. Gmapping uses the Rao-Blackwellized particle filter (RBPF) to construct a SLAM system, but the algorithm has a large computational amount and is not suitable for large scenes. The Karto algorithm introduces backend optimization and loop detection, effectively reducing the cumulative error, but the algorithm is time-consuming. The Cartographer algorithm is a 2D lidar SLAM algorithm based on graph optimization, which is divided into a front end and a backend. The front end uses a filtering method to fuse lidar point cloud data and IMU data to obtain the robot pose, and at the same time uses the inter-frame matching technology to construct a sub-map. The backend optimizes according to the constraint relationship between sub-maps to correct the pose nodes of the sub-maps to complete map construction. The map constructed by this algorithm has no obvious improvement in accuracy, and the positioning accuracy is low. Summary of the Invention

[0005] In view of the problems existing in the current 2D lidar SLAM of mobile robots and the requirements for positioning and mapping of mobile robots in indoor environments, the present invention improves on the basis of the Cartographer algorithm and proposes an adaptive multi-fusion SLAM method for different sensor data.

[0006] The specific steps of the adaptive multi-fusion SLAM method for different sensor data are as follows:

[0007] Step 1: For a mobile robot in an unknown indoor environment, obtain its sensor data and process it. At the same time, extract the features of the images obtained by the depth camera to generate global depth visual information and local depth visual information.

[0008] 1) The process of generating global depth visual information is as follows:

[0009] First, obtain the grayscale image of the depth camera and use the GCNv2 neural network to process the grayscale image to generate a feature map.

[0010] Then, homogenize the feature points on the feature map.

[0011] Finally, process the homogenized feature points through loss function training and binary description respectively, generate a feature point cloud and a descriptor and save them as the global depth visual information.

[0012] The loss function is as follows:

[0013] L det = L ce (o cur (x cur ,0)) + L ce (o tar (x tar ,x cur ))

[0014] Among them, (x cur ,0) is the position of the feature point in the current frame, (x tar ,x cur ) is the position where x cur is matched in the target frame. L ce is the weighted cross entropy. o cur is the probability distance of the current frame, and o tar is the probability distance of the target frame.

[0015] The binary description formula is as follows:

[0016]

[0017] Among them, b(x) is the eigenvalue obtained by distinguishing according to the value of f(x), and f(x) is the observation probability of the feature point.

[0018] 2) The process of generating local depth visual information is as follows:

[0019] First, taking the center of the 2D lidar as the origin, using the data of the IMU (Inertial Measurement Unit) and the consistency between the plane projection of the depth camera and the two-dimensional coordinates of the lidar, a global coordinate system of the mobile robot is constructed and the observation data of the IMU is obtained.

[0020] Assume that the mobile robot is undergoing uniformly variable motion. In the global coordinate system, the state variables of the mobile robot are as follows:

[0021]

[0022] In the above formula are respectively the abscissa, ordinate and heading angle of the mobile robot at time t k in the global coordinate system, v k and ω k are respectively the linear velocity and angular velocity of the mobile robot at time t k .

[0023] Using the global coordinate system, the observation data of the IMU is obtained. Since the output frequency of the IMU is very high, the time period [t k-1 , t k can be divided into n segments, and its formula is as follows:

[0024]

[0025] In the above formula and are respectively the measured value of the angular velocity and the measured value of the acceleration of the mobile robot, and are respectively the actual value of the angular velocity and the actual value of the acceleration of the mobile robot, and are respectively the gyroscope deviation and the accelerometer deviation, n g and n a are respectively the IMU measurement noises modeled by Gaussian white noise.

[0026] At the same time, the depth camera data is acquired, and the local features of the image obtained by the depth camera are extracted using the improved LPP algorithm.

[0027] The formula of the improved LPP algorithm combined with the motion consistency of the mobile robot is as follows:

[0028]

[0029] Among them, i, j are two different positions passed by the mobile robot in the global coordinate system, X ij is the motion state between the two points i, j of the mobile robot in the global coordinate system, l i , l jis the distance from two points i and j in the global coordinate system to the initial coordinate point, W ij is the distance weight coefficient matrix between two points i and j in the global coordinate system.

[0030] Then, the local features of the image are combined with the IMU observation data to describe the movement of the mobile robot, and the local depth visual information is generated by the R-CNN neural network introducing a weighted objective function.

[0031] The weighted objective function is as follows:

[0032]

[0033] Among them, L exist is the existing image feature, L add is the newly added image feature, and ρ(ω,a) is the weight of the IMU observation data.

[0034] Step 2: Use the improved IMLS-ICP algorithm to perform point cloud matching on the lidar data, global depth visual information, and local depth visual information for data fusion and generating a point cloud map.

[0035] The specific steps of point cloud matching are as follows:

[0036] Step 201: Use the transformation between the state quantities of the mobile robot obtained from the local depth visual information to remove the motion distortion in the lidar data.

[0037] Step 202: Delete the ground point cloud and clustering, and remove the moving objects in the global coordinate system according to the motion transformation between the mobile robot and the feature point clouds of the lidar data point cloud, global depth visual information, and local depth visual information point cloud.

[0038] Step 203: Use the point cloud within a certain range from the center point of the laser point cloud to the visual reference point cloud to construct a surface, and finally perform dimensionality reduction processing on the surface to generate a 2D map.

[0039] The visual reference point cloud is the visual point cloud without normalization processing;

[0040] The surface formula is as follows:

[0041]

[0042] Among them, ((x,y)-p i ) is the normal projection of the center point (x,y) of the laser point cloud to p on the visual reference point cloud, W i (x,y) is the reference weight, p i is the selected reference point on the visual reference point cloud, P k is the set of visual reference point clouds, is the constructed surface, is the unit vector of ((x,y)-p i ), and h is the adjustment coefficient.

[0043] Step 204: Select a certain number of laser point clouds and visual point clouds, and perform matching and solution on the selected laser point clouds and visual point clouds according to the constructed surface to obtain the point cloud registration result.

[0044] The point cloud matching formula is as follows:

[0045]

[0046] where (u i ,v i ) is the projection of a point (x i ,y i ) in the newly constructed two-dimensional map on the surface, is the distance from (x i ,y i ) to the surface, A is the matching distance between (x i ,y i ) and (u i ,v i ), is the adjustment vector of the newly constructed two-dimensional map, and R(x i ,y i ) is the average distance from (x i ,y i ) to the surface.

[0047] Step 3: Use the improved fuzzy adaptive UKF algorithm to fuse the IMU data and the point cloud registration data to obtain the pose of the mobile robot.

[0048] The specific steps are as follows:

[0049] Step 301: Perform χ 2 test on the IMU data and the point cloud registration data, and evaluate the pose information and map information of the mobile robot. The formula is as follows:

[0050]

[0051] where q k is the χ 2 test value at time k, z k is the measurement vector at time k, is the one-step measurement update at time k-1, and P zz is the optimal probability between the pose and the map.

[0052] Step 302: Further introduce fuzzy adaptive rules to estimate the process noise and observation noise of the IMU data and the point cloud registration data. The formula is as follows:

[0053]

[0054] wherein, are process noise and observation noise respectively, and α > 0, β > 0, γ > 0, η > 0 are selected constants, is the observation error.

[0055] Step 303: Perform UKF fusion on the IMU data and point cloud registration data after χ 2 test and fuzzy adaptive rule correction to obtain the pose of the mobile robot;

[0056] The pose calculation formula of the mobile robot is as follows:

[0057]

[0058] wherein, Z k is the measurement variable, u k is the control input command, W k-1 is the noise variable in the system, h(X k ) is the non-linear function of the observed value, V k is the noise variable in the measurement, f(X k-1 , u k ) is the state transition variable.

[0059] Step 4: Perform closed-loop detection on the motion trajectory of the mobile robot, and use the robot pose, point cloud map and closed-loop detection result as constraints for joint optimization to obtain the path map of the mobile robot.

[0060] The specific steps are as follows:

[0061] Step 401: Match the descriptors in the global depth vision information with the jointly optimized pose to detect whether the path of the mobile robot is a closed loop. If the path is a closed loop, obtain the pose transformation of the mobile robot from the point cloud registration to form a closed-loop constraint, and execute Step 402. If it is not a closed loop, continue to search for the starting point position until a closed-loop constraint is reached.

[0062] Step 402: Utilize the convergence of the Gauss-Newton method to perform joint optimization using the robot pose, point cloud map and closed-loop detection result as constraints to obtain the path map of the mobile robot;

[0063] The formula of the optimization function is as follows:

[0064]

[0065] wherein, F(x k ) is the state quantity x of the mobile robotk The loss function. G(x k ) is the optimization function of pose and map.

[0066] The advantages of the present invention are as follows:

[0067] (1) The present invention is an adaptive multi-fusion SLAM method for different sensor data, which makes full use of IMU data and introduces global depth vision information and local depth vision information, providing more accurate pose and map data for point cloud registration, loop closure optimization and map update.

[0068] (2) The present invention is an adaptive multi-fusion SLAM method for different sensor data. By improving the IMLS-ICP algorithm, it matches the representative 2D lidar point cloud and 3D camera point cloud, effectively reducing the computational amount of point cloud registration and improving the accuracy of positioning and mapping.

[0069] (3) The present invention is an adaptive multi-fusion SLAM method for different sensor data. By using the improved fuzzy adaptive UKF algorithm to fuse IMU and point cloud registration data, it makes full use of sensor data, reduces noise and leakage, obtains more accurate pose state quantities, and improves the robustness of the mobile robot to the environment.

[0070] (4) The present invention is an adaptive multi-fusion SLAM method for different sensor data. By jointly optimizing pose constraints, point cloud registration constraints and loop closure detection constraints, it corrects the pose nodes of the sub-map, reduces the pose estimation error of the mobile robot, and makes the map built by the mobile robot more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a schematic diagram of the principle framework of an adaptive multi-fusion SLAM method for different sensor data according to the present invention;

[0072] Figure 2 is a schematic diagram of the process of generating global depth vision information by using the GCNv2 neural network according to the present invention;

[0073] Figure 3 is a schematic diagram of the process of extracting local feature points of camera data by using the improved LPP algorithm and generating local depth vision information by combining IMU description with the improved R-CNN neural network according to the present invention;

[0074] Figure 4 is a schematic diagram of point cloud matching between lidar point cloud and visual point cloud by using the improved IMLS-ICP algorithm in the global coordinate system according to the present invention;

[0075] Figure 5Schematic diagram of the process of using the improved fuzzy adaptive UKF algorithm in the present invention to fuse IMU and point cloud matching data to obtain the pose of the mobile robot;

[0076] Figure 6 Schematic diagram of the constraint relationship among the robot pose, point cloud map and loop detection result in the present invention. Specific embodiments

[0077] The present invention will be described in detail below with reference to the accompanying drawings.

[0078] The present invention seeks a SLAM method that fuses data from multiple sensors such as lidar, depth camera, and inertial measurement unit (IMU), which can improve the performance of mobile robot positioning and mapping, reduce the pose estimation error of the mobile robot, enhance the loop detection effect of the mobile robot, make the map built by the mobile robot more accurate, and improve the robustness of the mobile robot to the surrounding environment.

[0079] Aiming at the problems of large computational complexity of 2D lidar SLAM laser point cloud registration, poor loop detection effect and large pose estimation error, the present invention introduces global depth vision information and local depth vision information for point cloud registration, loop detection and map update. Aiming at the problems of easy omission of 2D lidar point cloud registration data and low mapping accuracy, the improved IMLS-ICP algorithm is used to match the lidar point cloud, global depth vision information point cloud and local depth vision information point cloud for data fusion and generation of point cloud map. Aiming at the problems of easy generation of noise and leakage and insufficient utilization of sensor data in 2D lidar SLAM, the improved fuzzy adaptive UKF algorithm is used to fuse IMU and point cloud matching data to obtain the pose of the mobile robot. The present invention jointly optimizes the robot pose, point cloud map and loop detection result as constraints, and the performance has been significantly improved, reducing the pose estimation error of the mobile robot, making the detected closed path closer to the real closed path, making the map built by the mobile robot more accurate, and improving the robustness of the mobile robot to the surrounding environment.

[0080] An adaptive multi-fusion SLAM method for different sensor data, the principle is as Figure 1 shown, and the specific steps are as follows:

[0081] Step 1: For a mobile robot in an unknown indoor environment, obtain its sensor data and process it. At the same time, extract the features of the image obtained by the depth camera to generate global depth vision information and local depth vision information.

[0082] 1) The process of generating global depth vision information is as Figure 2 shown, and specifically:

[0083] First, obtain the grayscale image of the depth camera, and use the GCNv2 neural network to process the grayscale image to generate a feature map with a size of 1280×720×256 pixels.

[0084] Then, homogenize the feature points on the feature map.

[0085] Finally, process the homogenized feature points through loss function training and binary description respectively, generate the feature point cloud and descriptor and save them as the global depth visual information.

[0086] The loss function is as follows:

[0087] L det = L ce (o cur (x cur , 0)) + L ce (o tar (x tar , x cur ))

[0088] Among them, (x cur , 0) is the position of the feature point in the current frame, and (x tar , x cur ) is the position matched by x cur in the target frame. L ce is the weighted cross entropy. o cur is the probability distance of the current frame, and o tar is the probability distance of the target frame.

[0089] The binary description formula is as follows:

[0090]

[0091] Among them, b(x) is the feature value obtained by distinguishing according to the value of f(x), and f(x) is the observation probability of the feature point.

[0092] 2) The process of generating local depth visual information is as Figure 3 shown below:

[0093] First, with the center of the 2D lidar as the origin, use the IMU (Inertial Measurement Unit) data, the consistency of the depth camera plane projection and the lidar two-dimensional coordinates to construct the global coordinate system of the mobile robot and obtain the IMU observation data.

[0094] Assume that the mobile robot is moving with uniform variable acceleration. In the global coordinate system, the state variables of the mobile robot are as follows:

[0095]

[0096] In the above formula are respectively the abscissa, ordinate and heading angle of the mobile robot t k at the moment in the global coordinate system, v k and ω k are respectively the linear velocity and angular velocity of the mobile robot at time t k .

[0097] Using the global coordinate system, the observation data of the IMU is obtained. Since the output frequency of the IMU is very high, the time period [t k-1 , t k can be divided into n segments, and its formula is as follows:

[0098]

[0099] In the above formula and are respectively the measured value of the angular velocity and the measured value of the acceleration of the mobile robot, and are respectively the actual value of the angular velocity and the actual value of the acceleration of the mobile robot, and are respectively the gyroscope deviation and the accelerometer deviation, n g and n a are respectively the IMU measurement noises modeled by Gaussian white noise.

[0100] At the same time, the depth camera data is obtained, and the local features of the image obtained by the depth camera are extracted using the improved LPP algorithm.

[0101] The improved LPP algorithm formula combined with the motion consistency of the mobile robot is as follows:

[0102]

[0103] Among them, i, j are two different positions passed by the mobile robot in the global coordinate system, X ij is the motion state between points i and j of the mobile robot in the global coordinate system, l i , l j are the distances from points i and j to the initial coordinate point in the global coordinate system, W ij is the distance weight coefficient matrix between points i and j in the global coordinate system.

[0104] Then, the local features of the image are combined with the description of the motion of the mobile robot by the IMU observation data, and the local depth visual information is generated by the R-CNN neural network introducing the weighted objective function.

[0105] The weighted objective function is as follows:

[0106]

[0107] Among them, L exist is the existing image feature, L add is the newly added image feature, and ρ(ω,a) is the weight of the IMU observation data.

[0108] Step 2: Use the improved IMLS-ICP algorithm to perform point cloud matching on the lidar data point cloud, the global depth vision information point cloud, and the local depth vision information for data fusion and generating a point cloud map.

[0109] The principle of point cloud matching is as Figure 4 shown. The coordinate system in the figure is the global coordinate system of the mobile robot. The dot is the lidar point cloud, and the curved surface is the vision point cloud. The specific steps are as follows:

[0110] Step 201: Use the transformation between the state quantities of the mobile robot obtained from the local depth vision information to remove the motion distortion in the lidar data.

[0111] Step 202: Delete the ground point cloud and clustering, and remove the moving objects in the global coordinate system according to the motion transformation between the mobile robot, the lidar point cloud, and the vision point cloud.

[0112] Step 203: Select the lidar point cloud and the vision point cloud with rich features, small curvature, and balanced distribution, and ensure that the selected lidar point cloud and vision point cloud have a high degree of fit to make full use of the lidar point cloud and the vision point cloud.

[0113] Step 204: Use the point cloud within a certain range between the center point of the lidar point cloud and the vision reference point cloud (the vision point cloud without normalization processing) to construct a curved surface, and finally perform dimensionality reduction processing on the curved surface to generate a 2D map.

[0114] The formula of the curved surface is as follows:

[0115]

[0116] Among them, ((x,y)-p i ) is the normal projection of the center point (x,y) of the lidar point cloud to p on the vision reference point cloud, W i (x,y) is the reference weight, p i is the selected reference point on the vision reference point cloud, P k is the set of vision reference point clouds, is the constructed curved surface, is the unit vector of ((x,y)-p i ), and h is the adjustment coefficient.

[0117] Step 205: According to the constructed curved surface, perform matching and solution on the selected lidar point cloud and vision point cloud to obtain the point cloud registration result;

[0118] The point cloud matching formula is as follows:

[0119]

[0120] Among them, (u i , v i ) is the projection of a point (x i , y i ) on the surface in the newly constructed two-dimensional map, is the distance from (x i , y i ) to the surface, A is the matching distance between (x i , y i ) and (u i , v i ), is the adjustment vector of the newly constructed two-dimensional map, and R(x i , y i ) is the average distance from (x i , y i ) to the surface.

[0121] Step 3: Use the improved fuzzy adaptive UKF algorithm to fuse the IMU data and the point cloud registration data to obtain the pose of the mobile robot.

[0122] The principle is as Figure 5 shown. This process is to first perform χ 2 test and fuzzy adaptive rule correction on the IMU data and the point cloud registration data, and then perform UKF fusion to obtain the pose of the mobile robot. The specific steps are as follows:

[0123] Step 301: Perform χ 2 test on the IMU data and the point cloud registration data, and evaluate the pose information and map information of the mobile robot. The formula is as follows:

[0124]

[0125] Among them, q k is the χ 2 test value at time k, z k is the measurement vector at time k, is the one-step measurement update at time k - 1, and P zz is the optimal probability between the pose and the map.

[0126] Step 302: Further introduce fuzzy adaptive rules to estimate the process noise and observation noise of the IMU data and the point cloud registration data. The formula is as follows:

[0127]

[0128] wherein, are process noise and observation noise respectively, α > 0, β > 0, γ > 0, η > 0 are selected constants, is the observation error.

[0129] Step 303: Perform UKF fusion on the IMU and point cloud matching data after χ 2 test and fuzzy adaptive rule correction to obtain the pose of the mobile robot. The formula is as follows:

[0130]

[0131] wherein, Z k is the measurement variable, u k is the control input command, W k-1 is the noise variable in the system, h(X k ) is the non-linear function of the observed value, V k is the noise variable in the measurement, f(X k-1 , u k ) is the state transition variable.

[0132] Step 4: Perform closed-loop detection on the motion trajectory of the mobile robot, and jointly optimize the robot pose, point cloud map and closed-loop detection result as constraints to obtain the path map of the mobile robot.

[0133] The constraint relationship between each state quantity is as Figure 6 shown in the figure, where X0, X1, X2,..., X N are the state quantities of the mobile robot, and X R is the closed-loop detection result. The specific steps are as follows:

[0134] Step 401: Match the descriptor in the global depth vision information with the pose jointly optimized in step three to detect whether the path of the mobile robot is a closed loop. If the path is a closed loop, obtain the pose transformation of the mobile robot from point cloud registration to form a closed-loop constraint, and execute step 402. If it is not a closed loop, continue to search for the starting point position until a closed-loop constraint is reached.

[0135] Step 402: Utilize the convergence of the Gauss-Newton method to jointly optimize the robot pose, point cloud map and closed-loop detection result as constraints to obtain the path map of the mobile robot

[0136] The formula of the optimization function is as follows:

[0137]

[0138] wherein, F(x k) is the state variable x of the mobile robot k of the loss function. G(x k ) is the optimization function of the pose and the map.

Claims

1. An adaptive multiple fusion SLAM method for different sensor data, characterized in that Specifically: First, for a mobile robot integrating lidar, depth camera and several sensors in an unknown indoor environment, its sensor data is acquired and processed. Meanwhile, image features obtained from the depth camera are extracted to generate global depth visual information and local depth visual information; Then, the improved IMLS-ICP algorithm is used to perform point cloud matching on lidar data, global depth visual information and local depth visual information to generate a point cloud map, and the improved fuzzy adaptive UKF algorithm is used for data fusion to obtain the pose of the mobile robot; Finally, closed-loop detection is performed on the motion trajectory of the mobile robot, and the robot pose, point cloud map and closed-loop detection results are used as constraints for joint optimization to obtain the path map of the mobile robot; The specific process is as follows: First, the descriptors in the global depth visual information are matched with the jointly optimized pose to detect whether the path of the mobile robot is a closed loop. If so, the pose transformation of the mobile robot is obtained from point cloud registration to form a closed-loop constraint, and the next step is executed; otherwise, continue to search for the starting point position until the closed-loop constraint is reached; Then, using the convergence of the Gauss-Newton method, the robot pose, point cloud map and closed-loop detection results are used as constraints for joint optimization to obtain the path map of the mobile robot; The formula of the optimization function is as follows: Among them, F(x k ) is the loss function of the mobile robot state quantity x k ; G(x k ) is the optimization function of the pose and the map.

2. The adaptive multiple fusion SLAM method for different sensor data according to claim 1, wherein The process of generating the global depth visual information is as follows: First, the grayscale image of the depth camera is obtained, and the GCNv2 neural network is used to process the grayscale image to generate a feature map; Then, the feature points on the feature map are homogenized; Finally, the homogenized feature points are processed through loss function training and binary description respectively to generate and save the feature point cloud and descriptors as the global depth visual information; The loss function is as follows: L det = L ce (o cur (x cur , 0)) + L ce (o tar (x tar , x cur )) Among them, (x cur , 0) is the position of the feature point in the current frame, (x tar , x cur ) is the position where x cur is matched in the target frame; L ce is the weighted cross-entropy; o cur is the probability distance of the current frame, and o tar is the probability distance of the target frame; The binary description formula is as follows: Among them, b(x) is the feature value distinguished according to the value of f(x), and f(x) is the observation probability of the feature point.

3. An adaptive multiple fusion SLAM method for different sensor data according to claim 1, characterized in that The process of generating the local depth visual information is as follows: First, with the center of the 2D lidar as the origin, using the consistency of IMU data, the plane projection of the depth camera and the two-dimensional coordinates of the lidar, the global coordinate system of the mobile robot is constructed and the observation data of the IMU is obtained; Assume that the mobile robot is doing uniformly variable motion. In the global coordinate system, the state variables of the mobile robot are as follows: In the above formula, x k , y k , are respectively the abscissa, ordinate and heading angle of the mobile robot t k at the global coordinate system, and v k and ω k are respectively the linear velocity and angular velocity of the mobile robot t k at that moment; Using the global coordinate system, the observation data of the IMU is obtained. Since the output frequency of the IMU is very high, the time interval from k-1 , t k can be divided into n segments, and the formula is as follows: In the above formula and are the measured angular velocity value and the measured acceleration value of the mobile robot respectively, and are the actual angular velocity value and the actual acceleration value of the mobile robot respectively, and are the gyroscope bias and the accelerometer bias respectively, n g and n a are the IMU measurement noises modeled using Gaussian white noise respectively; Meanwhile, depth camera data is acquired, and the improved LPP algorithm is used to extract the local features of the image obtained by the depth camera; The formula of the improved LPP algorithm combined with the motion consistency of the mobile robot is as follows: where i and j are two different positions passed by the mobile robot in the global coordinate system, and X ij is the motion state between the two points i and j of the mobile robot in the global coordinate system, and l i , l j is the distance from the two points i and j to the initial coordinate point in the global coordinate system, and W ij is the distance weight coefficient matrix between the two points i and j in the global coordinate system; Then, the local features of the image combined with the description of the mobile robot motion by the IMU observation data are used to generate local depth visual information with the R-CNN neural network introducing a weighted objective function; The weighted objective function is as follows: Among them, L exist is the existing image feature, and L add is the newly added image feature. ρ(ω,a) is the weight of IMU observation data.

4. An adaptive multi - fusion SLAM method for different sensor data according to claim 1, characterized in that The specific steps of the point cloud matching are as follows: Step 201: Use the transformation between the state variables of the mobile robot obtained from the local depth visual information to remove the motion distortion in the lidar data; Step 202: Delete the ground point cloud and clustering, and remove the moving objects in the global coordinate system according to the motion transformation between the mobile robot, the lidar data point cloud, the feature point cloud of the global depth vision information, and the local depth vision information point cloud; Step 203: Use the point cloud within a certain range between the center point of the laser point cloud and the visual reference point cloud to construct a surface, and finally perform dimensionality reduction processing on the surface to generate a 2D map; The visual reference point cloud is the visual point cloud that has not been normalized; The surface formula is as follows: Among them, ((x, y) - p i ) is the normal projection of the center point (x, y) of the laser point cloud onto the p i on the visual reference point cloud, W i (x, y) is the reference weight, p i is the selected reference point on the visual reference point cloud, P k is the set of visual reference point clouds, is the constructed surface, is the unit vector of ((x, y) - p i ), h is the adjustment coefficient; L det is the loss function, L total is the weighted objective function; Step 204: Select a certain number of laser point clouds and visual point clouds, and perform matching and solution on the selected laser point clouds and visual point clouds according to the constructed surface to obtain the point cloud registration result; The point cloud matching formula is as follows: Among them, (u i , v i ) is the projection of a point (x i , y i ) on the surface in the newly constructed two-dimensional map. is the distance from (x i , y i ) to the surface, A is the matching distance between (x i , y i ) and (u i , v i ). is the adjustment vector of the newly constructed two-dimensional map, and R(x i , y i ) is the average distance from (x i , y i ) to the surface.

5. An adaptive multi-fusion SLAM method for different sensor data according to claim 1, characterized in that The pose calculation method of the mobile robot is as follows: Step 301: Perform χ test on IMU data and point cloud registration data, and evaluate the pose information and map information of the mobile robot. The formula is as follows: 2 Check, evaluate the pose information and map information of the mobile robot. The formula is as follows: where q k is the chi 2 test value at time k, z k is the measurement vector at time k, is the one-step measurement update at time k-1, P zz is the optimal probability between the pose and the map; Step 302: Further introduce fuzzy adaptive rules to estimate the process noise and observation noise of the IMU data and the point cloud registration data. The formula is as follows: wherein, are process noise and measurement noise respectively, α>0, β>0, γ>0, η>0 are selected constants, is the measurement error; Step 303: Perform UKF fusion on the IMU data and point cloud registration data after χ 2 checksum and fuzzy adaptive rule correction to obtain the pose of the mobile robot; The pose calculation formula of the mobile robot is as follows: Among them, Z k is a measurement variable, u k is a control input command, W k-1 is a noise variable in the system, h(X k ) is a non-linear function of the observed value, V k is a noise variable in the measurement, f(X k-1 , u k ) is a state transition variable.

Images