Robot intelligent inspection method and system based on SLAM navigation and multi-sensor fusion

Abstract

The invention discloses a robot intelligent inspection method and system based on SLAM navigation and multi-sensor fusion, and belongs to the technical field of intelligent navigation control, and the method comprises the steps: carrying out the manual control through a robot inspection vehicle provided with a plurality of sensors, carrying out the initial inspection in a target inspection region, collecting the route data through a multi-sensor collection vehicle, and carrying out the inspection of the robot. The method comprises the following steps: constructing an inspection navigation map by combining an SLAM navigation algorithm, determining a running source point on the inspection navigation map, generating a high-precision running track according to a coordinate source point, normalizing an inspection running path according to the high-precision running track, and performing intelligent inspection by a robot inspection vehicle according to the inspection running path, so that complex map construction requirements can be effectively met; meanwhile, a global identifier is generated for trajectory data, trajectory management is facilitated, SLAM positioning and visual positioning results are judged through multiple positioning signals, the uncertainty of visual positioning is reduced, optimization of the inspection trajectory is achieved, and the robot inspection vehicle can conduct stable inspection in a complex area.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent navigation control, and in particular relates to a robot intelligent inspection method and system based on SLAM navigation and multi-sensor fusion. BACKGROUND

[0002] With the progress of technology and the improvement of production requirements, industrial and agricultural inspection robots have achieved certain technical results. Although industrial and agricultural inspection robots have brought a lot of convenience in practical application, there are still some deficiencies, especially in emergency obstacle avoidance and handling of complex terrain.

[0003] For example, under the influence of environmental factors such as light, wind, and branch shelter, the accuracy of some position sensors may be affected; most SLAM systems have poor adaptability to dynamic changes in the environment, especially in complex and variable industrial scene areas; in terms of path planning, although agricultural inspection robots have autonomous path planning function, in complex terrain and environment, the accuracy and passability of path planning are still insufficient, which can easily lead to the phenomenon of robot getting lost, thereby affecting the normal use of the inspection robot. SUMMARY

[0004] To solve the above problems and technical defects, the application adopts the following technical solution: a robot intelligent inspection method based on SLAM navigation and multi-sensor fusion, comprising the following steps:

[0005] Using a robot inspection vehicle equipped with multiple sensors for manual control, initial inspection is performed in the target inspection area;

[0006] Using a multi-sensor collection vehicle to collect route data, and combining a SLAM navigation algorithm to construct an inspection navigation map;

[0007] Determining a running source point on the inspection navigation map, and generating a high-precision running track according to the coordinate source point;

[0008] Regularizing the inspection running path according to the high-precision running track, and the robot inspection vehicle performs intelligent inspection according to the inspection running path.

[0009] Preferably, after collecting the route data, the data needs to be filtered by a three-level filtering method, and the process is as follows:

[0010] Calculating the collected route data to calculate the longitude and latitude of the four points intersected by the view frustum and the terrain, and the first bounding box parallel to the coordinate axis is formed by the four points, the tiles contained in the first bounding box are calculated, the tile data is extracted and put into the first cache;

[0011] Traverse each element in each tile, judge whether the second bounding box of each element intersects with the first bounding box, if it intersects, put the element into the second cache;

[0012] Traverse all elements in the second cache, and multiply them with the MVP matrix to obtain normalized bounding box coordinates, judge whether each component of each bounding box coordinate is in the range of [-1, 1], if one component is in, put the element into the third cache.

[0013] Further, the construction process of the inspection navigation map is as follows:

[0014] Collect laser SLAM positioning data through multiple sensors, generate high-precision points according to the laser SLAM positioning data, splice the high-precision points in order, take out the repeated part, and generate a high-precision trajectory;

[0015] According to the initial inspection trajectory, generate a sparse trajectory, compare and combine the high-precision trajectory with the sparse trajectory, and generate an inspection navigation trajectory,

[0016] Mark the start point coordinates, end point coordinates, intersection type and intersection direction in the inspection navigation trajectory, and construct an inspection navigation map.

[0017] Preferably, the generation process of the high-precision running trajectory is as follows:

[0018] Obtain the data collected by the sensors of the robot inspection vehicle in the last inspection process, and load the inspection trajectory of the last inspection;

[0019] Convert the data collected by all sensors into positioning, convert it to the body coordinate system of the inspection vehicle, calculate the distance difference between the visual positioning and the SLAM positioning of the camera shooting, and judge whether the visual positioning is normal;

[0020] If the visual positioning is abnormal, record the coordinates of the abnormal position, use the roadside equipment to confirm, update and repair the high-precision running trajectory according to the confirmation result, and use the repaired high-precision running trajectory as the inspection running path.

[0021] Further, the process of judging whether the visual positioning is normal is as follows:

[0022] Combine and splice the segmented graphics in the visual positioning, match the spliced combined image with the marked points in the inspection navigation map, and judge whether it is consistent;

[0023] If it is consistent, calculate the distance deviation between the marked points in the combined image, and judge whether the visual positioning is normal according to the calculation result.

[0024] A robot intelligent inspection system based on SLAM navigation and multi-sensor fusion, comprising:

[0025] a sensor positioning module mounted on the robot inspection vehicle, configured to perform SLAM positioning, visual positioning, ultrasonic positioning, and route data collection;

[0026] a navigation map construction module configured to construct an inspection navigation map by combining the data collected by the sensor positioning module with a SLAM navigation algorithm;

[0027] a trajectory generation module configured to determine a running source point on the inspection navigation map and generate a high-precision running trajectory according to the coordinate source point;

[0028] an inspection control module configured to regularize an inspection running path according to the high-precision running trajectory and control the robot inspection vehicle to perform intelligent inspection according to the inspection running path.

[0029] Preferably, the collected route data needs to be filtered by a three-stage filtering method, and the process is as follows:

[0030] The collected route data is calculated to calculate the longitude and latitude of the four points intersected by the view frustum and the terrain, and a first bounding box parallel to the coordinate axis is formed by the four points. The tiles contained in the first bounding box are calculated, and the tile data is extracted and placed in the first cache.

[0031] Each element in each tile is traversed, and it is determined whether the second bounding box of each element intersects the first bounding box. If it intersects, the element is placed in the second cache.

[0032] All elements in the second cache are traversed, and multiplied by the MVP matrix to obtain normalized bounding box coordinates. It is determined whether each component of each bounding box coordinate is within the range of [-1, 1]. If one component is within the range, the element is placed in the third cache.

[0033] Further, the construction process of the inspection navigation map is as follows:

[0034] Laser SLAM positioning data is collected by multiple sensors, high-precision points are generated according to the laser SLAM positioning data, the high-precision points are spliced in order, the repeated parts are removed, and a high-precision trajectory is generated.

[0035] A sparse trajectory is generated according to the inspection trajectory of the initial inspection, and the high-precision trajectory and the sparse trajectory are compared and combined to generate an inspection navigation trajectory.

[0036] The starting point coordinates, end point coordinates, intersection type and intersection direction are marked in the inspection navigation trajectory, and an inspection navigation map is constructed.

[0037] Preferably, the generation process of the high-precision running trajectory is as follows:

[0038] Acquire the data collected by sensors of the robot inspection vehicle in the last inspection process, and load the inspection track of the last inspection;

[0039] Convert the data collected by all sensors into positioning conversion, convert them into the coordinate system of the inspection vehicle body, calculate the distance difference between the visual positioning and the SLAM positioning of the camera shooting, and determine whether the visual positioning is normal;

[0040] If the visual positioning is abnormal, record the coordinates of the abnormal position, use the roadside equipment to confirm, update and repair the high-precision running track according to the confirmation result, and use the repaired high-precision running track as the inspection running path.

[0041] Further, the process of determining whether the visual positioning is normal is as follows:

[0042] The segmented graphics in the visual positioning are combined and spliced, the spliced combined image is matched with the labeled points in the inspection navigation map, and it is determined whether it is consistent or not.

[0043] If it is consistent, the distance deviation between the labeled points in the combined image is calculated, and it is determined whether the visual positioning is normal according to the calculation result.

[0044] Compared with the prior art, the beneficial effects of the present application are:

[0045] The present application filters the route data through a three-level filtering method, quickly excludes invisible elements, reduces the processing amount, improves the rendering efficiency and performance, effectively meets the complex map construction requirements, generates global identification for the track data, facilitates the management of the track, judges the SLAM positioning and visual positioning results through multiple positioning signals, reduces the uncertainty of visual positioning, optimizes the inspection track, and enables the robot inspection vehicle to stably inspect in complex areas. BRIEF DESCRIPTION OF DRAWINGS

[0046] In the drawings:

[0047] Figure 1 The method steps of the embodiments of the present application are shown in the figure;

[0048] Figure 2 The system structure diagram of the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0050] Embodiment 1

[0051] As Figure 1 shown, a robot intelligent inspection method based on SLAM navigation and multi-sensor fusion includes the following steps:

[0052] Using a robot inspection vehicle equipped with multiple sensors for manual control, initial inspection is performed in the target inspection area;

[0053] Using a multi-sensor collection vehicle to collect route data, combined with a SLAM navigation algorithm to construct an inspection navigation map;

[0054] After collecting route data, it needs to be filtered through a three-level filtering method, the process is as follows:

[0055] The collected route data is calculated to calculate the longitude and latitude of the four points intersected by the view frustum and the terrain, and the first bounding box parallel to the coordinate axis is formed by the four points. The tiles contained in the first bounding box are calculated, the tile data is extracted, and placed in the first cache;

[0056] Traverse each element in each tile, judge whether the second bounding box of each element intersects with the first bounding box, if it intersects, put the element into the second cache;

[0057] Traverse all elements in the second cache and multiply them with the MVP matrix to get the normalized bounding box coordinates. Determine whether each component of each bounding box coordinate is within the range of [-1, 1], if so, put the element into the third cache.

[0058] The construction process of the inspection navigation map is as follows:

[0059] Collect laser SLAM positioning data through multi-sensor, generate high-precision points according to laser SLAM positioning data, splice high-precision points in order, remove duplicate parts, and generate high-precision trajectory;

[0060] According to the inspection trajectory of the initial inspection, generate a sparse trajectory, compare and combine the high-precision trajectory with the sparse trajectory, and generate an inspection navigation trajectory,

[0061] Mark the start point coordinates, end point coordinates, intersection type and intersection direction in the inspection navigation trajectory, and construct the inspection navigation map.

[0062] Determine the running source point on the inspection navigation map, and generate a high-precision running trajectory according to the coordinate source point;

[0063] The generation process of the high-precision running trajectory is as follows:

[0064] Acquire the data collected by sensors of the robot inspection vehicle in the last inspection process, and load the inspection track of the last inspection;

[0065] Convert the data collected by all sensors into positioning conversion, convert them into the coordinate system of the inspection vehicle, calculate the distance difference between the visual positioning and the SLAM positioning of the camera shooting, and determine whether the visual positioning is normal;

[0066] If the visual positioning is abnormal, record the coordinates of the abnormal position, use the roadside equipment for confirmation, update and repair the high-precision running track according to the confirmation result, and use the repaired high-precision running track as the inspection running path.

[0067] The process of determining whether the visual positioning is normal is as follows:

[0068] The segmented graphics in the visual positioning are combined and spliced, the spliced combined image is matched with the labeled points in the inspection navigation map, and it is determined whether it is consistent;

[0069] If it is consistent, the distance deviation between the labeled points in the combined image is calculated, and it is determined whether the visual positioning is normal according to the calculation result.

[0070] According to the high-precision running track, the robot inspection vehicle performs intelligent inspection according to the inspection running path.

[0071] Embodiment 2

[0072] As shown in Figure 2 A robot intelligent inspection system based on SLAM navigation and multi-sensor fusion, comprising:

[0073] The sensor positioning module is installed on the robot inspection vehicle and is used for SLAM positioning, visual positioning, ultrasonic positioning and route data collection; after collecting the route data, the data needs to be filtered by a three-level filtering method, and the process is as follows:

[0074] The collected route data is calculated, the longitude and latitude of the four points intersected by the view frustum and the terrain are calculated, a first bounding box parallel to the coordinate axis is formed by the four points, the tiles contained in the first bounding box are calculated, the tile data is extracted and put into the first cache;

[0075] Traverse each element in each tile, determine whether the second bounding box of each element intersects with the first bounding box, and if it intersects, put the element into the second cache;

[0076] Traverse all elements in the second cache, multiply them by the MVP matrix, get the normalized bounding box coordinates, and determine whether each component of each bounding box coordinate is within the range of [-1, 1], if one component is within the range, put the element into the third cache.

[0077] The navigation map construction module is used to combine the data collected by the sensor positioning module with the SLAM navigation algorithm to construct an inspection navigation map. The construction process of the inspection navigation map is as follows:

[0078] Collect laser SLAM positioning data through multiple sensors, generate high-precision points based on the laser SLAM positioning data, splice the high-precision points in sequence, remove the repeated parts, and generate a high-precision trajectory;

[0079] Generate a thinning trajectory based on the inspection trajectory of the initial inspection, compare and combine the high-precision trajectory with the thinning trajectory to generate an inspection navigation trajectory,

[0080] Mark the starting point coordinates, end point coordinates, intersection type and intersection direction in the inspection navigation trajectory to build an inspection navigation map.

[0081] The trajectory generation module is used to determine the running source point on the inspection navigation map and generate a high-precision running trajectory based on the coordinate source point. The process of generating a high-precision running trajectory is as follows:

[0082] Obtain the data collected by the robot inspection vehicle's sensors during the previous inspection process, and then load the inspection trajectory of the previous inspection;

[0083] Perform positioning conversion on the data collected by all sensors, convert it into the inspection vehicle coordinate system, calculate the distance difference between the visual positioning captured by the camera and the SLAM positioning, and determine whether the visual positioning is normal;

[0084] If the visual positioning is abnormal, record the coordinates of the abnormal location and use the roadside equipment to confirm it. Based on the confirmation result, update and repair the high-precision running trajectory, and use the repaired high-precision running trajectory as the inspection running path.

[0085] The process of judging whether the visual positioning is normal is as follows:

[0086] Combine and stitch the segmented graphics in the visual positioning, match the stitched combined image with the marked points in the inspection navigation map to determine whether they are consistent;

[0087] If they match, the distance deviation between the marked points in the combined image is calculated, and whether the visual positioning is normal is determined based on the calculation results.

[0088] The inspection control module is used to standardize the inspection operation path according to the high-precision operation trajectory and control the robotic inspection vehicle to perform intelligent inspection according to the inspection operation path.

[0089] The above-described embodiments are merely preferred embodiments of the present application, which are described in more detail and specifically, but should not be understood as limiting the scope of the present application. It should be noted that, for those skilled in the art, several modifications, improvements and substitutions can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A robot intelligent inspection method based on SLAM navigation and multi-sensor fusion, characterized in that: The following steps are involved: Use a robotic inspection vehicle equipped with multiple sensors for manual control to conduct initial inspections in the target inspection area; Use multi-sensor collection vehicles to collect route data and combine it with SLAM navigation algorithms to build inspection navigation maps; Determine the running source point on the inspection navigation map and generate a high-precision running trajectory based on the coordinate source point; The inspection operation path is standardized according to the high-precision operation trajectory, and the robotic inspection vehicle performs intelligent inspection according to the inspection operation path.

2. A robot intelligent inspection method based on SLAM navigation and multi-sensor fusion according to claim 1, characterized in that: After collecting the route data, it is necessary to filter the data using a three-level filtering method. The process is as follows: Calculate the collected route data to determine the latitude and longitude of the four points where the view frustum intersects the terrain. Use the four points to form a first bounding box parallel to the coordinate axis. Calculate the tiles contained in the first bounding box, extract the tile data, and store them in a first cache. Traverse the elements in each tile and determine whether the second bounding box of each element intersects with the first bounding box. If so, put the element into the second cache; Traverse all elements in the second cache and multiply them with the MVP matrix to obtain the normalized bounding box coordinates. Determine whether each component of each bounding box coordinate is in the range of [-1,1]. If there is a component in the range, put the element into the third cache.

3. A robot intelligent inspection method based on SLAM navigation and multi-sensor fusion according to claim 2, characterized in that: The construction process of the inspection navigation map is as follows: Collect laser SLAM positioning data through multiple sensors, generate high-precision points based on the laser SLAM positioning data, splice the high-precision points in sequence, remove the repeated parts, and generate a high-precision trajectory; Generate a thinning trajectory based on the inspection trajectory of the initial inspection, compare and combine the high-precision trajectory with the thinning trajectory to generate an inspection navigation trajectory, Mark the starting point coordinates, end point coordinates, intersection type and intersection direction in the inspection navigation trajectory to build an inspection navigation map.

4. The robot intelligent inspection method based on SLAM navigation and multi-sensor fusion according to claim 1 is characterized in that: The generation process of the high-precision running trajectory is as follows: Obtain the data collected by the robot inspection vehicle's sensors during the previous inspection process, and then load the inspection trajectory of the previous inspection; Perform positioning conversion on the data collected by all sensors, convert it into the inspection vehicle coordinate system, calculate the distance difference between the visual positioning captured by the camera and the SLAM positioning, and determine whether the visual positioning is normal; If the visual positioning is abnormal, record the coordinates of the abnormal location and use the roadside equipment to confirm it. Based on the confirmation result, update and repair the high-precision running trajectory, and use the repaired high-precision running trajectory as the inspection running path.

5. A robot intelligent inspection method based on SLAM navigation and multi-sensor fusion according to claim 4, characterized in that: The process of judging whether the visual positioning is normal is as follows: Combine and stitch the segmented graphics in the visual positioning, match the stitched combined image with the marked points in the inspection navigation map to determine whether they are consistent; If they match, the distance deviation between the marked points in the combined image is calculated, and whether the visual positioning is normal is determined based on the calculation results.

6. A robot intelligent inspection system based on SLAM navigation and multi-sensor fusion, characterized in that: include: The sensor positioning module is installed on the robotic inspection vehicle and is used for SLAM positioning, visual positioning, ultrasonic positioning, and route data collection; The navigation map construction module is used to combine the data collected by the sensor positioning module with the SLAM navigation algorithm to construct an inspection navigation map; The trajectory generation module is used to determine the running source point on the inspection navigation map and generate a high-precision running trajectory based on the coordinate source point; The inspection control module is used to standardize the inspection operation path according to the high-precision operation trajectory and control the robotic inspection vehicle to perform intelligent inspection according to the inspection operation path.

7. The robot intelligent inspection system based on SLAM navigation and multi-sensor fusion according to claim 1, characterized in that: After collecting the route data, it is necessary to filter the data using a three-level filtering method. The process is as follows: Calculate the collected route data to determine the latitude and longitude of the four points where the view frustum intersects the terrain. Use the four points to form a first bounding box parallel to the coordinate axis. Calculate the tiles contained in the first bounding box, extract the tile data, and store them in a first cache. Traverse the elements in each tile and determine whether the second bounding box of each element intersects with the first bounding box. If so, put the element into the second cache; Traverse all elements in the second cache and multiply them with the MVP matrix to obtain the normalized bounding box coordinates. Determine whether each component of each bounding box coordinate is in the range of [-1,1]. If there is a component in the range, put the element into the third cache.

8. The robot intelligent inspection system based on SLAM navigation and multi-sensor fusion according to claim 7, characterized in that: The construction process of the inspection navigation map is as follows: Collect laser SLAM positioning data through multiple sensors, generate high-precision points based on the laser SLAM positioning data, splice the high-precision points in sequence, remove the repeated parts, and generate a high-precision trajectory; Generate a thinning trajectory based on the inspection trajectory of the initial inspection, compare and combine the high-precision trajectory with the thinning trajectory to generate an inspection navigation trajectory, Mark the starting point coordinates, end point coordinates, intersection type and intersection direction in the inspection navigation trajectory to build an inspection navigation map.

9. The robot intelligent inspection system based on SLAM navigation and multi-sensor fusion according to claim 1, characterized in that: The generation process of the high-precision running trajectory is as follows: Obtain the data collected by the robot inspection vehicle's sensors during the previous inspection process, and then load the inspection trajectory of the previous inspection; Perform positioning conversion on the data collected by all sensors, convert it into the inspection vehicle coordinate system, calculate the distance difference between the visual positioning captured by the camera and the SLAM positioning, and determine whether the visual positioning is normal; If the visual positioning is abnormal, record the coordinates of the abnormal location and use the roadside equipment to confirm it. Based on the confirmation result, update and repair the high-precision running trajectory, and use the repaired high-precision running trajectory as the inspection running path.

10. The robot intelligent inspection system based on SLAM navigation and multi-sensor fusion according to claim 9, characterized in that: The process of judging whether the visual positioning is normal is as follows: Combine and stitch the segmented graphics in the visual positioning, match the stitched combined image with the marked points in the inspection navigation map to determine whether they are consistent; If they match, the distance deviation between the marked points in the combined image is calculated, and whether the visual positioning is normal is determined based on the calculation results.

Images