A robot intelligent inspection method and system based on SLAM navigation and multi-sensor fusion

Abstract

The application discloses a kind of robot intelligent inspection method and system based on SLAM navigation and multi-sensor fusion, belong to intelligent navigation control technical field, including using robot inspection vehicle equipped with multiple sensors to carry out manual control, carry out initial inspection in target inspection area, use multi-sensor collection vehicle to collect route data, combined with SLAM navigation algorithm to construct inspection navigation map, determine running source point on inspection navigation map, generate high-precision running track according to coordinate source point, normalize inspection running path according to high-precision running track, robot inspection vehicle carries out intelligent inspection according to inspection running path, can effectively respond to complex map construction demand, simultaneously to track data generation global identification, facilitate to manage track, judge SLAM positioning and visual positioning result by a variety of positioning signals again, reduce the uncertainty of visual positioning, realize the optimization of inspection track, so that robot inspection vehicle can carry out stable inspection in complex area.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent navigation control, and in particular relates to a robot intelligent inspection method and system based on SLAM navigation and multi-sensor fusion. BACKGROUND

[0002] With the progress of technology and the improvement of production requirements, industrial and agricultural inspection robots have achieved certain technical results. Although industrial and agricultural inspection robots have brought a lot of convenience in practical application, there are still some deficiencies, especially in emergency obstacle avoidance and handling of complex terrain.

[0003] For example, under the influence of environmental factors such as light, wind, and branch shelter, the accuracy of some position sensors may be affected; most SLAM systems have poor adaptability to dynamic changes in the environment, especially in complex and variable industrial scene areas; in terms of path planning, although agricultural inspection robots have autonomous path planning function, in complex terrain and environment, the accuracy and passability of path planning are still insufficient, which can easily lead to the phenomenon of robot getting lost, thereby affecting the normal use of the inspection robot. SUMMARY

[0004] To solve the above problems and technical defects, the application adopts the following technical solution: a robot intelligent inspection method based on SLAM navigation and multi-sensor fusion, comprising the following steps:

[0005] Using a robot inspection vehicle equipped with multiple sensors for manual control, initial inspection is performed in the target inspection area;

[0006] Using a multi-sensor collection vehicle to collect route data, and combining a SLAM navigation algorithm to construct an inspection navigation map;

[0007] Determining a running source point on the inspection navigation map, and generating a high-precision running track according to the running source point;

[0008] Regularizing an inspection running path according to the high-precision running track, and the robot inspection vehicle performing intelligent inspection according to the inspection running path.

[0009] Preferably, after collecting the route data, a three-level filtering method is used for data filtering, and the process is as follows:

[0010] Calculating the collected route data to calculate the longitude and latitude of the four points intersected by the view frustum and the terrain, and a first bounding box parallel to the coordinate axis is formed by the four points, the tiles contained in the first bounding box are calculated, the tile data is extracted, and is put into the first cache;

[0011] Traverse each element in each tile, judge whether the second bounding box of each element intersects with the first bounding box, if it intersects, put the element into the second cache;

[0012] Traverse all elements in the second cache, and multiply them with the MVP matrix to obtain normalized bounding box coordinates, judge whether each component of each bounding box coordinate is in the range of [-1, 1], if one component is in, put the element into the third cache.

[0013] Further, the construction process of the inspection navigation map is as follows:

[0014] Collect laser SLAM positioning data through multiple sensors, generate high-precision points according to the laser SLAM positioning data, splice the high-precision points in order, take out the repeated part, and generate a high-precision trajectory;

[0015] According to the initial inspection trajectory, generate a sparse trajectory, compare and combine the high-precision trajectory with the sparse trajectory, and generate an inspection navigation trajectory,

[0016] Mark the start point coordinates, end point coordinates, intersection type and intersection direction in the inspection navigation trajectory, and construct an inspection navigation map.

[0017] Preferably, the generation process of the high-precision running trajectory is as follows:

[0018] Obtain the data collected by the sensors of the robot inspection vehicle in the last inspection process, and load the inspection trajectory of the last inspection;

[0019] Convert the data collected by all sensors into positioning, convert it to the body coordinate system of the inspection vehicle, calculate the distance difference between the visual positioning and the SLAM positioning of the camera shooting, and judge whether the visual positioning is normal;

[0020] If the visual positioning is abnormal, record the coordinates of the abnormal position, use the roadside equipment to confirm, update and repair the high-precision running trajectory according to the confirmation result, and use the repaired high-precision running trajectory as the inspection running path.

[0021] Further, the process of judging whether the visual positioning is normal is as follows:

[0022] Combine and splice the segmented graphics in the visual positioning, match the spliced combined image with the marked points in the inspection navigation map, and judge whether it is consistent;

[0023] If it is consistent, calculate the distance deviation between the marked points in the combined image, and judge whether the visual positioning is normal according to the calculation result.

[0024] A robot intelligent inspection system based on SLAM navigation and multi-sensor fusion, comprising:

[0025] a sensor positioning module mounted on the robot inspection vehicle, configured to perform SLAM positioning, visual positioning, ultrasonic positioning, and route data collection;

[0026] a navigation map construction module configured to construct an inspection navigation map by combining the data collected by the sensor positioning module with a SLAM navigation algorithm;

[0027] a trajectory generation module configured to determine a running source point on the inspection navigation map and generate a high-precision running trajectory according to the running source point;

[0028] an inspection control module configured to regularize an inspection running path according to the high-precision running trajectory and control the robot inspection vehicle to perform intelligent inspection according to the inspection running path.

[0029] Preferably, the collected route data needs to be filtered by a three-stage filtering method, and the process is as follows:

[0030] The collected route data is calculated to obtain the longitude and latitude of the four points where the view frustum and the terrain intersect, a first bounding box parallel to the coordinate axis is formed by the four points, the tiles contained in the first bounding box are calculated, and the tile data is extracted and placed in a first cache;

[0031] Each element in each tile is traversed, and it is determined whether the second bounding box of each element intersects with the first bounding box. If they intersect, the element is placed in a second cache;

[0032] All elements in the second cache are traversed, multiplied by an MVP matrix to obtain normalized bounding box coordinates, and it is determined whether each component of each bounding box coordinate is within the range of [-1, 1]. If one component is within the range, the element is placed in a third cache.

[0033] Further, the construction process of the inspection navigation map is as follows:

[0034] Laser SLAM positioning data is collected by multiple sensors, high-precision points are generated according to the laser SLAM positioning data, the high-precision points are spliced in order, the repeated parts are removed, and a high-precision trajectory is generated;

[0035] A sparse trajectory is generated according to the inspection trajectory of the initial inspection, the high-precision trajectory is compared and combined with the sparse trajectory to generate an inspection navigation trajectory,

[0036] The starting point coordinates, end point coordinates, intersection type, and intersection direction are marked in the inspection navigation trajectory, and an inspection navigation map is constructed.

[0037] Preferably, the generation process of the high-precision running trajectory is as follows:

[0038] Acquire the data collected by sensors of the robot inspection vehicle in the last inspection process, and load the inspection track of the last inspection;

[0039] Convert the data collected by all sensors into positioning conversion, convert them into the coordinate system of the inspection vehicle body, calculate the distance difference between the visual positioning and the SLAM positioning of the camera shooting, and determine whether the visual positioning is normal;

[0040] If the visual positioning is abnormal, record the coordinates of the abnormal position, use the roadside equipment to confirm, update and repair the high-precision running track according to the confirmation result, and use the repaired high-precision running track as the inspection running path.

[0041] Further, the process of determining whether the visual positioning is normal is as follows:

[0042] The segmented graphics in the visual positioning are combined and spliced, the spliced combined image is matched with the labeled points in the inspection navigation map, and it is determined whether it is consistent or not.

[0043] If it is consistent, the distance deviation between the labeled points in the combined image is calculated, and it is determined whether the visual positioning is normal according to the calculation result.

[0044] Compared with the prior art, the beneficial effects of the present application are:

[0045] The present application filters the route data through a three-level filtering method, quickly excludes invisible elements, reduces the processing amount, improves the rendering efficiency and performance, effectively meets the complex map construction requirements, generates global identification for the track data, facilitates the management of the track, judges the SLAM positioning and visual positioning results through multiple positioning signals, reduces the uncertainty of visual positioning, optimizes the inspection track, and enables the robot inspection vehicle to stably inspect in complex areas. BRIEF DESCRIPTION OF DRAWINGS

[0046] In the drawings:

[0047] Figure 1 The method steps of the embodiments of the present application are shown in the figure;

[0048] Figure 2 The system structure diagram of the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Example

[0050] like Figure 1 As shown, a robot intelligent inspection method based on SLAM navigation and multi-sensor fusion includes the following steps:

[0051] The robot inspection vehicle, equipped with multiple sensors, is manually operated to conduct the initial inspection of the target inspection area.

[0052] A multi-sensor data collection vehicle collects route data, which is then combined with SLAM navigation algorithms to construct an inspection navigation map.

[0053] After collecting route data, it needs to be filtered using a three-level filtering method, as follows:

[0054] The collected route data is calculated to determine the latitude and longitude of the four points where the view frustum and the terrain intersect. The four points form a first bounding box parallel to the coordinate axis. The tiles contained in the first bounding box are calculated, and the tile data is extracted and placed into the first buffer.

[0055] Iterate through the elements in each tile and determine whether the second bounding box of each element intersects with the first bounding box. If they intersect, put the element into the second buffer.

[0056] Iterate through all elements in the second cache and multiply them by the MVP matrix to obtain normalized bounding box coordinates. Determine if each component of each bounding box coordinate is within the range [-1, 1]. If any component is within the range, put the element into the third cache.

[0057] The process of constructing the inspection navigation map is as follows:

[0058] Laser SLAM positioning data is collected by multiple sensors, high-precision points are generated based on the laser SLAM positioning data, and the high-precision points are stitched together in sequence. Repeated parts are removed to generate a high-precision trajectory.

[0059] A thinned trajectory is generated based on the initial inspection trajectory. The high-precision trajectory is then compared and combined with the thinned trajectory to generate the inspection navigation trajectory.

[0060] Mark the starting point coordinates, ending point coordinates, intersection type, and intersection direction in the inspection navigation trajectory to construct an inspection navigation map.

[0061] Determine the operating source point on the inspection navigation map, and generate a high-precision operating trajectory based on the operating source point;

[0062] The process of generating a high-precision running trajectory is as follows:

[0063] The robot inspection vehicle acquires data collected by its sensors during the previous inspection and then loads the inspection trajectory from the previous inspection.

[0064] The data collected by all sensors is converted to the coordinate system of the inspection vehicle, the distance difference between the visual positioning and the SLAM positioning of the camera shooting is calculated, and whether the visual positioning is normal is judged;

[0065] If the visual positioning is abnormal, the coordinates of the abnormal position are recorded, the roadside equipment is used for confirmation, the high-precision running track is updated and repaired according to the confirmation result, and the repaired high-precision running track is used as the inspection running path.

[0066] The process of judging whether the visual positioning is normal is as follows:

[0067] The segmented graphics in the visual positioning are combined and spliced, the spliced combined image is matched with the labeled points in the inspection navigation map, and whether it is consistent is judged.

[0068] If it is consistent, the distance deviation between the labeled points in the combined image is calculated, and whether the visual positioning is normal is judged according to the calculation result.

[0069] According to the high-precision running track, the robot inspection vehicle carries out intelligent inspection according to the inspection running path. Embodiment

[0070] As shown in Figure 2 A robot intelligent inspection system based on SLAM navigation and multi-sensor fusion, comprising:

[0071] The sensor positioning module is installed on the robot inspection vehicle and is used for SLAM positioning, visual positioning, ultrasonic positioning and route data collection; after collecting the route data, the data needs to be filtered by a three-level filtering method, and the process is as follows:

[0072] The collected route data is calculated, the longitude and latitude of the four points intersected by the view frustum and the terrain are calculated, a first bounding box parallel to the coordinate axis is formed by the four points, the tiles contained in the first bounding box are calculated, the tile data is extracted and put into the first cache;

[0073] Traverse each element in each tile, judge whether the second bounding box of each element intersects with the first bounding box, if it intersects, put the element into the second cache;

[0074] Traverse all elements in the second cache, and multiply them with the MVP matrix to get the normalized bounding box coordinates, judge whether each component of each bounding box coordinate is in the range of [-1, 1], if one component is in, put the element into the third cache.

[0075] The navigation map construction module is used for combining the data collected by the sensor positioning module with the SLAM navigation algorithm to construct the inspection navigation map; the construction process of the inspection navigation map is as follows:

[0076] The laser SLAM positioning data is collected by the multi-sensor, high-precision points are generated according to the laser SLAM positioning data, the high-precision points are spliced in sequence, the repeated parts are taken out, and a high-precision trajectory is generated;

[0077] The sparse trajectory is generated according to the initial inspection trajectory, the high-precision trajectory is compared and combined with the sparse trajectory, and an inspection navigation trajectory is generated,

[0078] The start point coordinates, end point coordinates, intersection types and intersection directions are marked in the inspection navigation trajectory, and an inspection navigation map is constructed.

[0079] The trajectory generation module is used to determine a running source point on the inspection navigation map, and generate a high-precision running trajectory according to the running source point; the generation process of the high-precision running trajectory is as follows:

[0080] Data collected by sensors in the last inspection process of the robot inspection vehicle is obtained, and the inspection trajectory of the last inspection is loaded;

[0081] The data collected by all sensors is converted by positioning, and is converted into the body coordinate system of the inspection vehicle, the distance difference between the visual positioning and the SLAM positioning of the camera shooting is calculated, and whether the visual positioning is normal is judged;

[0082] If the visual positioning is abnormal, the coordinates of the abnormal position are recorded, the roadside equipment is used for confirmation, the high-precision running trajectory is updated and repaired according to the confirmation result, and the repaired high-precision running trajectory is used as the inspection running path.

[0083] The process of judging whether the visual positioning is normal is as follows:

[0084] The segmented graphics in the visual positioning are combined and spliced, the combined image after splicing is matched with the marked points in the inspection navigation map, and whether it is consistent is judged;

[0085] If it is consistent, the distance deviation between the marked points in the combined image is calculated, and whether the visual positioning is normal is judged according to the calculation result.

[0086] The inspection control module is used to regularize the inspection running path according to the high-precision running trajectory, and control the robot inspection vehicle to intelligently inspect according to the inspection running path.

[0087] The above-described embodiments only express the preferred embodiments of the present application, and the description is more specific and detailed, but it cannot be understood as limiting the scope of the patent of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications, improvements and substitutions can be made, which are within the scope of protection of the present application.

Claims

1. A robot intelligent inspection method based on SLAM navigation and multi-sensor fusion, characterized in that, The method comprises the following steps: manually operating a robot inspection vehicle equipped with multiple sensors to perform initial inspection in a target inspection area; collecting route data using the multi-sensor collection vehicle, and constructing an inspection navigation map by combining a SLAM navigation algorithm; determining a running source point on the inspection navigation map, and generating a high-precision running track according to the running source point; after collecting the route data, performing data filtering by a three-stage filtering method, and the process is as follows: calculating the collected route data, calculating the longitude and latitude of the four points intersected by the view frustum and the terrain, forming a first bounding box parallel to the coordinate axis from the four points, calculating the tiles contained in the first bounding box, extracting tile data, and placing it in the first cache; traversing each element in each tile to determine whether the second bounding box of each element intersects the first bounding box, and if it does, placing the element in the second cache; traversing all elements in the second cache and multiplying them by the MVP matrix to obtain normalized bounding box coordinates, and determining whether each component of each bounding box coordinate is within the range of [-1, 1], and if so, placing the element in the third cache normalizing the inspection running path according to the high-precision running track, and the robot inspection vehicle intelligently inspects according to the inspection running path. 2.The robot intelligent inspection method based on SLAM navigation and multi-sensor fusion of claim 1, wherein, The construction process of the inspection navigation map is as follows: collecting laser SLAM positioning data through multiple sensors, generating high-precision points according to the laser SLAM positioning data, splicing the high-precision points in order, removing the repeated parts, and generating a high-precision track; generating a sparse track according to the inspection track of the initial inspection, comparing and combining the high-precision track with the sparse track to generate an inspection navigation track, labeling the start point coordinates, end point coordinates, intersection type and intersection direction in the inspection navigation track, and constructing the inspection navigation map. 3.The robot intelligent inspection method based on SLAM navigation and multi-sensor fusion of claim 1, wherein, The generation process of the high-precision running track is as follows: obtaining the data collected by the sensors of the robot inspection vehicle during the last inspection, and loading the inspection track of the last inspection; performing positioning conversion on all sensor collected data, converting it to the body coordinate system of the inspection vehicle, calculating the distance difference between the visual positioning and the SLAM positioning of the camera shooting, and determining whether the visual positioning is normal; if the visual positioning is abnormal, recording the coordinates of the abnormal position, using roadside equipment for confirmation, and updating and repairing the high-precision running track according to the confirmation result, and using the repaired high-precision running track as the inspection running path.

4. The robot intelligent inspection method based on SLAM navigation and multi-sensor fusion according to claim 3, characterized in that, The process of determining whether the visual positioning is normal is as follows: combining and splicing the segmented graphics in the visual positioning, matching the spliced combined image with the labeled points in the inspection navigation map, and determining whether it is correct; if it is correct, calculate the distance deviation between the labeled points in the combined image, and determine whether the visual positioning is normal according to the calculation result.

5. A robot intelligent inspection system based on SLAM navigation and multi-sensor fusion, characterized in that, It comprises: a sensor positioning module installed on the robot inspection vehicle, used for SLAM positioning, visual positioning, ultrasonic positioning and route data collection, and after collecting the route data, data filtering is performed by a three-stage filtering method, and the process is as follows: The collected route data is calculated to obtain the longitude and latitude of four points intersected by the view frustum and the terrain, a first bounding box parallel to the coordinate axes is formed by the four points, tiles contained in the first bounding box are calculated, tile data is extracted and placed in a first cache; Elements in each tile are traversed, and it is judged whether the second bounding box of each element intersects the first bounding box. If the first bounding box is intersected, the element is placed in a second cache; All elements in the second cache are traversed, and multiplied by the MVP matrix to obtain normalized bounding box coordinates. It is judged whether each component of each bounding box coordinate is within the range of [-1, 1]. If one component is within the range, the element is placed in a third cache The navigation map construction module is used to combine the data collected by the sensor positioning module with the SLAM navigation algorithm to construct the inspection navigation map. The trajectory generation module is used to determine the running source point on the inspection navigation map and generate a high-precision running trajectory according to the running source point. The inspection control module is used to regularize the inspection running path according to the high-precision running trajectory and control the robot inspection vehicle to intelligently inspect according to the inspection running path. 6.The robot intelligent inspection system based on SLAM navigation and multi-sensor fusion of claim 5, wherein, The construction process of the inspection navigation map is as follows: Laser SLAM positioning data is collected by multiple sensors, high-precision points are generated according to the laser SLAM positioning data, the high-precision points are spliced in order, the repeated parts are removed, and a high-precision trajectory is generated; A sparse trajectory is generated according to the inspection trajectory of the initial inspection, the high-precision trajectory is compared and combined with the sparse trajectory to generate an inspection navigation trajectory, The starting point coordinates, end point coordinates, intersection type and intersection direction are marked in the inspection navigation trajectory, and the inspection navigation map is constructed. 7.The robot intelligent inspection system based on SLAM navigation and multi-sensor fusion of claim 5, wherein, The generation process of the high-precision running trajectory is as follows: The data collected by the sensors of the robot inspection vehicle in the last inspection process is obtained, and the inspection trajectory of the last inspection is loaded; The data collected by all sensors is converted by positioning and converted into the body coordinate system of the inspection vehicle, the distance difference between the visual positioning and the SLAM positioning of the camera shooting is calculated, and it is judged whether the visual positioning is normal. If the visual positioning is abnormal, the coordinates of the abnormal position are recorded, the roadside equipment is used for confirmation, the high-precision running trajectory is updated and repaired according to the confirmation result, and the repaired high-precision running trajectory is used as the inspection running path. 8.The robot intelligent inspection system based on SLAM navigation and multi-sensor fusion of claim 7, wherein, The process of judging whether the visual positioning is normal is as follows: The segmented graphics in the visual positioning are combined and spliced, the spliced combined image is matched with the marked points in the inspection navigation map, and it is judged whether it is consistent; If it is consistent, the distance deviation between the marked points in the combined image is calculated, and it is judged whether the visual positioning is normal according to the calculation result.

Images