ROS-based laser radar and point cloud image processing fused multi-point navigation and target detection algorithm

Abstract

The invention provides a multi-point navigation and target detection algorithm based on laser radar and point cloud image processing fusion of an ROS operating system, and relates to the technical field of mobile robot navigation and target detection. The algorithm is based on an ROS operating system and is composed of a laser radar, a data storage and management module, a point cloud image processing module, a traversal and path planning module and a navigation and target detection module. The laser radar scans the environment in real time and transmits data to the ROS operation system to generate point cloud map data; the image processing module is used for filtering the point cloud image stored in the navigation process, so that the accuracy of the map is improved; and the traversal and path planning module calculates the parking position and direction of the robot according to the processed point cloud map data and performs path planning, thereby realizing accurate navigation and target detection. The method has the advantages of real-time map generation and storage, point cloud image processing and calculation, accurate path planning, strong adaptability and the like, and is suitable for industrial automation, unmanned warehouses and logistics centers, search and rescue, agricultural application, intelligent inspection, unmanned driving and other complex or dynamic unknown environments.

Description

Technical Field

[0001] The present invention relates to the field of mobile robot navigation and target detection technology, and in particular to a multi-point navigation and target detection algorithm based on ROS-based laser radar and point cloud image processing fusion. Background Art

[0002] With the widespread application of mobile robots in industries such as industry, logistics, and services, the demand for autonomous navigation and precise target detection in complex environments is increasing. Most existing navigation methods rely on a single sensor (such as lidar or camera), which may not be able to meet the requirements of high-precision navigation and target detection in complex environments. Therefore, navigation algorithms based on multi-sensor data fusion have become the focus of current research, especially the fusion of lidar and point cloud image processing technology to improve the robustness and accuracy of navigation.

[0003] ROS operating system is an open source operating system widely used in the field of robotics. It provides a rich set of tools and function packages to support a variety of tasks such as robot environment mapping, path planning, and autonomous navigation. The modular design of the ROS operating system allows developers to easily integrate different sensors and algorithms to achieve complex robot functions. However, in the ROS operating system environment, how to efficiently manage, process, and store a variety of sensor data to ensure reliable navigation and target detection of mobile robots is still a difficult problem that current technology needs to solve.

[0004] As a key means of environmental perception, point cloud data plays a vital role in robot navigation and target detection. The point cloud data generated by LiDAR can accurately capture information about obstacles and target objects in the environment. On this basis, combined with image processing algorithms, target features can be further extracted and path planning can be performed. However, how to efficiently save and process multiple point cloud images during the movement of mobile robots, and how to achieve multi-point navigation and target detection based on these data, is an important technical problem that needs to be solved at present. Summary of the invention

[0005] The purpose of the present invention includes providing a multi-point navigation and target detection algorithm based on the fusion of ROS laser radar and point cloud image processing, which can improve the problems of low robustness and poor accuracy of mobile robots when using existing single sensors, and at the same time improve the use effect in complex environments.

[0006] The embodiments of the present invention can be implemented as follows:

[0007] An embodiment of the present invention provides a data storage method, which can obtain and save point cloud data of a local map from a ROS operating system every t seconds, and process the data before starting navigation.

[0008] The embodiment of the present invention provides a data management method, which can generate an accurate point cloud image of an area of ​​interest by synthesizing multiple point cloud data and performing initial image processing, thereby providing reliable data support for multi-point navigation.

[0009] The embodiment of the present invention provides a point cloud image processing algorithm, which can obtain the accurate position and normal direction of each target plate and generate a high-precision two-dimensional environment model.

[0010] An embodiment of the present invention provides a parking position calculation method. According to the position of a target board, the parking position and direction of a mobile robot in a maximum parking area in front of the target board can be determined by calculation. The parking position and direction have high accuracy and are also applicable to complex environments.

[0011] An embodiment of the present invention provides a traversal and path planning method, which determines the optimal parking position and direction of a mobile robot in front of a target board based on point cloud image processing results, screens out valid target boards after traversing each target board, performs path planning based on two factors: the time consumed and the moving distance of the mobile robot, and generates a navigation profile.

[0012] An embodiment of the present invention provides a navigation method, in which a mobile robot makes decisions from an initial position to a parking position in front of a valid target board one by one through an autonomous multi-point navigation algorithm according to the target of a navigation configuration file, and continues to complete the current autonomous multi-point navigation task after making the decision.

[0013] An embodiment of the invention provides a target detection method. After the mobile robot reaches a parking position in front of a valid target board, image target detection is performed. When the mobile robot reaches each position, the image information on the target board is identified, and corresponding decisions are made according to the image detection results.

[0014] Compared with the prior art, the present invention has the following advantages: 1. The present invention significantly improves the navigation robustness and target detection accuracy of mobile robots in complex environments by integrating lidar and point cloud image processing technology. 2. A ROS-based timed data storage and management method was introduced, which can synthesize and process multiple point cloud data, generate accurate environmental models, and provide reliable support for multi-point navigation and target detection. 3. The present invention proposes an accurate parking position calculation and path planning method to ensure that the mobile robot can efficiently navigate to multiple target locations in a complex environment and accurately perform target detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 These are 10 point cloud data images acquired at time t.

[0016] Figure 2 This is a point cloud image synthesized from 10 point cloud data.

[0017] Figure 3 This is the image after the point cloud image is processed for the first time.

[0018] Figure 4 This is the image of the parking position and direction after image processing.

[0019] Figure 5 Schematic diagram of navigation and path planning. DETAILED DESCRIPTION

[0020] The present invention is described below in conjunction with the accompanying drawings and specific embodiments.

[0021] The application of this algorithm is based on the ROS operating system, but is not limited to it. Basic functional packages such as radar, mapping, navigation, and robot body drive should be installed in the workspace. During the movement, the laser radar scans the surrounding environment in real time, collects obstacle distances to generate point cloud data, and builds global cost maps and local cost maps in real time for path planning and navigation. Configure the relevant parameters of the configuration files such as radar, map, and navigation to ensure that the sensors of the mobile robot achieve the best effect. Among them, the more critical parameter settings include the global coordinate system in the local cost map, the base coordinate system of the mobile robot body, the update frequency of the local map, the release frequency of the local map, the length and width of the local cost map, etc.

[0022] The point cloud data storage and management methods of LiDAR are as follows: During the movement of the mobile robot, we first convert the map data from the world coordinate system to the map coordinate system, set and initialize a node for saving point cloud data: "local_saver_node", subscribe to the topic published by the node " / move_base / global_costmap / costmap" for creating a local map in the ROS operating system, and obtain the range and frequency of the point cloud data and pass the received point cloud data message to the callback function, so that the point cloud data of the local map can be obtained.

[0023] Starting from the command to save the point cloud data image, every t seconds, the point cloud data of the local map obtained from the ROS operating system is saved and converted into an image format. The converted image is saved locally and the image counter is updated at the same time.

[0024] like Figure 1As shown in the figure, after receiving n point cloud images, these point cloud images are subjected to initial image processing. The processing steps include rotating the point cloud image 90 degrees counterclockwise, and then performing a bitwise AND operation on the n point cloud images (performing a binary bitwise AND operation on each pixel value of the image) to generate a synthetic final point cloud image. This processing method helps to extract consistent features and areas in the map, such as obstacles or free space.

[0025] like Figure 2 As shown in the figure, the final synthesized point cloud image will be saved to the image file in the specified path. Then the ROS node will be closed and the program will end. This process ensures that when a sufficient number of point cloud images are saved, the program will automatically stop to prevent it from running indefinitely.

[0026] In order to ensure that there is no thread contention problem when processing map data, thread locks are used in the process of saving point cloud data, which can ensure the consistency and security of data when multiple threads access and modify data at the same time.

[0027] The point cloud image processing algorithm is as follows: The coordinates and direction information of the parking position are generated by image processing and geometric calculation of the point cloud image, and the restricted areas in the mobile robot environment, the size of the mobile robot itself, the map edge and obstacles are taken into account to select the position and direction that meet the parking requirements of the mobile robot. The output is in the form of plane coordinates (x, y) and spatial quaternion posture (qx, qy, qz, qw), which facilitates the accurate navigation of the mobile robot.

[0028] Quaternions are used in the algorithm to represent the direction of the target point. Compared with Euler angles or other representations, quaternions can avoid singularity problems and can more accurately represent the target direction. This is especially important for robots to navigate in complex environments.

[0029] like Figure 3 As shown in the figure, the final synthesized point cloud image is read from the specified path, and the image is mirrored and dilated. The purpose of mirroring is to make the coordinate system of the point cloud image consistent with the set map coordinate system; the purpose of the dilation operation is to enhance the target features in the image, remove small noise and isolated pixels in the image due to sensor errors or other external interference, make the target area more obvious, and expand the bright spots (obstacles and areas of interest) in the image to facilitate subsequent target plate recognition.

[0030] like Figure 4As shown in the figure, the image is binarized and converted into a binary image. By performing a secondary dilation operation on the binary image, the features in the image are further enhanced and the target area is shrunk, making the edge of the target area clearer and more precise. These contours represent possible target areas in the image.

[0031] Next, the image is subjected to contour detection to identify possible target parking areas. For each detected contour, the target parking area is screened by setting parameters such as aspect ratio and length threshold. Only contour areas whose aspect ratio and length threshold meet the characteristics of the target plate are retained to filter out non-target parking areas that are too large or too small, and retain contours that meet the target size. Figure 4 As shown, after screening, the circumscribed outline of the target plate is made, represented by a green rectangle, and its center point position is calculated.

[0032] The algorithm for obtaining the parking position of the mobile robot is as follows: For each contour that passes the filter, Figure 4 As shown by the red straight line, the perpendicular bisector passing through the center point and perpendicular to its long side is calculated, and two normal vectors in opposite directions are generated. and Represents the direction of the mobile robot in the target parking area.

[0033] like Figure 4 As shown in the yellow rectangle, the maximum area that can be parked in front of the target board is calculated; Figure 4 As shown by the red dot, the center position of the maximum area where the mobile robot can park is calculated.

[0034] The mobile robot determines whether the maximum area in front of the target board where the robot can park falls partially or entirely in the restricted area S, outside the overall environment area, or other designated areas. To this end, coordinate transformation and area judgment are used to check whether the edge of the target area is within these areas. If the target area intersects these areas, these parking positions will be filtered out to avoid using them as target points, which will affect the path planning and navigation of the mobile robot.

[0035] like Figure 4 As shown in the figure, for those valid parking areas that have passed the screening, in order to facilitate target detection, the detection distance is slightly away, the center position of the mobile robot is represented by a purple dot, and the center position coordinates (x, y) and orientation (represented by spatial quaternion attitude (qx, qy, qz, qw) of the mobile robot when it is parked are saved as the parking target G. Finally, the center position coordinates and spatial quaternion attitude data of the mobile robot are saved in a specified text file for subsequent robot navigation.

[0036] The path planning of the mobile robot is as follows: like Figure 5 As shown in the figure, the black dot is the center position of the mobile robot when it is parked. The path planning is performed based on the position advantage of the mobile robot in different scenes. When planning the path, not only the distance the mobile robot moves should be considered, but also the time factor spent by the mobile robot when moving. The target board here is large in size, and it needs to travel a long distance when detecting the front and back sides in sequence. The robot will also consume a lot of time when stopping. Therefore, it is proposed to perform path planning based on the vertical distance from the coordinate axis direction, which can not only reduce the distance the robot moves, but also reduce the time consumed by the mobile robot when moving.

[0037] like Figure 5 As shown, let the lower left corner of the image be the coordinate system point O(0,0), the right is the positive direction of the x-axis, and the right is the positive direction of the y-axis. Starting from the initial position ①, all valid parking areas are sorted according to the size of the coordinates of the center position of the mobile robot from the x-axis or y-axis coordinates in each valid parking area. The green line is the path planning method based on the vertical y-axis distance from large to small. And write these parking center position coordinates and spatial quaternion posture information into the specified text file.

[0038] The point positions and spatial quaternion attitude information of the path gauge are made into a navigation configuration file in a data format that can be directly used by the navigation algorithm of the mobile robot. It contains the number of navigation points n (here 0), the position coordinates of each navigation point (x1, y1), ..., (xn, yn) and the quaternion attitude of each point mobile robot is facing (qx1, qy1, qz1, qw1), ..., (qxn, qyn, qzn, qwn).

[0039] Mobile robot navigation and target detection are as follows: The mobile robot reads multiple navigation target points from the specified navigation configuration file. The target point information includes the center position coordinates (x, y) and the spatial quaternion posture (qx, qy, qz, qw) of the mobile robot when it is parked. Multiple target points are stored in the file, and the code extracts these points in sequence according to the number of navigation points n read. According to the target point information in the navigation configuration file, the map coordinate system is used as the reference coordinate system, and the move_base node is used to control the mobile robot to perform autonomous multi-point navigation.

[0040] If the target location is not reached within 20 seconds, the mobile robot will adjust the path planning, wait briefly or retry, cancel and transfer the target, etc. according to the current surrounding environment conditions to continue navigating to the next target. If it reaches a valid parking area, the mobile robot will execute the target detection script. When it reaches each valid parking area, it will identify the image information on the target board and make corresponding decisions. After the decision is made, it will continue to complete the current autonomous multi-point navigation task until the navigation task is interrupted or terminated.

[0041] The above description is only the preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications, combinations and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the claims of the present invention.

Claims

1. A multi-point navigation and target detection algorithm based on ROS-based laser radar and point cloud image processing fusion, characterized by: Under the ROS operating system, the mobile robot uses the lidar to scan the environment in real time and collect distance data to generate a local map; then, the data storage and management module saves the collected local maps at fixed intervals and calls these maps for processing after the navigation is completed; then, the point cloud image processing module processes the saved local maps to parse the position and normal direction of each target plate; then, the traversal and path planning module determines the parking position and direction of the mobile robot in front of the baffle according to the calculated position, performs traversal and path planning, and generates a configuration file for the target position and posture; finally, the navigation and target detection module controls the mobile robot to perform autonomous navigation according to the results of path planning, starting from the initial position, accurately reaches the target position through the navigation points of path planning, and performs target detection.

2. The mobile robot according to claim 1, characterized in that: The installed ROS operating system should have a workspace, and the function package directory of the workspace should also include basic function packages such as radar, mapping, navigation, and robot body driver.

3. The mobile robot according to claim 1, characterized in that: During the movement, the surrounding environment is scanned in real time by lidar, obstacle distances are collected to generate point cloud data, and global and local cost maps are constructed in real time for path planning and navigation.

4. The local cost map according to claim 3, characterized in that: Relevant parameters should be configured for the local map, including the global coordinate system in the local cost map, the base coordinate system of the mobile robot body, the update frequency of the local map, the publishing frequency of the local map, the length and width of the local cost map, etc.

5. The data storage and management module according to claim 1, characterized in that: Subscribe to topics published by relevant nodes in the ROS operating system to obtain map-related messages, and process and save the obtained messages.

6. The method of obtaining map-related information from the ROS operating system according to claim 5, characterized in that: Initialize the 'local_saver_node' node, subscribe to the topic published by ' / move_base / global_costmap / costmap' in the ROS operating system, and pass the received map message to the callback function.

7. According to claim 5, data processing and storage of the acquired messages from the ROS operating system is characterized in that: The timer periodically saves the received map messages, converts the points in the world coordinate system into points in the map coordinate system, and saves them as point cloud image files for subsequent use.

8. The point cloud image processing module according to claim 1, characterized in that: During the movement of the mobile robot, several point cloud image files are saved at regular intervals, point operations are performed to synthesize them into an image, and then relevant image processing algorithms are used to improve the accuracy of the image.

9. The point cloud image file according to claim 8, characterized in that: In the point cloud image, the pixel color of the obstacle point is black, and the pixel color of the non-obstacle point is white.

10. The method for periodically saving a plurality of point cloud image files according to claim 8, characterized in that: During the movement of the mobile robot, a point cloud image will be saved every time t until the point cloud image saving task is completed. Point cloud images at different times and locations are saved, with a total of n point cloud images.

11. The point operation synthesis into one image according to claim 8, characterized in that: Based on the map coordinate system, the power data is initially processed, and the n saved point cloud images are bitwise ANDed to finally obtain the point cloud image of the area of ​​interest.

12. The correlation image processing algorithm according to claim 8, characterized in that: Image processing algorithms such as cropping, rotation, and threshold filtering are used to reduce noise interference and obtain more accurate point cloud images of the area of ​​interest.

13. The method for analyzing the position and normal direction of each target plate according to claim 1, characterized in that: There are several target boards in the area, among which the valid target board has an image target. The mobile robot needs to park in front of the valid target board and perform image target detection. The saved point cloud image is processed using an image processing algorithm to calculate the parking position and the normal direction of the target board.

14. The traversal and path planning module according to claim 1, characterized in that: Each target board has two parking positions, positive and negative. Taking into account the size of the mobile robot and the obstacles such as restricted area S and walls in the area, all legal parking positions (x, y) can be obtained by removing unreasonable positions. The direction of the normal is converted into the plane coordinate system angle and then into the spatial quaternion posture (qx, qy, qz, qw). The parking position and direction of the mobile robot in front of the target board are determined according to the calculated position.

15. The path planning and traversal according to claim 1, characterized in that: Using the calculated parking position (x, y) and the spatial quaternion posture (qx, qy, qz, qw) of the orientation, a vertical distance shortest first algorithm is proposed for path planning. Each possible parking position is traversed, the vehicle stops at the parking position and performs image target detection.

16. The configuration file for generating a target position and orientation according to claim 1, characterized in that: The position and direction information of the path gauge is made into a navigation configuration file, which includes the number of navigation points n, the position coordinates of each navigation point (x1, y1),…, (xn, yn) and the quaternion posture of the mobile robot at each point (qx1, qy1, qz1, qw1),…, (qxn, qyn, qzn, qwn).

17. The navigation and target detection module according to claim 1, characterized in that: According to the target point information in the navigation configuration file, use the move_base node to control the mobile robot to perform autonomous multi-point navigation. Starting from the initial position, it reaches the set positions one by one and executes the target detection script. When it reaches each position, it will identify the image information on the target board and make corresponding decisions. After the decision is made, it will continue to complete the current autonomous multi-point navigation task.

Images