A method, system and device for detecting the adaptive drivable area of ​​an intelligent wheelchair

Abstract

The present invention provides a method for detecting an adaptive drivable area of ​​an intelligent wheelchair, comprising: obtaining a three-dimensional point cloud in a wheelchair driving environment through a laser radar; performing voxel filtering on the obtained three-dimensional point cloud data and dividing the area of ​​interest; dividing the polar coordinate grid, and segmenting the ground point cloud and the non-ground point cloud; dividing the grid from fine to coarse, and calculating the shortest distance of the non-ground point cloud in each direction of the laser radar, and further determining the drivable grid; splicing the above grids to obtain the drivable area. The present invention requires very few computing resources and can obtain more reliable results; it can be well applied to wheelchairs and is more adaptable to various complex unstructured driving environments.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent wheelchair environmental perception, and in particular relates to an intelligent wheelchair adaptive drivable area detection method, system and equipment. Background Art

[0002] In recent years, ground mobile robot technology has gradually been promoted and applied in wheelchairs, especially robot autonomous navigation technology has gradually become an essential capability of robot wheelchairs. Robot wheelchairs must travel within the drivable area. The concept of drivable area refers to the area where the intelligent mobile platform can travel freely. Compared with smart cars, the unstructured road environment for wheelchairs is more challenging than the structured environment. The detection of drivable areas in unstructured roads also faces many difficulties, such as the lack of high-precision maps, the need to detect pedestrians, animals, non-motor vehicles and other fixed obstacles around in real time, irregular terrain and unstructured road boundaries and other complex driving environments to ensure safety during driving.

[0003] The methods for detecting drivable areas that have been developed can be divided into three categories according to the type of sensor: 1) Vision-based methods, which process scene information through traditional image processing or deep learning methods, but this method performs poorly due to the unstructured road boundaries, complex driving environment, and easy to be affected by light, rain and snow; 2) LiDAR-based methods, LiDAR can accurately measure and perceive the surrounding environment information, and is less affected by light; 3) Methods that integrate visual information and LiDAR point cloud information, which have good calculation results, but due to factors such as volume and cost, it is not convenient to install many sensors and computing resources on wheelchairs. The patent "Drivable Area Detection Method, Device, Computer Equipment and Storage Medium (201910484995.6)" proposes a drivable area detection algorithm, which realizes the detection of the drivable area of ​​the intelligent driving device by sectorizing the point cloud information collected by LiDAR and detecting the drivable points in the sector area, but this method has certain shortcomings: this method cannot adapt well to the complex environment such as small slopes and small pits in the unstructured roads where wheelchairs travel. The method proposed in this patent can adapt well to unstructured roads for wheelchairs to travel on, and can also detect small ramps, local ramps or potholes in the road well, and accurately divide the free area where the wheelchair can travel. Summary of the invention

[0004] In order to solve the problems that the existing drivable area detection algorithm requires a lot of computing resources and cannot be well applied in edge devices; the traditional drivable area detection algorithm has low recognition accuracy in unstructured roads. The present invention proposes an intelligent wheelchair adaptive drivable area detection method, system and device, which constructs a wheelchair drivable area based on an accurate ground segmentation algorithm and spatial distance information, requires very few computing resources and can obtain more reliable results; can be well applied in wheelchairs, and is more adaptable to various complex unstructured driving environments.

[0005] A method for detecting an adaptive drivable area of ​​an intelligent wheelchair comprises acquiring a three-dimensional point cloud in a wheelchair driving environment through a laser radar; performing voxel filtering on the acquired three-dimensional point cloud data and dividing an area of ​​interest; dividing a polar coordinate grid and segmenting a ground point cloud and a non-ground point cloud; dividing the grid from fine to coarse, and calculating the shortest distance of the non-ground point cloud in each direction of the laser radar, and further determining a drivable grid; and splicing the above grids to obtain a drivable area.

[0006] The specific steps include:

[0007] Step 1: Obtain the 3D point cloud data collected by the laser radar within 360° on the smart wheelchair, divide the data into polar coordinate networks after data preprocessing, and fit the local plane;

[0008] Step 2: According to the normal vector of the fitted plane and the distance T from the point cloud in the grid to the plane h , determine the ground points in each grid; initially, the threshold set after grid division is unified. After the ground point cloud of the first frame is determined, the threshold of each grid is updated by the average distance from the point cloud divided as ground points in the previous frame to the local plane in the grid.

[0009] Step 3: Perform secondary classification on the points divided in the grid based on the height threshold of the possible ground; Step 4: Divide the grid from fine to coarse, and divide the non-ground point cloud into different fan-shaped areas;

[0010] Step 5: Calculate the distance dmin between each sector area and the nearest non-ground point, and traverse different step lengths to mark the grids with a distance less than dmin as drivable grids;

[0011] The ground point cloud segmentation method specifically includes dividing the polar coordinate grid and fitting a local plane in each grid, comparing the normal vector of the above-mentioned fitted plane with the set threshold one, and screening out the plane that meets the requirements; calculating the distance from the point cloud in the grid to the above-mentioned plane, and comparing it with the set threshold two, preliminarily dividing the ground points and non-ground points according to the threshold; updating the set threshold two of the grid according to the mean value of the distance from the point cloud that has been screened as ground points to the local plane.

[0012] Step 6: Smooth the driving grids and merge all the drivable grids to obtain the drivable area.

[0013] An intelligent wheelchair adaptive drivable area detection system comprises an acquisition module for acquiring point cloud information during the wheelchair process; a ground point cloud segmentation module for dividing the acquired point cloud into ground points and non-ground points; a grid division module for subsequently determining the wheelchair drivable area; a distance detection module for detecting the shortest distance from a non-ground point to a laser radar at each angle, and further determining the drivable grid area; a smoothing filter module for smoothing the grid determined to be drivable; and an area construction module for splicing all drivable grids to further obtain a free area where the wheelchair can pass.

[0014] An intelligent wheelchair adaptive drivable area detection device includes one or more processors S402; a memory S404, the memory is used to store the program and data of the above-mentioned drivable area detection algorithm, the processor S402 is used to execute the program in the memory, and when the computer program is executed, it can be executed in the order of the steps of the above-mentioned method. The computer program can be downloaded and installed from the network through a communication device. The processor S402, the memory S404 and the I / O interface S408 are connected to each other through a bus. The following devices are connected to the bus through the I / O interface: input device S410 includes a keyboard, a mouse, etc.; output device S412 includes a speaker, a liquid crystal display, etc.; storage part S414 includes a hard disk, etc.; and communication device S416 includes a modem, a local area network card, etc.

[0015] Beneficial effects of the present invention:

[0016] 1. Set the threshold for dynamic update: The ground point cloud changes in consecutive frames are relatively stable, and the ground height in each grid rarely changes suddenly. In order to better screen the real ground points, the threshold 2 of each ground grid in each frame is updated in real time, and the average distance from all ground points in the grid of the previous frame to the fitting plane is calculated as the basis for determining the ground points in the next frame; the dynamic update of threshold 2 helps to accurately identify the small slopes on the road surface.

[0017] 2. Secondary classification of point cloud based on possible ground height: There may be uneven road surfaces such as small potholes on semi-structured roads. When there are many obstacles in the grid, some ground points may be missed. Therefore, the point cloud after preliminary classification is screened again based on the possible height threshold of the ground point. At the same time, median filtering is used to further level the ground height. Missing ground point clouds are common due to occlusion. The missing ground units are filled, and the missing height values ​​are replaced by the median of the adjacent grids to obtain a better ground estimate.

[0018] 3. Divide the drivable grid from fine to coarse: As the distance increases, the point cloud becomes sparse and we are more concerned about the drivable area within a certain range around the wheelchair. Therefore, the drivable area grid is divided from fine to coarse. Different distances are divided into grids of different densities, which helps to better utilize computing resources and obtain a more reliable drivable area.

[0019] 4. The present invention requires very few computing resources and can obtain more reliable results; it can be well applied in wheelchairs and is more adaptable to various complex unstructured driving environments.

[0020] 5. There are two thresholds involved in the classification of ground point clouds in this method: the first is the one mentioned in step 2, and this threshold is dynamically updated; the local plane of the present invention is grid-fitted, and each grid fits its own plane and updates its own threshold. The drivable area of ​​this method is a grid with different resolutions set according to different distances. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 , flow chart of the drivable area detection algorithm of the present invention;

[0022] Figure 2 , Schematic diagram of the drivable grid division from fine to coarse;

[0023] Figure 3 , structural block diagram of the drivable area detection system of the present invention;

[0024] Figure 4 , hardware structure block diagram of the intelligent wheelchair drivable area detection device of the present invention. DETAILED DESCRIPTION

[0025] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0026] This embodiment provides a method for detecting an adaptive drivable area of ​​an intelligent wheelchair. The algorithm flow is as follows: Figure 1 As shown, the wheelchair-accessible area is constructed based on an accurate ground segmentation algorithm and spatial distance information, which requires very few computing resources and can obtain more reliable results; it can be well applied in wheelchairs and is more adaptable to various complex unstructured driving environments.

[0027] Step 1: Obtain the three-dimensional point cloud data collected by the lidar on the smart wheelchair within 360°. The point cloud data includes roads, buildings, pedestrians and other obstacles, and each point includes three-dimensional coordinates (XYZ).

[0028] Step 2: Preprocess the collected point cloud data. Specifically, use the voxelization method to reduce the number of point clouds. It will calculate a cube that can wrap the input point cloud, divide it into different voxel grids according to the set resolution, and approximate all point clouds in the voxel grid by the centroid coordinates; use the CropBox filter to select the area of ​​interest in the point cloud data.

[0029] Step 3: Polar coordinate meshing: Divide the original point cloud into a polar coordinate network of nChannels x mBins.

[0030] The height information of the lowest point in each polar coordinate grid is mapped to the cell attribute. The mapping of each point in the point cloud to the polar coordinate grid is given by the formula, where Rmin corresponds to the minimum radius that the radar can scan.

[0031]

[0032]

[0033] Step 4: Local plane fitting: Use the principal component analysis method to perform local plane fitting (Z = Ax + By + C) on the point cloud in each polar coordinate grid, and then determine the possible ground points. Specifically, the ground should be relatively flat, and planes that meet a certain normal vector threshold range are selected. The grids where these planes are located are more likely to include ground point clouds.

[0034] Step 5: Calculate the distance from all data points in the grid to the fitting plane, which will be less than the dynamic threshold T h All points are classified as ground points.

[0035] Step 6: T h Dynamic update: The ground point cloud changes of consecutive frames are relatively stable, and the ground height in each grid rarely changes suddenly. In order to better screen the real ground points, the height threshold of each frame of the ground grid is updated in real time, and the average distance from all ground points to the fitting plane in the previous frame is calculated as the height threshold T of the grid in the next frame. h , to obtain better ground estimation effect.

[0036] Step 7: Secondary screening of point cloud: The height of the laser radar in the wheelchair is a priori. Without considering the large slope of the road surface, the possible height of the ground should be lower than the height of the sensor; the possible height of the ground of adjacent grids should be close. Based on this assumption, the ground points and non-ground points in the grid segmented by principal component analysis are screened twice, and the ground points segmented as non-ground points and the non-ground points segmented as ground points are reclassified.

[0037] Step 8: Specifically, based on the threshold [Thmin ,T hmax ]Classify ground points and non-ground points based on the prior sensors

[0038] Height s , iteratively obtain the lowest point in the polar coordinate grid, called the reference point Z in the grid i , Z i Below the threshold T hmin When Set to T hmin , Z i Above the threshold T hmax When the height of the grid is updated to the height H of the sensor s , when the reference point is within the threshold, the grid height value is set to Z i . Set the grid height value to be smaller than T hmax The grids with a value smaller than the threshold are marked as ground grids. The points are marked as ground points.

[0039] Step 9: Median filtering further smoothes the ground height: Missing ground point clouds are common due to occlusion. Missing ground units are filled and the missing height values ​​are replaced by the median of the adjacent grids to obtain better ground estimates.

[0040] Step 10: Grid division from fine to coarse: As the distance increases, the point cloud becomes sparse and we are more concerned about the drivable area within a certain range around the wheelchair. Therefore, a simple calculation method is proposed to divide the grids of different densities at different distances. ij ,like Figure 2 As shown, according to different grid densities, it is divided into a central area S202, a second area S204, a third area S206 and an outer area S208.

[0041] Step 11: Point cloud projection: Taking the two-dimensional grid as the basic unit, project the non-ground point cloud after removing the ground, set a certain height threshold H as the projection constraint, and project the point cloud to the XOY two-dimensional plane.

[0042] Step 12: Build a distance detector: Divide the circular scanning area of ​​the radar into sectors θ with equal angles △θ i , the number of sectors N is shown in the formula. Each sector has a distance attribute, which indicates the shortest Euclidean distance from the wheelchair to the point cloud at that angle. The smaller the sector partition angle, the higher the accuracy of wheelchair drivable area detection.

[0043]

[0044] Step 13: Divide the point cloud into different sectors. Specifically, according to the coordinate information of the point cloud projection, calculate the angle θ with the x-axis, tan(θ) = y / x, if θ>2Π, then θ = θ-2Π; if θ<0, θ = θ+2Π, ensuring that θ is in the range of [0,2Π]. According to (θ / △θ), we get k by rounding, which belongs to the kth sector.

[0045] Step 14: Initialize the distance attributes of N sector areas to R, and traverse the point cloud point by point according to the point cloud index value, and calculate the minimum distance of each sector area using the Euclidean formula If the following formula is satisfied, the distance attribute value of the angle is replaced.

[0046]

[0047] Step 15: After traversing all points in the grid, we can get θ i The shortest distance (i=1,2,…N)

[0048] Using the step-by-step search method, we traverse different step lengths l at each angle. t , consistent with the grid division from fine to coarse, the step size increases from small to large, and the grid that satisfies the following formula is marked as a drivable grid.

[0049]

[0050] Step 16: Construct a smoothing filter. Perform a smoothing filter on the grids marked as drivable, select points around the grids in the center area S202 and the second area S204 that meet the first grid number, and if all are drivable grids, mark the grids around the second grid number as drivable grids; if not, traverse the next grid that meets the condition.

[0051] Step 17: Splice all the drivable grids to finally get an obstacle-free area where the wheelchair can travel.

[0052] This embodiment proposes a smart wheelchair adaptive drivable area detection system, the block diagram of which is as follows: Figure 3 As shown, it includes an acquisition module for collecting point cloud information during the wheelchair process; a ground point cloud segmentation module for dividing the collected point cloud into ground points and non-ground points; a grid division module for subsequently determining the area where the wheelchair can travel; a distance detection module for detecting the shortest distance from the non-ground point to the laser radar at each angle, and further determining the grid area that can be traveled; a smoothing filter module for smoothing the grid determined to be travelable; and an area construction module for splicing all the drivable grids to further obtain the free area where the wheelchair can travel.

[0053] like Figure 4 As shown, the present invention provides an adaptive drivable area detection device for an intelligent wheelchair, comprising one or more processors S402; a memory S404, the memory is used to store the program and data of the above-mentioned drivable area detection algorithm, and the processor is used to execute the program in the memory. When the computer program is executed, it can be executed in the order of the steps of the above-mentioned method. The computer program can be downloaded and installed from the network through a communication device S416. The processor, the memory and the I / O interface S408 are connected to each other through a bus S406. The following devices are connected to the bus through the I / O interface S408: the input device S410 includes a keyboard, a mouse, etc.; the output device S412 includes a speaker, a liquid crystal display, etc.; the storage part S414 includes a hard disk, etc.; and a communication device S416 including a modem, a local area network card, etc.

[0054] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above technical features.

Claims

1. A method for detecting an adaptive drivable area of ​​an intelligent wheelchair, characterized in that: The following steps are involved: Step 1: Obtain the 3D point cloud data collected by the laser radar at a height of Hs installed on the smart wheelchair within 360°, divide it into polar coordinate grids after data preprocessing, and fit the local plane grid by grid; Step 2: Based on the normal vector of the fitted plane and the distance from the point cloud in the grid to the plane , determine the ground points in each grid; initially, the threshold set after grid division is unified. After the ground point cloud of the first frame is determined, the threshold of each grid is updated by the average distance from the point cloud divided as ground points in the previous frame to the local plane in the grid; Step 3: The ground height threshold is used to perform secondary classification on the points divided in the grid; the step 3 specifically includes: Step 31: Secondary screening of the point cloud, secondary screening of the ground points and non-ground points in the grid segmented by principal component analysis, and reclassification of the ground points segmented as non-ground points and the non-ground points segmented as ground points; Step 32: Threshold-based Classify ground points and non-ground points based on a priori sensor height , iteratively obtain the lowest point in the polar coordinate grid, called the reference point in the grid , Below threshold When Set as , Above threshold When the grid height is updated to the sensor height , when the reference point is within the threshold, the grid height value is set to ; Set the grid height value to be less than The grids with a value smaller than the threshold are marked as ground grids. The points are marked as ground points; Step 33: Median filtering further smoothes the ground height: missing ground point clouds are common due to occlusion, and the missing ground units are filled. The missing height values ​​are replaced by the median of the four adjacent grids around the grid to obtain a better ground estimate; Step 4: Divide the grid from fine to coarse, and divide the non-ground point cloud into different sector areas; Step 5: Calculate the distance dmin between each sector area and the nearest non-ground point, and traverse different step lengths to mark the grids with a distance less than dmin as drivable grids; Step 6: Smooth filter the drivable grids and merge all drivable grids to obtain the drivable area.

2. The method for detecting an adaptive drivable area of ​​an intelligent wheelchair according to claim 1, characterized in that: The step 1 specifically includes: Step 11: Obtain the three-dimensional point cloud data collected by the laser radar on the smart wheelchair within 360°. The point cloud data includes roads, buildings, pedestrians and other obstacles. Each point includes three-dimensional coordinates XYZ; Step 12: The preprocessing of the collected point cloud data specifically includes using voxelization to reduce the number of point clouds, calculating a cube that can wrap the input point cloud, dividing it into different voxel grids according to the set resolution, and approximating all point clouds in the voxel grid by the centroid coordinates; using the CropBox filter to select the area of ​​interest in the point cloud data, and finally obtaining the range of the three directions of x, y, and z: (Xmin, Xmax), (Ymin, Ymax), (Zmin, Zmax); Step 13: Segment ground points and non-ground points by polar grid division: divide the original point cloud into a polar grid of mChannels x nBins, and the height information of the lowest point in each polar grid is mapped to the cell attribute; the mapping of each point in the point cloud to the polar grid is given by the formula, and Rmin corresponds to the minimum radius that the radar can scan; ； Step 14: Fit the local plane. Use the principal component analysis method to fit the local plane Z=Ax+By+C to the point cloud in each polar coordinate grid. The ground is relatively flat. The plane whose local plane normal vector is less than the threshold is taken as the candidate plane. The ground point is determined based on the distance from the point cloud to the candidate plane.

3. The method for detecting an adaptive drivable area of ​​an intelligent wheelchair according to claim 1, characterized in that: The step 2 specifically includes: Step 21: Calculate the distance from all data points in the grid to the fitting plane, which will be less than the dynamic threshold All points are classified as ground points; Step 22: Dynamic update: the height threshold of each frame of the ground grid is updated in real time, and the average distance from all ground points to the fitting plane in the previous frame is calculated as the height threshold of the grid in the next frame. , to obtain better ground estimation effect.

4. The method for detecting an adaptive drivable area of ​​an intelligent wheelchair according to claim 1, characterized in that: The step 4 specifically includes: Step 41: Grid division from fine to coarse: divide grids of different densities at different distances ,According to the different grid densities, they are divided into the central area, the second area, the third area and the outer area; Step 42: Cloud projection: Using the two-dimensional grid as the basic unit, project the non-ground point cloud after removing the ground. Step 43: A certain height threshold H is used as a projection constraint to project the point cloud onto the XOY two-dimensional plane; Step 44: Constructing the Distance Detector: Dividing the Radar's Circular Scanning Area into Equal Angles Sector , The number of areas N divided by the fan is shown in the formula; each fan-shaped area has a distance attribute, indicating the shortest Euclidean distance from the wheelchair to the point cloud at that angle; ; Step 45: Divide the point cloud into different fan-shaped areas; specifically, according to the coordinate information of the point cloud projection, calculate the angle θ with the x-axis, tan(θ) = y / x, if θ>2π, then θ=θ-2π; if θ<0, θ=θ+2π, ensuring that θ is in the range of [0,2π]; according to Round to get k, which belongs to the kth sector area; The step 5 specifically includes: Step 51: Initialize the distance attributes of the N sector areas to R, and traverse the point cloud point by point according to the point cloud index value, and calculate the minimum distance of each sector area using the Euclidean formula , if the following formula is satisfied, then replace the distance attribute value of the angle; ; Step 52: After traversing all points in the grid, we get The shortest distance ; Use the step-by-step search method to traverse different step lengths at each angle ; Consistent with the grid division from fine to coarse, the step size increases from small to large, and the grids that meet the conditions are marked as drivable grids.

5. The method for detecting an adaptive drivable area of ​​an intelligent wheelchair according to claim 1, characterized in that: The step 6 includes: performing smoothing filtering on the grids marked as drivable, selecting points around the central area and the second area grids that meet the first grid number, and if all are drivable grids, marking the surrounding grids of the second grid number as drivable grids; if not, traversing the next grid that meets the conditions; splicing all drivable grids, and finally obtaining an obstacle-free area for wheelchair travel.

6. A smart wheelchair adaptive drivable area detection system is used to implement the smart wheelchair adaptive drivable area detection method according to claim 1, characterized in that: It includes an acquisition module for collecting point cloud information during the wheelchair process; a ground point cloud segmentation module for dividing the collected point cloud into ground points and non-ground points; and a grid division module for subsequently determining the area where the wheelchair can travel; The distance detection module is used to detect the shortest distance from the non-ground point to the laser radar at each angle, and further determine the grid area for driving; the smoothing filter module is used to smooth the grids determined to be drivable; the area construction module is used to splice all the drivable grids to further obtain the free area for wheelchair access.

7. A smart wheelchair adaptive drivable area detection device is used to implement the smart wheelchair adaptive drivable area detection method according to claim 1, characterized in that: It includes a processor, a memory, a bus, an I / O interface, an input device, an output device, a storage device and a communication device; the memory is used to store the program and data of the above-mentioned drivable area detection algorithm, and the processor is used to execute the program in the memory. When the program is executed, it is executed in the order of the steps of the above-mentioned method; the program is downloaded and installed from the network through the communication device; the processor, the memory and the I / O interface are connected to each other through the bus; wherein the input device, the output device, the storage device and the communication device are all connected to the bus through the I / O interface.

8. The intelligent wheelchair adaptive drivable area detection device according to claim 7, characterized in that: The number of processors is one or more.

Images