Lidar-based detection method, apparatus, and computer-readable storage medium

Abstract

Embodiments of the present application relate to the field of heterogeneous computing, and disclose a detection method and device based on a laser radar and a computer readable storage medium. The detection method based on the laser radar comprises: acquiring scanning data of the laser radar; performing algorithm splitting on a feature algorithm used for detecting according to the scanning data, to obtain at least one sub-algorithm in the feature algorithm capable of being processed in parallel; performing heterogeneous acceleration on the at least one sub-algorithm to process the scanning data, and obtaining a detected obstacle position and a drivable area according to a processing result. The detection method and device based on the laser radar and the computer readable storage medium provided by the present application can improve the data operation speed in the laser radar sensing process, thereby improving the reliability of the laser radar.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of heterogeneous computing, and in particular to a detection method, device and computer-readable storage medium based on lidar. Background Technology

[0002] Traditional autonomous driving systems can be divided into modules such as localization, perception, fusion, decision-making and planning, and control. Among these, the perception module is a crucial component of the autonomous driving system. It enables the vehicle to acquire surrounding road condition information in a timely manner and deeply understand changes in the dynamic environment, facilitating subsequent effective information fusion and decision-making. During autonomous driving, surrounding road condition information is transmitted to the vehicle's Electronic Control Unit (ECU) via various sensors such as cameras, millimeter-wave radar, ultrasonic radar, and lidar.

[0003] LiDAR is a crucial sensor in autonomous driving, characterized by high measurement resolution, strong anti-interference capabilities, strong anti-stealth capabilities, high penetration ability, and all-weather operation. It utilizes the time it takes for a laser beam projected onto a target and reflected back to calculate the target's distance and create a 3D point cloud map. Furthermore, the laser beam generates different surface feature information based on the varying reflectivity of the object, providing invaluable reference for target detection and tracking. Using LiDAR to perceive the vehicle's surrounding environment can accurately detect obstacle distances and object surface properties, achieving a level of accuracy unmatched by cameras, millimeter-wave radar, and other sensors. However, the large amount of data computation involved in LiDAR sensing, coupled with the current relatively slow data processing speed, directly impacts its reliability. Summary of the Invention

[0004] The purpose of this invention is to provide a detection method, device, and computer-readable storage medium based on lidar, which can improve the data processing speed in the lidar sensing process, thereby improving the reliability of lidar.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a detection method based on lidar, comprising:

[0006] Acquire scanning data from a lidar radar; decompose the feature algorithm used for detection based on the scanning data to obtain at least one sub-algorithm capable of parallel processing; perform heterogeneous acceleration on at least one sub-algorithm to process the scanning data, and obtain the detected obstacle positions and drivable areas based on the processing results.

[0007] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described lidar-based detection method.

[0008] An embodiment of the present invention also provides a lidar-based detection device, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the lidar-based detection method described above.

[0009] Compared to existing technologies, the embodiments of this invention decompose the feature algorithm used for detection based on the scan data to obtain at least one sub-algorithm capable of parallel processing. Since the feature algorithm has multiple implementation processes, decomposing these processes to obtain at least one sub-algorithm capable of parallel processing satisfies the conditions for heterogeneous acceleration. Subsequently, at least one sub-algorithm is heterogeneously accelerated to process the scan data. Because the sub-algorithms capable of parallel processing are all algorithms with large data volumes, long computation times, and high CPU utilization, heterogeneous acceleration of at least one sub-algorithm effectively reduces the overall time consumption of the LiDAR sensing algorithm, thereby improving the data processing speed during LiDAR sensing and thus enhancing the reliability of the LiDAR. Furthermore, since the CPU utilization of sub-algorithms with high CPU utilization is significantly reduced after heterogeneous acceleration, lower-power and lower-cost processors and heterogeneous computing devices can be selected, reducing the production cost of LiDAR-based target detection devices.

[0010] Furthermore, the step of decomposing the feature algorithm used for detection based on the scan data to obtain at least one sub-algorithm capable of parallel processing specifically includes: decomposing a first feature algorithm for detecting obstacle positions based on the scan data and a second feature algorithm for detecting drivable areas based on the scan data to obtain multiple first sub-algorithms capable of parallel processing in the first feature algorithm and multiple second sub-algorithms capable of parallel processing in the second feature algorithm; the step of heterogeneously accelerating at least one of the sub-algorithms to process the scan data specifically includes: heterogeneously accelerating at least one of the multiple first sub-algorithms and / or at least one of the multiple second sub-algorithms to process the scan data.

[0011] In addition, the first feature algorithm includes: a fitting algorithm for fitting the ground based on the scan data; a segmentation and clustering algorithm for segmenting and clustering the obstacle based on the result of the fitted ground; an edge extraction algorithm for extracting the edge bounding box of the obstacle based on the result of the segmentation and clustering; and a location recognition algorithm for identifying the location of the obstacle based on the edge bounding box. The first feature algorithm for detecting the location of the obstacle based on the scan data is further divided into algorithms, specifically including: dividing at least one of the fitting algorithm, the segmentation and clustering algorithm, the edge extraction algorithm, and the location recognition algorithm.

[0012] Furthermore, when decomposing the edge extraction algorithm, the algorithm is broken down into the following steps: Based on the segmentation and clustering results, extract the coordinates of M two-dimensional mapped outer contour points of the obstacle, where M is an integer greater than 1; obtain M fitted straight lines based on the coordinates of the M two-dimensional mapped outer contour points, and calculate the M tilt angles of the M fitted straight lines, where one two-dimensional mapped outer contour point corresponds to one tilt angle; rotate the corresponding coordinates of the two-dimensional mapped outer contour points according to the tilt angles to obtain the cube border of the obstacle. In this way, the edge extraction algorithm, which has a long computation time in the first feature algorithm, can be decomposed to heterogeneously accelerate the steps in the edge extraction algorithm that meet the heterogeneous acceleration conditions, thereby improving the data processing speed in the LiDAR perception process and enhancing the reliability of the LiDAR.

[0013] In addition, the heterogeneous acceleration of at least one sub-algorithm among the plurality of first sub-algorithms and / or at least one sub-algorithm among the plurality of second sub-algorithms specifically includes: heterogeneous acceleration of at least one step in the edge extraction algorithm; the heterogeneous acceleration of at least one step in the edge extraction algorithm specifically includes: parallel processing of obtaining M fitted straight lines based on the coordinates of the M two-dimensional mapped outer contour points.

[0014] In addition, the second feature algorithm includes: a classification and storage algorithm for storing the scanned data in 360-degree grids; a ground point judgment algorithm for obtaining ground points in the scanned data and performing preliminary coloring processing on the ground points based on the results of the 360-degree grid classification and storage; a grid image for building a grid image based on the ground points and performing fine coloring processing on the grid image to obtain the drivable area; and an algorithm decomposition of the second feature algorithm for detecting drivable areas using lidar, specifically including: decomposing at least one of the classification and storage algorithm, the ground point judgment algorithm, and the grid coloring algorithm.

[0015] Furthermore, when the classification and storage algorithm is decomposed, it is broken down into the following steps: Based on the original LiDAR point cloud data, the polar coordinates of the original LiDAR points are obtained; the scanning area of ​​the LiDAR is divided into N sectors by a preset radius, where N is an integer greater than 1; based on the sector position of each original LiDAR point's polar coordinates, each original LiDAR point is stored in a two-dimensional array according to the sectors. This method decomposes the computationally intensive classification and storage algorithm in the second feature algorithm, enabling it to meet the heterogeneous acceleration conditions. This heterogeneous acceleration further improves the data processing speed during LiDAR perception, thereby further enhancing the reliability of the LiDAR.

[0016] In addition, the heterogeneous acceleration of at least one sub-algorithm among the plurality of first sub-algorithms and / or at least one sub-algorithm among the plurality of second sub-algorithms specifically includes: heterogeneous acceleration of at least one step in the classification and storage algorithm; the heterogeneous acceleration of at least one step in the classification and storage algorithm specifically includes: parallel processing of the original lidar points in the plurality of sector locations.

[0017] Furthermore, the algorithmic decomposition of at least one of the classification and storage algorithm, the ground point judgment algorithm, and the raster coloring algorithm specifically involves: decomposing the raster coloring algorithm; specifically, decomposing the raster coloring algorithm into the following steps: determining whether each raster in the raster image is ground, and coloring the raster determined to be ground. In this way, the raster coloring algorithm, which has a long computation time in the second feature algorithm, can be decomposed, thereby enabling the raster coloring algorithm to meet the heterogeneous acceleration conditions, further improving the data processing speed in the LiDAR sensing process, and thus further improving the reliability of the LiDAR.

[0018] In addition, the heterogeneous acceleration of at least one of the plurality of first sub-algorithms and / or at least one of the plurality of second sub-algorithms specifically includes: heterogeneous acceleration of at least one of the plurality of second sub-algorithms; the heterogeneous acceleration of at least one of the plurality of second sub-algorithms specifically includes: parallel processing to determine whether each grid in the raster image is ground. Attached Figure Description

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1 This is a flowchart of a lidar-based detection method provided according to a first embodiment of the present invention;

[0021] Figure 2 This is a flowchart of a lidar-based detection method provided according to a second embodiment of the present invention;

[0022] Figure 3 This is a flowchart of a target detection and tracking method provided according to a second embodiment of the present invention;

[0023] Figure 4 This is a polar coordinate grid point cloud partitioning diagram provided according to the second embodiment of the present invention;

[0024] Figure 5 This is a flowchart of point cloud clustering provided according to the second embodiment of the present invention;

[0025] Figure 6 This is a flowchart of obstacle outer contour border extraction provided by the second embodiment of the present invention;

[0026] Figure 7 This is a flowchart of the target detection and tracking algorithm splitting provided by the second embodiment of the present invention;

[0027] Figure 8 This is a flowchart of a lidar-based detection method provided according to a third embodiment of the present invention;

[0028] Figure 9 This is a flowchart of the drivable area identification algorithm provided by the third embodiment of the present invention;

[0029] Figure 10 This is a flowchart of a lidar-based detection method provided according to the fourth embodiment of the present invention;

[0030] Figure 11 This is a ground coloring effect diagram provided according to the fourth embodiment of the present invention;

[0031] Figure 12 This is a schematic diagram of the structure of a lidar-based detection device provided according to the sixth embodiment of the present invention;

[0032] Figure 13 This is a schematic diagram of the structure of a lidar-based detection device provided according to the seventh embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of the present invention to enable the reader to better understand the present invention. However, the technical solutions claimed in the present invention can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0034] The first embodiment of the present invention relates to a detection method based on lidar, and the specific process of this embodiment is as follows: Figure 1 As shown, it includes:

[0035] S101: Acquire scanning data from the lidar.

[0036] Specifically, during the operation of a lidar system, a large amount of scanning data is generated. This scanning data includes, but is not limited to, the distance and position coordinates of the original lidar points. In this embodiment, the acquired scanning data is the original lidar point cloud data.

[0037] S102: Decompose the feature algorithm used for detection based on scan data to obtain at least one sub-algorithm that can be processed in parallel.

[0038] Specifically, in this embodiment, the feature algorithm can be one or more. For example, in this embodiment, the feature algorithm may include a target detection and tracking algorithm and a drivable area recognition algorithm. The target detection and tracking algorithm can detect the location of obstacles and track the obstacles when they are detected; the drivable area recognition algorithm can detect drivable areas, thereby meeting the basic conditions for autonomous driving.

[0039] S103: Perform heterogeneous acceleration on at least one sub-algorithm to process the scan data, and obtain the detected obstacle locations and drivable areas based on the processing results.

[0040] Specifically, heterogeneous acceleration is a special form of parallel and distributed acceleration. It either uses a single independent computer capable of simultaneously supporting SIMD and MIMD methods, or a group of independent computers interconnected by a high-speed network, to complete the computational task. It coordinates the use of geographically dispersed machines with varying performance and architectures to meet different computational needs, enabling code (or code segments) to execute in a way that maximizes overall performance. Taking a feature-based target detection and tracking algorithm as an example, a target detection algorithm contains multiple sub-algorithms. By heterogeneously accelerating at least one of these sub-algorithms to process the scan data, the system's computational speed can be increased while reducing the algorithm's CPU usage.

[0041] Compared to existing technologies, the embodiments of this invention decompose the feature algorithm used for detection based on the scan data to obtain at least one sub-algorithm capable of parallel processing. Since the feature algorithm has multiple implementation processes, decomposing these processes to obtain at least one sub-algorithm capable of parallel processing satisfies the conditions for heterogeneous acceleration. Subsequently, at least one sub-algorithm is heterogeneously accelerated to process the scan data. Because the sub-algorithms capable of parallel processing are all algorithms with large data volumes, long computation times, and high CPU utilization, heterogeneous acceleration of at least one sub-algorithm effectively reduces the overall time consumption of the LiDAR sensing algorithm, thereby improving the data processing speed during LiDAR sensing and thus enhancing the reliability of the LiDAR. Furthermore, since the CPU utilization of sub-algorithms with high CPU utilization is significantly reduced after heterogeneous acceleration, lower-power and lower-cost processors and heterogeneous computing devices can be selected, reducing the production cost of LiDAR-based target detection devices.

[0042] The second embodiment of the present invention relates to a detection method based on lidar. The second embodiment is a further improvement on the first embodiment. Specifically, the feature algorithm is further refined in the second embodiment: the feature algorithm includes a first feature algorithm for detecting the location of obstacles based on the scan data and a second feature algorithm for detecting drivable areas based on the scan data. The first feature algorithm includes a fitting algorithm, a segmentation and clustering algorithm, and a location recognition algorithm. In the second embodiment, the fitting algorithm of the first feature algorithm is decomposed. In this way, the edge extraction algorithm, which has a long computation time in the first feature algorithm, can be decomposed, so that the algorithm steps of the edge extraction algorithm meet the heterogeneous acceleration condition to perform heterogeneous acceleration, thereby further improving the data processing speed in the lidar perception process and thus further improving the reliability of lidar.

[0043] The specific process of this implementation method is as follows: Figure 2 As shown, it includes:

[0044] S201: Acquire scanning data from the lidar.

[0045] S202: Decompose the edge extraction algorithm to obtain at least one sub-algorithm that can be processed in parallel.

[0046] Specifically, in this embodiment, the first feature algorithm is a target detection and tracking algorithm, including a fitting algorithm for fitting the ground based on the scanned data, a segmentation and clustering algorithm for segmenting and clustering the obstacle based on the fitted ground result, an edge extraction algorithm for extracting the edge bounding boxes of the obstacle based on the segmentation and clustering result, and a position recognition algorithm for identifying the location of the obstacle based on the edge bounding boxes. It can be understood that this embodiment improves the data processing speed in the LiDAR perception process by decomposing the edge extraction algorithm.

[0047] To facilitate understanding and to provide a more detailed explanation of the target detection and tracking algorithm, such as Figure 3 As shown, the target detection and tracking algorithm may include the following steps:

[0048] S2021: Determine the validity of the scan data and store the scan data that is determined to be valid.

[0049] Regarding step S2021, specifically, the scanning data includes the distance and height information of the original LiDAR points. The distance is the distance between the original LiDAR point and the initial center point of the LiDAR, and the height is the height of the original LiDAR point above the ground. Using preset distance and height thresholds, if the distance exceeds the distance threshold, the original LiDAR point is deemed invalid; if the height exceeds the height threshold, the original LiDAR point is deemed invalid. It is worth noting that in this embodiment, valid data from the scanning data is stored sequentially. All stored point cloud data is divided according to polar coordinates, with each sector being 2 degrees. Based on the sector position of each point, all coordinate points are stored as a one-dimensional array according to the sectors. It is understood that those skilled in the art can also divide the sectors according to other degrees; this embodiment does not specifically limit this.

[0050] S2022: Based on the results of storing the scanned data, filter the ground points in the scanned data.

[0051] Regarding step S2022, specifically, for each sector as an individual unit, the lowest point in the Z-axis direction among all laser point cloud data within each sector is found. Each point within that sector is compared to this lowest point, and simultaneously, the Z-axis value of that point is compared to the height of the laser radar installation above the ground. If it is below a certain height, the laser point is preliminarily considered a ground point. This process is repeated for all point sets within all sectors. In other words, when the height of the laser point cloud data is below a certain height (which can be preset according to actual needs), the point can be determined to be a ground point.

[0052] S2023: Fit a two-dimensional plane based on the selected ground points.

[0053] Regarding step S2023, specifically, using the selected ground points, the RANSAC algorithm is used to fit a two-dimensional plane, obtaining the parameter values ​​of the plane equation ax + by + cz + d = 0 (where a, b, c, and d are constants, and x, y, and z are independent variables), while ensuring that the parameter c of z in the plane equation is positive. Using the plane equation fitted by the above method, the value of each point in the effective point cloud dataset is compared with the value of the Z-axis direction of the points at the same X and Y axis positions on the plane. Points within a reasonable range (a reasonable range can be preset according to actual needs) are considered ground points, and the remaining points are classified as non-ground points.

[0054] It should be noted that steps S2021 to S2023 are steps of the fitting algorithm. By executing steps S2021 to S2023, the fitted ground can be obtained.

[0055] S2024: Use the scanned data after removing ground points as the raw point cloud data, and classify the raw point cloud data.

[0056] Regarding step S2024, specifically, as follows: Figure 4 As shown, the original point cloud data is separated according to the distance and height exceeding the maximum. The remaining data is the separable portion. This portion of data is saved using polar coordinate rasterization, resulting in a two-dimensional array. Each array cell stores the data of all points within that raster. Using the saved raster data, a corresponding two-dimensional array P (200*360) is created. Each raster with data is marked with 1, and otherwise marked with 0. The resulting two-dimensional array is then expanded to eliminate isolated raster points. It can be understood that data in the original point cloud data that is far from the center point of the LiDAR or at a high altitude can be considered invalid data (i.e., this data does not represent obstacles). By separating the original point cloud data according to the distance and height exceeding the maximum, invalid data is separated out, which greatly reduces the computational load of subsequent steps and further improves the computational performance of the LiDAR perception algorithm.

[0057] S2025: Based on the classification results of the original point cloud data, cluster the original point cloud data.

[0058] Regarding step S2025, specifically, based on the two-dimensional array P data from the above steps, each data point is traversed, and it is determined whether the points in the surrounding 8 positions of that point are 1. If the result is 1, these grid area points are classified into the same category, that is, points composed of the same obstacle. If the determination result is 0, the above determination is performed on the next P data point position until all points in P have been traversed. The grid points corresponding to all P points of the same category constitute the point set of the same obstacle. Figure 5As shown, the original point cloud data can be clustered using the following steps: S1: Mapping the original point cloud data to the label matrix elements. S2: Inflating the raster label matrix. S3: Determining if the raster and its surrounding elements (up, down, left, and right) are empty; if yes, proceed to step S1; otherwise, proceed to step S4. S4: Labeling elements with the same numerical value. S5: Determining if the raster needs to be traversed; if yes, proceed to step S6; otherwise, proceed to step S1. S6: Clustering label matrix.

[0059] S2026: Based on the clustering results of the original point cloud data, obtain the point cloud data of a single obstacle.

[0060] Regarding step S2026, specifically, using each obtained obstacle point, all laser point cloud data within its grid are grouped into the same category to obtain obstacle point cloud data.

[0061] It is worth mentioning that steps S2024 to S2026 are steps of the segmentation and clustering algorithm. By executing steps S2024 to S2026, obstacles can be segmented and clustered.

[0062] S2027: Based on the obstacle point cloud data, extract the coordinates of M two-dimensional mapped outer contour points of the obstacle.

[0063] Regarding step S2027, specifically, M is an integer greater than 1. In this embodiment, the coordinates of the two-dimensional mapped outer contour points can be extracted as follows: First, design a grid, find the maximum and minimum points in the X or Y direction of the obstacle point cloud data (hereinafter, the X direction shall be taken as the reference), and set a grid with equal arcs (based on the laser radar beam angle and distance) appropriately according to the distance from these points to the origin. The farther the distance, the larger the grid spacing. Segment the point cloud, find the grid where all points belong according to their X-axis coordinate values ​​using a quicksort method, and put them into the grid. And save the maximum and minimum points in the Y-axis direction of each grid.

[0064] S2028: Obtain M fitted straight lines based on the coordinates of M two-dimensional mapped outer contour points, and calculate the M tilt angles of the M fitted straight lines.

[0065] Regarding step S2028, specifically, each of the two-dimensional mapped outer contour points corresponds to a tilt angle. Using all the point sets obtained in S2027, a straight line in the plane is fitted using the random sampling consistency principle, and the angle β between the straight line and the X-axis direction is calculated, thus obtaining the tilt angle of the straight line.

[0066] S2029: Based on the coordinates of the two-dimensional mapped outer contour points corresponding to the tilt angle, obtain the cube border of the obstacle.

[0067] Regarding step S2029, specifically, based on the angle β obtained in step two (the angle between the fitted straight line and the X-axis direction), rotate the coordinates of all the point sets obtained in step one by β, that is:

[0068] Point[i].x_new=Point[i].x*cosβ-Point[i].y*sinβ

[0069] Point[i].y_new=Point[i].x*sinβ+Point[i].y*cosβ

[0070] The obstacle information in the drawn cube border includes the obstacle's speed, number of points, center point coordinates, 3D box corner coordinates, tracking ID, category, and the time when it was last tracked.

[0071] It is worth mentioning that steps S2027 to S2029 above are steps in the edge extraction algorithm. By executing steps S2027 to S2029, the location of obstacles can be detected. In summary, as Figure 6 As shown, the extraction of the outer contour of an obstacle can be divided into several steps: extraction of outer contour points by two-dimensional coordinate mapping of laser point cloud, calculation of fitted straight line by RANSAC, extraction of extreme coordinates by rotating outer contour points, and calculation of the three-dimensional boundary of the obstacle by extreme values.

[0072] Preferably, after obtaining the cube bounding boxes of obstacles, a queue is created to store the tracked targets. Then, the coordinate information of all obstacles is transformed into the global coordinate system. The specific tracking process is as follows: The correlation matrix is ​​calculated. Based on all obstacles in the point cloud of the previous frame and all obstacles in the point cloud of the current frame, the cost function is calculated by calculating the distance difference and point count difference between each target object between the two frames, thereby establishing a correlation matrix for all targets between the two frames. The Hungarian algorithm is used for matching to obtain the optimal matching tracking result between the two frames. The latest list of tracked objects is updated. New obstacles are assigned new IDs, and their bounding boxes are redrawn and their categories recorded.

[0073] like Figure 7As shown, in general, the target detection and tracking algorithm can be divided into several steps: plane fitting, ground segmentation, clustering, bounding box drawing, and tracking. The plane fitting process can be further divided into the steps of "classifying and storing the original point set by sector", "preliminarily screening ground points in each sector and recording them uniformly", and "generating a plane equation for all pre-selected ground points using RANSAC". The ground segmentation process involves dividing the original point set into ground points and non-ground points using the plane equation. The clustering process can be further divided into the steps of "classifying and storing non-ground points by 360-degree grid", "clustering by grid", and "extracting points from the clustered grid and saving them as different objects". The bounding box drawing process can be further divided into the steps of "extracting the outer contour points of the laser point cloud 2D mapping", "fitting a straight line using Hough transform and calculating the inclination of the line", and "rotating the coordinates of the point set and drawing the cube border". The tracking process can be further divided into the steps of "coordinate system transformation", "calculating the correlation matrix", and "calculating the best match using the Hungarian algorithm".

[0074] S203: Perform heterogeneous acceleration on at least one sub-algorithm to process the scan data, and obtain the detected obstacle locations and drivable areas based on the processing results.

[0075] Specifically, in this embodiment, at least one sub-algorithm is heterogeneously accelerated to process the scanned data. This can be achieved by: parallel processing to obtain M fitted straight lines based on the coordinates of M two-dimensional mapped outer contour points, and calculating the M tilt angles of the M fitted straight lines. During the bounding box process of the object detection algorithm, the Hough transform is used to fit planar straight lines to the edges of a single object, thereby obtaining the object's three-dimensional boundary rectangle. The Hough transform is one of the fundamental methods for recognizing geometric shapes from images in image processing. Compared to other methods, this method can better reduce noise interference when finding straight lines. Its basic principle lies in utilizing the duality between points and lines to transform a given curve in the original image space into a point in the parameter space through a curve expression. This transforms the problem of detecting a given curve in the original image into the problem of detecting peak values ​​in the parameter space. The Hough transform performs the same processing on each point, resulting in a large amount of repetitive work. Heterogeneous acceleration is used to process different points in parallel, implementing pipelined operations for the computation process of each point, greatly reducing time consumption.

[0076] To make it easier to understand, the following is a detailed example of how to draw the cube outline of an obstacle:

[0077] First, a straight line in polar coordinates is represented as r = x·cosθ + y·sinθ. The Hough transform processes a set of image points. For each point (x0, y0), all straight lines passing through it are plotted in the polar coordinate plane relative to the polar radius and polar angle, resulting in a sine curve. If the curves obtained from the above operations for two different points intersect in the plane θ-r, it means they pass through the same straight line. This method performs the same processing for each point, resulting in a large amount of repetitive work. When executed by a CPU, it iterates through all points in the image sequentially, and the time consumed is proportional to the number of points. If implemented using an FPGA, different points can be processed in parallel, and the computation flow for each point can be piped, greatly reducing the time consumption.

[0078] The HLS software was used to provide compilation guidance and optimization for the Hough transform algorithm implementation, generating RTL-level hardware description code. The compilation guidance and optimization followed these principles:

[0079] (1) Define internal variables using fixed-point types instead of floating-point types as much as possible. In HLS, data precision has a great impact on the utilization of hardware resources. HLS supports configurable fixed-point arithmetic. Under the premise of meeting the precision requirements, using fixed-point numbers can reduce hardware resource consumption and run at a higher clock frequency.

[0080] (2) For single-level loops, design the loop as a pipelined mode and use the pipeline optimization instruction set_directive_pipeline. Pipeline mode can increase resource utilization, and the next loop does not need to wait for the previous loop to complete before starting, thereby speeding up the parallelism and computation speed.

[0081] (3) For nested loops, the pipeline optimization instruction `set_directive_pipeline` can be used on the inner loop. This feature is that loops below the current layer using the pipeline are automatically optimized and unrolled, while loops above the current layer (including that layer) are reduced to single-layer loops. After the inner loop is unrolled, the hardware circuits corresponding to the inner operation steps are copied multiple times to achieve parallel processing.

[0082] (4) Split the array using the splitting optimization directive `set_directive_array_partition`. Note that array splitting optimization is not isolated and needs to be combined with specific usage. After splitting, the array is synthesized into multiple block RAMs (or other types of RAM can be specified), satisfying simultaneous access from multiple interfaces and improving data throughput. When used in conjunction with multiple expanded hardware circuits, it can maximize the parallel execution capability of the hardware circuits.

[0083] Understandably, the Hough transform involves a considerable amount of code; to avoid repetition, it will not be detailed here. Each functional module containing a large number of image point-intensive operations can be designed and optimized according to the above principles, thus enabling FPGA parallel acceleration processing of the object detection bounding box function and significantly reducing processing latency.

[0084] It should be noted that in this embodiment, other steps of the edge extraction algorithm can also be heterogeneously accelerated, as can the fitting algorithm and other segmentation and clustering algorithms. That is to say, this embodiment does not specifically limit the types of algorithms that can be heterogeneously accelerated, nor does it specifically limit the number of algorithms that can be heterogeneously accelerated. For example, the edge extraction algorithm and the fitting algorithm can be heterogeneously accelerated at the same time, thereby further improving the computational performance of the system. In addition, how to obtain the detected obstacle position is explained in detail in steps S2021 to S2029 of this embodiment. To avoid repetition, it will not be repeated here.

[0085] Compared to existing technologies, the embodiments of this invention decompose the feature algorithm used for detection based on the scan data to obtain at least one sub-algorithm capable of parallel processing. Since the feature algorithm has multiple implementation processes, decomposing these processes to obtain at least one sub-algorithm capable of parallel processing satisfies the conditions for heterogeneous acceleration. Subsequently, at least one sub-algorithm is heterogeneously accelerated to process the scan data. Because the sub-algorithms capable of parallel processing are all algorithms with large data volumes, long computation times, and high CPU utilization, heterogeneous acceleration of at least one sub-algorithm effectively reduces the overall time consumption of the LiDAR sensing algorithm, thereby improving the data processing speed during LiDAR sensing and thus enhancing the reliability of the LiDAR. Furthermore, since the CPU utilization of sub-algorithms with high CPU utilization is significantly reduced after heterogeneous acceleration, lower-power and lower-cost processors and heterogeneous computing devices can be selected, reducing the production cost of LiDAR-based target detection devices.

[0086] The third embodiment of the present invention relates to a detection method based on lidar. The third embodiment is a further improvement on the first embodiment. Specifically, the improvement is that the second feature algorithm is further refined in the third embodiment. The second feature algorithm includes a classification and storage algorithm, a ground point judgment algorithm, and a grid coloring algorithm. In the third embodiment, the classification and storage algorithm of the second feature algorithm is decomposed. In this way, the classification and storage algorithm, which has a long computation time in the second feature algorithm, can be decomposed, so that the steps of the classification and storage algorithm meet the heterogeneous acceleration conditions to accelerate it heterogeneously. This further improves the data processing speed in the lidar perception process, thereby further improving the reliability of lidar.

[0087] The specific process of this implementation method is as follows: Figure 8 As shown, it includes:

[0088] S301: Acquire scanning data from the LiDAR.

[0089] S302: Decompose the classification and storage algorithm to obtain at least one sub-algorithm that can be processed in parallel.

[0090] Specifically, in this embodiment, the second feature algorithm can be a drivable area identification algorithm. Drivable area identification involves projecting the point cloud data collected by the LiDAR onto a horizontal plane, then segmenting the area, processing the point cloud data within the segmented area, and finally determining whether the area is drivable. Generally, the LiDAR rotates rapidly while emitting a high-frequency laser beam to continuously scan the external environment. This working mode is best suited for a region segmentation method in polar coordinates. The drivable area identification algorithm includes a classification and storage algorithm that categorizes and stores the scanned data into 360-degree grids; a ground point judgment algorithm that obtains ground points from the scanned data based on the 360-degree grid classification and storage results, and performs preliminary coloring processing on the ground points; and a grid coloring algorithm that builds a grid image based on the ground points, refines and colors the grid image, and obtains the drivable area. It can be understood that this embodiment improves the data processing speed in the LiDAR perception process by decomposing the classification and storage algorithm.

[0091] To facilitate understanding and to provide a more detailed explanation of the drivable area identification algorithm, such as... Figure 9 As shown, the drivable area identification algorithm may include the following steps:

[0092] S3021: Obtain the polar coordinates of the original lidar points based on the scanning data.

[0093] S3022: Divide the scanning area of ​​the lidar into N sectors by a preset arc.

[0094] Specifically, N is an integer greater than 1, and the N sectors are divided as follows: Figure 4 As shown.

[0095] S3023: Based on the sector position of the polar coordinates of each original LiDAR point, store each original LiDAR point in a two-dimensional array according to the sector.

[0096] It is worth mentioning that steps S3021 to S3023 are the steps of the classification and storage algorithm. By executing steps S3021 to S3023, the original laser points can be classified and stored.

[0097] S3024: Get the lowest point in the Z-axis direction for each grid cell.

[0098] S3025: Establish the first grid image in polar coordinate system based on the lowest point in the Z-axis direction of each grid.

[0099] Specifically, for each grid cell, a grid image with a corresponding polar coordinate system is created based on the lowest point along the Z-axis. All grid images are initialized to 1, and the polar coordinates are expanded outwards from the origin. Points without laser points are marked as 0. This is compared to the current grid's lowest point; if a point is identified as a ground laser point, it is marked as 0. If a point is determined to be non-ground, the entire grid cell from that point onwards, in a fan-shaped region, is marked as 1. After traversing all laser points, the first grid image is dilated to improve its clarity.

[0100] It is worth mentioning that steps S3024 to S3025 are the steps of the ground point judgment algorithm. By executing steps S3024 to S3025, ground points and non-ground points can be judged, and ground points can be colored.

[0101] S3026: Use the Cartesian coordinate system to establish a second grid image, and obtain the drivable area based on the first grid image and the second grid image.

[0102] Specifically, a higher-precision image grid of 500*500 is established using a Cartesian coordinate system, with each pixel covering an accuracy of 0.1*0.1 (m). This image grid is used to determine whether the same position in the polar coordinate grid is 0. If it is 0, it indicates a drivable area; otherwise, it is marked as an indestructible area.

[0103] It is worth mentioning that step S3026 above is a step of the raster coloring algorithm. By executing step S3026, the final drivable area can be drawn.

[0104] S303: Perform heterogeneous acceleration on at least one sub-algorithm to process the scan data, and obtain the detected obstacle locations and drivable areas based on the processing results.

[0105] Specifically, in this embodiment, at least one sub-algorithm is heterogeneously accelerated to process the scanning data. This can be achieved by: parallel processing to organize each of the original LiDAR points into a two-dimensional array according to the sectors and storing it. This process requires traversing all the original LiDAR point data, calculating the index position of each point mapped to the grid, and storing it in the grid. Heterogeneous acceleration is employed, using a pipelined model for the calculation of different points, so that the coordinate transformation and area attribute calculation of each point can be completed in just one clock cycle, improving efficiency by thousands of times compared to traditional CPUs.

[0106] To facilitate understanding, the following example illustrates how to process each of the original lidar points in parallel and store them as a two-dimensional array according to the sectors:

[0107] First, the point cloud data collected by the LiDAR is projected onto a horizontal plane. Then, the region is segmented, and the points are categorized and stored according to the segmented grid. This process requires traversing all the original point cloud data, calculating the index position of each point mapped to a grid, and storing it in the grid. Heterogeneous acceleration is employed, using a pipelined model for calculations on different points. This allows the coordinate transformation and region attribute calculation for each point to be completed in just one clock cycle, improving efficiency by thousands of times compared to traditional CPUs. Detailed steps are as follows;

[0108] The horizontal plane is divided into regions in the polar coordinate system according to a certain step size r and θ. Then, all point cloud data in the spatial Cartesian coordinate system are projected onto the horizontal plane to obtain feature description data in the plane rectangular coordinate system. Then, coordinate transformation is performed to obtain feature description data in the polar coordinate system. Then, based on the feature description data, it is calculated which segmented region the point is located in, and the feature data of the point is stored in the container corresponding to the region.

[0109] For ease of description, the meanings of the relevant symbols are first defined as follows:

[0110] r: Radius variable in polar coordinates;

[0111] θ: Polar angle variable in polar coordinate system;

[0112] L: The radius of the circle in the polar coordinate plane;

[0113] r′: Polar radius axis cutting step size;

[0114] θ′: Polar axis cutting step size;

[0115] M: Number of blocks cut along the polar radius direction;

[0116] N: Number of blocks cut along the polar angle direction;

[0117] X: The x-coordinate value of a point cloud feature data in a rectangular coordinate system;

[0118] Y: The y-coordinate direction value of a point cloud feature data in a rectangular coordinate system;

[0119] Z: The z-coordinate value of a point cloud feature data in a Cartesian coordinate system;

[0120] PointCloud_Array: Point cloud data array, where each element of the array is x, y, z;

[0121] M: The block number of a point cloud located in the polar radius direction;

[0122] N: The block number of a point cloud located in the polar direction;

[0123] PointAttr_Array: An array of point cloud attribute information, where each element of the array is m and n;

[0124] Size: Number of elements in the point cloud data array.

[0125] (1) Taking the lidar as the origin, the circle (radius L) with r(θ) = L is divided equally on the polar radius axis and polar angle axis on the horizontal plane. The cutting step of the polar radius axis is r′ and the cutting step of the polar angle axis is θ′. Then the total number of cutting regions is (L / r′)*(2π / θ′). Let L / r′ be M and 2π / θ′ be N. Then we divide the circle with radius L into M*N regions.

[0126] (2) First, point cloud data of the same data type are stored uniformly. A specific memory block is allocated based on the amount of point cloud data, and all point cloud feature data (x, y, z) sets in the spatial Cartesian coordinate system are stored sequentially in this memory, forming an array PointCloud_Array. Note that arrays are used to store point clouds or other data of the same type, avoiding the use of data structures with dynamic memory management, such as vectors. The purpose here is to facilitate the HLS optimization acceleration module's direct access to external storage devices using the AXI bus. Another function of the array is that it can be mapped to internal FPGA storage resources using HLS to achieve concurrent access, which will be used in subsequent embodiments. The following embodiments all follow this principle.

[0127] (3) Allocate a specific memory block according to the amount of point cloud data to store the attribute information of each point region calculated by hardware acceleration, forming an array PointAttr_Array, and the elements of this array correspond one-to-one with the elements of the PointCloud_Array array;

[0128] (4) Pass the starting address of the PointCloud_Array array and the array size as input parameters to the acceleration function. When designing the acceleration function, set the array access mode to sequential access mode. In this way, the process of FPGA moving data from memory to logic unit is continuous. At the same time, the zero_copy instruction is used to make the compiler directly connect the hardware accelerator interface to the processor's S_AXI port to complete the data transmission. This will greatly improve the performance of the interface part.

[0129] (5) Pass the starting address of the PointAttr_Array array as an output parameter to the acceleration function, and set the array access mode to sequential access, and use the zero_copy instruction to describe the data transfer mode of the array;

[0130] (6) The FPGA acceleration function internally consists of a single-layer loop with a loop count of size. Each loop processes one point cloud data point. We can optimize this loop using the pipeline instruction PIPELINE. This type of optimization allows the FPGA to read the data required for the (i+1)th loop while performing data calculations for the i-th loop, and may simultaneously store the calculation results output from the (i-1)th loop. Each loop works as follows: first, it reads a PointCloud_Array array element (x, y, z), and then uses the formula... We calculate the characteristic data (r,θ) of the polar coordinate system of the element using θ = atan2(y,x). Let m = r / r′ and m <= M, n = θ / θ′ and n <= N. Thus, we have calculated that the point (x,y,z) is located in the (m,n) region. Finally, we store (m,n) into the PointAttr_Array array element corresponding to the point.

[0131] It should be noted that the method for obtaining the detected drivable area has been explained in detail in steps S3021 to S3026 of this embodiment, and will not be repeated here to avoid repetition.

[0132] Compared to existing technologies, the embodiments of this invention decompose the feature algorithm used for detection based on the scan data to obtain at least one sub-algorithm capable of parallel processing. Since the feature algorithm has multiple implementation processes, decomposing these processes to obtain at least one sub-algorithm capable of parallel processing satisfies the conditions for heterogeneous acceleration. Subsequently, at least one sub-algorithm is heterogeneously accelerated to process the scan data. Because the sub-algorithms capable of parallel processing are all algorithms with large data volumes, long computation times, and high CPU utilization, heterogeneous acceleration of at least one sub-algorithm effectively reduces the overall time consumption of the LiDAR sensing algorithm, thereby improving the data processing speed during LiDAR sensing and thus enhancing the reliability of the LiDAR. Furthermore, since the CPU utilization of sub-algorithms with high CPU utilization is significantly reduced after heterogeneous acceleration, lower-power and lower-cost processors and heterogeneous computing devices can be selected, reducing the production cost of LiDAR-based target detection devices.

[0133] The fourth embodiment of the present invention relates to a detection method based on LiDAR. The fourth embodiment is largely the same as the third embodiment, with the main difference being that, in the fourth embodiment, at least one of the classification and storage algorithm, the ground point judgment algorithm, and the grid coloring algorithm is decomposed. Specifically, the grid coloring algorithm is decomposed into the following steps: determining whether each grid in the grid image is ground, and coloring the grids determined to be ground. This method decomposes the computationally expensive grid coloring algorithm in the second feature algorithm, allowing the steps of the grid coloring algorithm to meet the heterogeneous acceleration conditions, thereby improving the data processing speed in the LiDAR sensing process and further enhancing the reliability of the LiDAR.

[0134] The specific process of this implementation method is as follows: Figure 10 As shown, it includes:

[0135] S401: Acquire scanning data from the LiDAR.

[0136] S402: Decompose the raster coloring algorithm to obtain at least one sub-algorithm that can be processed in parallel.

[0137] Specifically, the drivable area identification algorithm includes a classification and storage algorithm that stores scanned data in 360-degree grids; a ground point judgment algorithm that obtains ground points from the scanned data based on the 360-degree grid classification and storage results, and performs preliminary coloring processing on the ground points; and a grid coloring algorithm that builds a grid image based on the ground points, refines and colors the grid image, and obtains the drivable area. It can be understood that this implementation improves the data processing speed in the LiDAR perception process by decomposing the grid coloring algorithm.

[0138] S403: Perform heterogeneous acceleration on at least one sub-algorithm to process the scan data, and obtain the detected obstacle locations and drivable areas based on the processing results.

[0139] Regarding step S404, specifically, in this embodiment, at least one sub-algorithm is heterogeneously accelerated to process the scan data. This can be achieved by: parallel processing to determine whether each grid in the raster image is ground, and coloring the grids determined to be ground. Ground area coloring involves marking drivable areas with color before sending them to a visualization interface for display. It iterates through all areas to be colored, calculating the corresponding positional relationship between the grids from the previous step and the area to be colored for each area, extracting ground information from the grids, and coloring them within squares. This heterogeneous acceleration approach, by appropriately parallelizing different areas and implementing pipelined operations between steps, significantly reduces time consumption.

[0140] To facilitate understanding, the following detailed examples illustrate how to process in parallel whether each grid cell in the raster image is the ground, and how to colorize the grid cells determined to be the ground:

[0141] The input to the processing unit handling ground coloring is the raster output from the previous step. Each raster has been identified as either ground or not. Figure 11 As shown in the first part, 0 represents the ground area, and 1 represents the non-ground area. The coloring area is limited to a square, fixedly divided into 500x500 parts. For each part of the coloring area, the corresponding position in the grid needs to be extracted and it needs to be determined whether it is the ground. If it is, the square is colored; otherwise, it is left uncolored. See... Figure 11 The second part, the numerical value represents the depth of the coloring, 60 represents a drivable road surface, and 90 represents a non-drivable area.

[0142] The above process requires traversing 500×500 sections of the area to be colored. For each section, the corresponding positional relationship between the raster and the area to be colored is calculated, the ground information in the raster is extracted, and the area is colored. The operation of each section is relatively independent and there is no dependency between them. If executed by CPU, it will be processed sequentially, and the time consumed is proportional to the number of sections. If implemented by FPGA, different colored areas can be processed in parallel, and the steps can be pipelined, greatly reducing the time consumption.

[0143] HLS is used to guide and optimize the compilation process, and then RTL-level hardware description code is generated. The structure of the shaded code is as follows:

[0144] Loop 1: for(x=0;x<500;x++){

[0145] Loop 2: for(y=0;y<500;y++){

[0146] Initialize the color of the area to be colored

[0147] e.g., colour[y*500+x]=90;

[0148] Calculate the raster positions a and b corresponding to the colored regions represented by index values ​​x and y.

[0149] Extract ground information from the grid area

[0150] Determine if it is ground and color it.

[0151]

[0152] In the code structure above, loops one and two traverse the entire colored area, with constant iteration counts. Variables x and y are used to index the position of the colored area, while variables a and b are used to index the position of the raster. The array grid_info[] stores raster information, and the array colour[] stores colored area information.

[0153] The code above is located within the acceleration module. Both the `grid_info[]` and `colour[]` arrays are internally defined. The acceleration function first copies the passed-in external array to the function's internal array. HLS then maps the internal array to the FPGA's internal storage resources, enabling concurrent access. During the execution of the acceleration module, there is no further data interaction between heterogeneous processors until the result data is output. This reduces frequent access between heterogeneous processors and greatly improves computational performance.

[0154] The pipeline optimization instruction `set_directive_pipeline` is applied to loop one. This instruction optimizes and expands loop two, and the hardware circuitry generated within the curly braces of loop two is copied multiple times, without affecting loop one itself. The partition optimization instruction `set_directive_array_partition` is applied to the array `colour[]`. The partition optimization instruction `set_directive_array_partition` can be applied to both dimensions of the array `grid_info[]`. After synthesis, the array is mapped to multiple block RAMs, and multiple data can be accessed simultaneously according to the number of partitions, matching the multiple hardware circuits expanded in loop two, thus achieving parallel data processing. The number of partitions for arrays `colour[]` and `grid_info[]` can be adjusted as needed. More partitions provide more RAM ports and better performance, but also consume more resources; conversely, fewer RAM ports result in worse performance, but less resource consumption. The above operations can realize the FPGA parallel acceleration processing of the ground area coloring function, greatly reducing processing latency.

[0155] Compared to existing technologies, the embodiments of this invention decompose the feature algorithm used for detection based on the scan data to obtain at least one sub-algorithm capable of parallel processing. Since the feature algorithm has multiple implementation processes, decomposing these processes to obtain at least one sub-algorithm capable of parallel processing satisfies the conditions for heterogeneous acceleration. Subsequently, at least one sub-algorithm is heterogeneously accelerated to process the scan data. Because the sub-algorithms capable of parallel processing are all algorithms with large data volumes, long computation times, and high CPU utilization, heterogeneous acceleration of at least one sub-algorithm effectively reduces the overall time consumption of the LiDAR sensing algorithm, thereby improving the data processing speed during LiDAR sensing and thus enhancing the reliability of the LiDAR. Furthermore, since the CPU utilization of sub-algorithms with high CPU utilization is significantly reduced after heterogeneous acceleration, lower-power and lower-cost processors and heterogeneous computing devices can be selected, reducing the production cost of LiDAR-based target detection devices.

[0156] The fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described method embodiments.

[0157] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0158] The sixth embodiment of the present invention relates to a detection device 100 based on lidar, the specific structure of which is as follows: Figure 12 As shown, it includes: an acquisition module 1, an algorithm splitting module 2, and a heterogeneous acceleration module 3; the acquisition module 1 is used to acquire the scanning data of the LiDAR and send the scanning data to the algorithm splitting module 2; the algorithm splitting module 2 is used to receive the scanning data and split the feature algorithm used for detection based on the scanning data to obtain at least one sub-algorithm that can be processed in parallel; the heterogeneous acceleration module 3 is used to perform heterogeneous acceleration on at least one of the sub-algorithms to process the scanning data, and obtain the detected obstacle positions and drivable areas based on the processing results.

[0159] It should be noted that the LiDAR-based detection device 100 may also omit the acquisition module 1. The scanning data of the LiDAR can be directly obtained by splitting the module 2 through an algorithm, thereby making the structure of the LiDAR-based detection device 100 simpler.

[0160] Specifically, the feature algorithm in this embodiment can be a target detection and tracking algorithm and a drivable area recognition algorithm. That is to say, the algorithm splitting module 2 has two functions: first, it implements the target detection and tracking algorithm and the drivable area recognition algorithm; second, it reasonably splits the implementation process of the two algorithms. The heterogeneous acceleration module 3 selects the parallel processing part in the algorithm splitting module 2 to achieve heterogeneous acceleration.

[0161] It is worth mentioning that any operation steps that, after being decomposed by the algorithm decomposition module, have the characteristics of "large data volume, time consumption, and can be implemented through parallel processing" can be implemented using heterogeneous acceleration methods, and fall within the scope of protection of this invention. Furthermore, while the embodiments may employ High-Level Synthesis (HLS) technology for FPGA implementation, this invention is not limited to this method and can also be implemented using traditional RTL languages.

[0162] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0163] The seventh embodiment of the present invention relates to a detection device based on lidar, such as... Figure 13 As shown, it includes: at least one processor 701; and a memory 702 communicatively connected to at least one processor 701; wherein the memory 702 stores instructions executable by at least one processor 701, the instructions being executed by at least one processor 701 to enable at least one processor 701 to perform the above-described LiDAR-based detection method.

[0164] The memory 702 and processor 701 are connected via a bus, which can include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 701 and memory 702 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 701 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 701.

[0165] Processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 702 can be used to store data used by processor 701 during operation.

[0166] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A detection method based on lidar, characterized in that, include: Acquire scanning data from the LiDAR; The feature algorithm used for detection based on the scan data is decomposed to obtain at least one sub-algorithm that can be processed in parallel. Heterogeneous acceleration is applied to at least one of the sub-algorithms to process the scan data, and the detected obstacle locations and drivable areas are obtained based on the processing results; The step of decomposing the feature algorithm used for detection based on the scan data includes: The first feature algorithm for detecting the location of obstacles based on the scan data is decomposed to obtain multiple first sub-algorithms that can be processed in parallel. The first feature algorithm includes: an edge extraction algorithm for extracting the edge bounding box of the obstacle based on the result of segmentation and clustering. When the edge extraction algorithm is broken down into its components, it is divided into the following steps: Based on the segmentation and clustering results, extract the coordinates of M two-dimensional mapped outer contour points of the obstacle, where M is an integer greater than 1; Based on the coordinates of the M two-dimensional mapped outer contour points, M fitting lines are obtained, and the M tilt angles of the M fitting lines are calculated, wherein one two-dimensional mapped outer contour point corresponds to one tilt angle. The cube border of the obstacle is obtained by rotating the corresponding two-dimensional mapped outer contour point coordinates according to the tilt angle. The step of heterogeneously accelerating at least one of the sub-algorithms includes: In parallel, M fitted straight lines are obtained based on the coordinates of the M two-dimensional mapped outer contour points, and the M tilt angles of the M fitted straight lines are calculated.

2. The detection method based on lidar according to claim 1, characterized in that, The step of decomposing the feature algorithm used for detection based on the scan data to obtain at least one sub-algorithm capable of parallel processing includes: The second feature algorithm for detecting drivable areas based on the scan data is decomposed to obtain multiple second sub-algorithms that can be processed in parallel within the second feature algorithm. The heterogeneous acceleration of at least one of the sub-algorithms to process the scan data specifically includes: Heterogeneous acceleration is applied to at least one sub-algorithm among the plurality of first sub-algorithms and / or at least one sub-algorithm among the plurality of second sub-algorithms to process the scan data.

3. The detection method based on lidar according to claim 2, characterized in that, The first feature algorithm includes: a fitting algorithm for fitting the ground based on the scan data; a segmentation and clustering algorithm for segmenting and clustering the obstacle based on the result of the fitted ground; and a location recognition algorithm for identifying the position of the obstacle based on the edge bounding box. The algorithm decomposition of the first feature algorithm used to detect the location of obstacles based on the scan data includes: At least one of the fitting algorithm, segmentation clustering algorithm, edge extraction algorithm, and location recognition algorithm is decomposed.

4. The detection method based on lidar according to claim 2, characterized in that, The second feature algorithm includes: a classification and storage algorithm for storing the scanned data according to a 360-degree grid; a ground point judgment algorithm for obtaining ground points in the scanned data based on the results of the 360-degree grid classification and storage, and performing preliminary coloring processing on the ground points; and a grid coloring algorithm for establishing a grid image based on the ground points, and performing fine coloring processing on the grid image to obtain the drivable area. The second feature algorithm used by LiDAR to detect drivable areas is broken down into its components, including: At least one of the classification and storage algorithm, the ground point judgment algorithm, and the raster coloring algorithm is split into algorithms.

5. The detection method based on lidar according to claim 4, characterized in that, When the classification and storage algorithm is decomposed, it is broken down into the following steps: Based on the scan data, the polar coordinates of the original lidar points are obtained; The scanning area of ​​the lidar is divided into N sectors by a preset arc, where N is an integer greater than 1; Based on the sector position of the polar coordinates of each original lidar point, each original lidar point is stored in a two-dimensional array according to the sector.

6. The detection method based on lidar according to claim 5, characterized in that, The heterogeneous acceleration of at least one sub-algorithm among the plurality of first sub-algorithms and / or at least one sub-algorithm among the plurality of second sub-algorithms specifically refers to heterogeneous acceleration of at least one algorithm among the classification and storage algorithms. The heterogeneous acceleration of at least one of the classification and storage algorithms includes: In parallel, based on the sector position where the polar coordinates of each original lidar point are located, each original lidar point is organized into a two-dimensional array according to the sector and stored.

7. The detection method based on lidar according to claim 4, characterized in that, Specifically, splitting at least one of the classification and storage algorithm, the ground point judgment algorithm, and the raster coloring algorithm involves splitting the raster coloring algorithm. The algorithm decomposition of the raster coloring algorithm specifically involves breaking down the raster coloring algorithm into the following steps: Determine whether each grid cell in the raster image is the ground, and color the grid cells determined to be the ground.

8. The detection method based on lidar according to claim 7, characterized in that, The heterogeneous acceleration of at least one of the plurality of first sub-algorithms and / or at least one of the plurality of second sub-algorithms specifically refers to: heterogeneous acceleration of at least one of the plurality of second sub-algorithms; The heterogeneous acceleration of at least one of the plurality of second sub-algorithms specifically includes: In parallel, it is determined whether each grid cell in the grid image is the ground, and the grid cells determined to be the ground are colored.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

10. A lidar-based detection device, comprising a memory and a processor, wherein the memory stores a computer program, which, when executed by the processor, implements the method as described in any one of claims 1 to 8.

Images