Point cloud clustering method and device using HDL, electronic equipment and storage medium

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for point cloud clustering using HDL. The method includes: processing point cloud data generated by an onboard LiDAR sequentially through a parsing stage, a calibration stage, a culling stage, a 2D meshing stage, an object search stage, and a clustering stage, converting the point cloud data from three dimensions to a two-dimensional plane in stages until point cloud clustering is completed; using HDL for parallel computation in each stage, and employing a FIFO buffer for data transmission between stages, to achieve point cloud clustering in a parallel pipeline manner within HDL-supported hardware circuits. This invention divides the point cloud clustering process into multiple stages based on the obstacle avoidance characteristics of autonomous driving, and uses HDL for parallel computation in each stage, executing it in a parallel pipeline manner within HDL-supported hardware circuits. This results in superior performance and a simpler implementation, enabling the technical solution of this invention to meet the real-time clustering requirements of autonomous driving.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method, apparatus, electronic device, and storage medium for point cloud clustering using HDL. Background Technology

[0002] With the emergence of large-scale datasets and the development of deep learning, LiDAR-based perception has made significant progress and is widely used in autonomous driving, artificial intelligence, military, 3D printing, VR / AR, medical, and ecological environment monitoring. A crucial aspect of LiDAR perception is point cloud segmentation, with classic clustering and segmentation algorithms including DBSCAN, KD-TREE, and KD-MEANS.

[0003] The DBSCAN algorithm has advantages such as noise resistance, no need to specify categories, and the ability to discover clusters of arbitrary shapes in spatial data. However, its computational complexity is high, with a neighborhood point search complexity of O(n^2), which requires high computing power. Autonomous driving requires real-time processing, which is unacceptable for the current computing capabilities.

[0004] The KD-TREE algorithm is a binary tree algorithm that accelerates nearest neighbor search, primarily by partitioning regions. However, this algorithm requires building a binary tree data structure in HDL (Hardware Description Language), making its implementation in HDL quite difficult. This is unacceptable for achieving autonomous driving.

[0005] KD-MEANS requires the number of target objects to be known before calculation, and the algorithm is affected by the initial centroid, resulting in different clustering results, which is unacceptable in the field of autonomous driving.

[0006] The above algorithms are all based on CPU instructions, which are difficult to implement in HDL languages. Therefore, there is an urgent need for a high-performance point cloud clustering method that can be implemented in HDL. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a method, apparatus, electronic device, and storage medium for point cloud clustering using HDL, aiming to solve the problem that the hardware circuits described by HDL in the prior art cannot solve the real-time problem of 3D clustering of point clouds.

[0008] This invention provides a point cloud clustering method using HDL, comprising:

[0009] The point cloud data generated by the vehicle-mounted LiDAR is processed in stages, including parsing, calibration, elimination, 2D meshing, object search, and clustering, transforming the point cloud data from three dimensions to a two-dimensional plane until point cloud clustering is completed.

[0010] Each stage employs HDL parallel computation, and data transfer between stages uses a FIFO buffer to achieve point cloud clustering in a parallel pipeline manner within the hardware circuit supported by HDL.

[0011] According to the present invention, a point cloud clustering method using HDL is provided, wherein the point cloud data generated by vehicle-mounted LiDAR is processed in stages, including a parsing stage, a calibration stage, a removal stage, a two-dimensional meshing stage, an object search stage, and a clustering stage, to transform the point cloud data from three dimensions to a two-dimensional plane until point cloud clustering is completed.

[0012] Analysis phase: Analyzing the point cloud data generated by the vehicle-mounted LiDAR;

[0013] Calibration phase: The point cloud data is converted to a three-dimensional vehicle coordinate system for calibration;

[0014] Elimination phase: Eliminate point cloud data that do not meet the preset conditions in the three-dimensional vehicle coordinate system;

[0015] Two-dimensional meshing stage: The point cloud data is mapped to a two-dimensional plane and meshed to generate a two-dimensional meshed point cloud plane;

[0016] Object search phase: Two-dimensional clustering object search is performed in the two-dimensional gridded point cloud plane, and the clustering identifiers corresponding to different objects are stored;

[0017] Clustering stage: The clustering identifiers are enumerated, and objects with the same clustering identifier are clustered together to complete the clustering of the point cloud data.

[0018] According to the present invention, a point cloud clustering method using HDL is provided, wherein removing point cloud data that does not meet preset conditions in the three-dimensional vehicle coordinate system includes:

[0019] In the three-dimensional vehicle coordinate system, the origin is set at the middle position of the vehicle's front.

[0020] The positive direction of the X-axis is set from left to right for the front of the vehicle, the positive direction of the vehicle's travel direction is set as the positive direction of the Y-axis, and the positive direction of the road is set as the positive direction of the Z-axis.

[0021] The X, Y, and Z axes are respectively set with a preset first length, a second length, and a third length. Point cloud data that does not meet the set range is deleted, that is, point cloud data outside the road is deleted, so that the point cloud data is concentrated on the road in the direction of vehicle travel.

[0022] According to the present invention, a point cloud clustering method using HDL is provided, wherein mapping the point cloud data to a two-dimensional plane and meshing it to generate a two-dimensional meshed point cloud plane includes:

[0023] The point cloud data within the specified range is mapped onto a two-dimensional plane formed by the X and Y axes, and a meshed model is established based on a preset grid.

[0024] The dimensions in the X and Y axes are determined based on the size of the grid.

[0025] Based on the aforementioned dimension, a first table and a second table of the same dimension are established. For each grid, the first table stores an indicator of whether the point cloud data exists in the grid, and the second table stores the point cloud data corresponding to the grid. After traversing all the grids, the two-dimensional gridded point cloud plane is generated in the form of a two-dimensional table.

[0026] According to the present invention, a point cloud clustering method using HDL is provided, wherein the step of searching for two-dimensional clustering objects in the two-dimensional meshed point cloud plane and storing clustering identifiers corresponding to different objects includes:

[0027] Based on the two-dimensional meshed point cloud plane, a third table and a fourth table with the same dimensions as the first table are established, wherein the third table is used to store the clustering identifier of the object, and the fourth table is used to store the access identifier;

[0028] After setting the search criteria, traverse the first table to search for grids containing the point cloud data.

[0029] During the traversal, for any grid containing the point cloud data, the point cloud data that meets the search conditions is taken as the searched object, an incremental identifier is used as the cluster identifier for different objects, and the neighboring points that meet the search conditions are written to the FIFO queue in an indexed manner.

[0030] Retrieve index information from the FIFO queue and use whether the corresponding point cloud data can be read as a judgment condition. If so, store the clustering identifier stored in the third table with the current value of the auto-increment identifier, and mark the access flag stored in the fourth table as accessed.

[0031] The above steps are repeated based on the values ​​of the access flag and the corresponding auto-incrementing identifier for each grid until the first table is traversed to complete the two-dimensional clustering object search.

[0032] According to the present invention, a point cloud clustering method using HDL is provided. The search conditions include a preset first threshold and a neighborhood range value. Accordingly, for any grid, a neighboring point search is performed using the neighborhood range value. The number of grids in the neighboring grids that contain point cloud data is compared with the first threshold. The number of grids that is greater than or equal to the first threshold is considered to satisfy the search conditions.

[0033] According to the present invention, a point cloud clustering method using HDL is provided, wherein enumerating the clustering identifiers and clustering objects with the same clustering identifier to complete the clustering of the point cloud data includes:

[0034] Enumerate the value of each cluster identifier in the third table, and cluster the point cloud data stored in the second table where objects with the same cluster identifier value are located, so as to complete the three-dimensional clustering of the point cloud data.

[0035] The present invention also provides a point cloud clustering device using HDL, comprising:

[0036] The point cloud clustering processing stage division module is used to process the point cloud data generated by the vehicle-mounted LiDAR in stages, including the parsing stage, calibration stage, elimination stage, two-dimensional meshing stage, object search stage and clustering stage, to transform the point cloud data from three-dimensional to two-dimensional plane until the point cloud clustering is completed.

[0037] The HDL parallel computing module is used to perform HDL parallel computing at each stage. Data transmission between stages is buffered using a FIFO buffer to achieve point cloud clustering in a parallel pipeline manner in the hardware circuit supported by HDL.

[0038] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the point cloud clustering method using HDL as described above.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the point cloud clustering method using HDL as described above.

[0040] This invention provides a method, apparatus, electronic device, and storage medium for point cloud clustering using HDL. Based on the obstacle avoidance characteristics of autonomous driving, this invention divides the point cloud clustering process into multiple stages: a parsing stage, a calibration stage, a elimination stage, a 2D meshing stage, an object search stage, and a clustering stage. Point cloud data is processed in stages, completing the transformation from 3D to a 2D plane until final point cloud clustering is achieved. Each stage employs parallel computation using HDL, executing in a parallel pipeline manner within HDL-supported hardware circuits, resulting in superior performance and a simpler implementation. Data transmission between stages uses a FIFO buffer, and the algorithm executes more rapidly, improving computation speed. This allows the technical solution of this invention to meet the real-time clustering requirements of autonomous driving. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This invention provides a flowchart illustrating a point cloud clustering method implemented using HDL.

[0043] Figure 2 This invention provides a logic diagram for implementing a point cloud clustering method using HDL, in which each stage is executed in parallel in a pipeline manner.

[0044] Figure 3 This is a schematic diagram illustrating the determination of the point cloud setting range based on a three-dimensional vehicle coordinate system in a point cloud clustering method using HDL provided by the present invention.

[0045] Figure 4 This is a schematic diagram illustrating the establishment of a two-dimensional mesh model in a point cloud clustering method using HDL provided by the present invention.

[0046] Figure 5 This is a schematic diagram illustrating how a point cloud clustering method using HDL maps point cloud data to a two-dimensional table and generates a two-dimensional gridded point cloud plane, as provided by the present invention.

[0047] Figure 6 This is a schematic diagram of a point cloud clustering device using HDL provided by the present invention;

[0048] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0049] Figure label:

[0050] 21: Point cloud clustering processing stage partitioning module; 22: HDL parallel computing module. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] LiDAR (Light Detection and Ranging) is a sensor that detects obstacles by sensing its surrounding environment and is widely used in fields such as autonomous vehicles and robotics. The process of LiDAR sensing its environment includes several main steps: segmentation / detection, tracking, recognition, and reconstruction. Among these, clustering algorithms are commonly used in the segmentation and detection processes. Clustering is the process of aggregating and classifying the points detected by the LiDAR; points belonging to the same cluster can be understood as points detected on the same object. Therefore, point clustering is crucial for LiDAR's environmental perception and directly affects the final obstacle localization result.

[0053] HDL is a hardware description language that describes the structure and behavior of digital system hardware in text form. It can be used to represent logic circuit diagrams, logic expressions, and the logical functions performed by digital logic systems.

[0054] Example 1

[0055] Reference Figure 1 As shown in the figure, this embodiment provides a point cloud clustering method using HDL, including:

[0056] Step S1: The point cloud data generated by the vehicle-mounted LiDAR is processed in stages, including parsing, calibration, elimination, 2D meshing, object search, and clustering, to transform the point cloud data from three-dimensional to two-dimensional plane until point cloud clustering is completed.

[0057] Step S2: The HDL is used to perform parallel computation in each stage, and the data transmission between stages is buffered using a FIFO buffer, so as to realize point cloud clustering in a parallel pipeline manner in the hardware circuit supported by HDL.

[0058] Specifically, the technical solution provided in this application is used to achieve point cloud clustering, which can be applied to various scenarios that require processing of LiDAR point cloud data, such as obstacle detection in autonomous vehicles during operation. This facilitates accurate and rapid obstacle detection, allowing for quick path planning based on obstacle information. Therefore, this method can be applied to obstacle detection systems (detection devices) for autonomous vehicles. Based on the obstacle avoidance characteristics of autonomous driving, this invention divides the point cloud clustering process into multiple stages: parsing, calibration, elimination, 2D meshing, object search, and clustering. Point cloud data is processed in stages, completing the transformation from three-dimensional to two-dimensional planes until point cloud clustering is finally achieved.

[0059] This invention employs HDL parallel computation for each of the aforementioned stages. Unlike existing algorithms, which are written in sequential languages ​​(such as C), where code executes in the order it's written, HDL parallel statements execute in parallel. Unlike C code, which is executed by a single CPU, HDL code is synthesized and adapted to specific FPGAs or CPLDs. Each parallel statement corresponds to a specific circuit, and each circuit can be considered a CPU, allowing for simultaneous execution. Therefore, applying these stages to HDL-supported hardware circuits in a parallel pipeline manner results in superior performance and a simpler implementation. Data transfer between stages uses a FIFO buffer, enabling hardware acceleration and faster computation, thus solving the real-time clustering problem of 3D point clouds. This ensures that the technical solution of this invention meets the real-time point cloud clustering requirements of autonomous driving, guaranteeing real-time clustering performance.

[0060] In this embodiment, step S1 includes:

[0061] Analysis Phase S100: Analyze the point cloud data generated by the vehicle-mounted LiDAR;

[0062] Calibration Phase S200: The point cloud data is converted to a 3D vehicle coordinate system for calibration;

[0063] Elimination phase S300: Eliminate point cloud data that does not meet the preset conditions in the three-dimensional vehicle coordinate system;

[0064] 2D Meshing Stage S400: Maps point cloud data to a 2D plane and meshes it to generate a 2D meshed point cloud plane;

[0065] Object search phase S500: Perform two-dimensional clustering object search in the two-dimensional gridded point cloud plane and store the clustering identifiers corresponding to different objects;

[0066] Clustering phase S600: Enumerate cluster identifiers and cluster objects with the same cluster identifier to complete the clustering of point cloud data.

[0067] Specifically, this embodiment is designed with 6 pipelines, from the parsing stage S100 to the clustering stage S600. These 6 stages are executed in parallel using 6 pipelines, achieving simultaneous parallel computation at each stage and optimizing resource consumption. Figure 2 As shown. This invention is not limited to six pipelines; the number of pipelines executing in parallel can be adjusted according to the actual hardware resources used.

[0068] In this embodiment, the elimination stage S300 involves eliminating point cloud data that does not meet preset conditions in the three-dimensional vehicle coordinate system, specifically including:

[0069] Step S301: Set the origin at the middle position of the vehicle's front in the three-dimensional vehicle coordinate system;

[0070] Step S302: Set the positive direction of the X-axis as the front of the vehicle from left to right, the positive direction of the vehicle's driving direction as the positive direction of the Y-axis, and the positive direction of the road as the positive direction of the Z-axis.

[0071] Step S303: Set the range in the X-axis, Y-axis and Z-axis directions with preset first length, second length and third length respectively, and delete the point cloud data that does not meet the range settings, that is, delete the point cloud data outside the road, so that the point cloud data is concentrated in the road in the direction of vehicle travel.

[0072] Specifically, LiDAR data transmission is via Ethernet's UDP protocol. The acquired point cloud data is transmitted in data frames. Parsing the point cloud data generated by the vehicle-mounted LiDAR is the first step. This point cloud data contains objects to be classified; the point cloud data generated in this step is called PO. The point cloud data is then converted to a 3D vehicle coordinate system for calibration, requiring multiplication by a correction matrix M (i.e., PO*M) to obtain the point cloud data P in the 3D vehicle coordinate system. During autonomous driving, point cloud data outside the road is useless and needs to be discarded. Therefore, as... Figure 3 As shown, in the 3D vehicle coordinate system, the center position of the vehicle's head is set as the origin. The positive direction of the X-axis is set from left to right of the vehicle's head, the positive direction of the vehicle's travel direction is set as the positive direction of the Y-axis, and the top of the road is set as the positive direction of the Z-axis. The first length on the X-axis is set to XSIZE, the second length on the Y-axis is set to YSIZE, and the third length on the Z-axis is set to ZSIZE. The range of the point cloud data required is set according to the following conditions:

[0073] X-axis direction: -XSIZE / 2≤PX≤XSIZE / 2;

[0074] Y-axis direction: 0≤PY≤YSIZE;

[0075] Z-axis direction: 0≤PY≤ZSIZE;

[0076] Point cloud data that does not meet any of the above conditions will be deleted from the point cloud dataset. This means deleting point cloud data outside the road and concentrating the point cloud data on the road in the direction of vehicle travel. For point cloud data within the road, the distances to the front, rear, left, and right of the vehicle are determined according to actual calculation needs, and are not limited to the range of the above conditions.

[0077] In this embodiment, the two-dimensional meshing stage S400: mapping the point cloud data to a two-dimensional plane and meshing it to generate a two-dimensional meshed point cloud plane, specifically includes:

[0078] Step S401: Map the point cloud data within the range setting to a two-dimensional plane formed by the X-axis and Y-axis, and establish a meshed model based on the preset grid;

[0079] Step S402: Determine the dimensions in the X and Y axes based on the grid dimensions;

[0080] Step S403: Establish a first table and a second table with the same dimension based on the dimension. For each grid, the first table stores an indicator of whether point cloud data exists in the grid, and the second table stores the point cloud data corresponding to the grid. After traversing all grids, generate a two-dimensional gridded point cloud plane in the form of a two-dimensional table.

[0081] Specifically, by projecting from the top to the ground, the bottom creates a two-dimensional plane formed by the X and Y axes. Autonomous vehicles use this two-dimensional plane grid to determine the presence of objects within the grid and perform clustering. Therefore, as... Figure 4 As shown, the 3D point cloud data is mapped to a 2D plane and meshed. The mesh size is set to CellX and CellY. Therefore, the dimension on the X-axis is XSIZE / CellX, denoted as KX, and the dimension on the Y-axis is YSIZE / CellY, denoted as KY. This creates a 2D table with KX and KY as dimensions. Based on the KX and KY dimensions, a first table T1 and a second table T2 are created. The first table T1 stores an indicator of whether point cloud data exists in the mesh, set using Boolean values ​​("1" indicates that point cloud data exists in the mesh, "0" indicates that point cloud data does not exist in the mesh). Figure 5As shown, if the current point is (x, y), then T1[x][y] is 1, otherwise it is 0. The second table T2 stores the point cloud data set, which is stored in array form. For example, if it stores 40 sets of point cloud data (i.e., T2.PointClound

[40] ), then the point cloud data of the current point is T2.PointClound[i] = (x, y). Therefore, after traversing all the grids, a two-dimensional gridded point cloud plane is generated in the form of a two-dimensional table. The size of the grid can be set according to actual needs.

[0082] In this embodiment, the object search stage S500 involves: performing a two-dimensional clustering object search in the two-dimensional meshed point cloud plane, and storing the clustering identifiers corresponding to different objects, including:

[0083] Based on the two-dimensional meshed point cloud plane, a third table and a fourth table with the same dimensions as the first table are established. The third table is used to store the clustering identifier of the object, and the fourth table is used to store the access identifier.

[0084] After setting the search criteria, iterate through the first table to search for grids containing point cloud data.

[0085] During the traversal, for any grid containing point cloud data, the point cloud data that meets the search criteria is taken as the searched object, the auto-increment identifier is used as the cluster identifier for different objects, and the neighboring points that meet the search criteria are written to the FIFO queue in an indexed manner.

[0086] Retrieve index information from the FIFO queue and use whether the corresponding point cloud data can be read as a criterion. If so, store the clustering identifier saved in the third table with the current auto-increment identifier value, and mark the access flag saved in the fourth table as accessed.

[0087] Repeat the above steps based on the access flag and the corresponding auto-incrementing flag value for each grid until the first table is traversed to complete the two-dimensional clustering object search.

[0088] Furthermore, the search conditions include a preset first threshold and a neighborhood range value. Accordingly, for any grid, a neighboring point search is performed using the neighborhood range value. The number of grids in the neighboring grids that contain point cloud data is compared with the first threshold. Grids with a number greater than or equal to the first threshold are considered to meet the search conditions.

[0089] Specifically, based on the two-dimensional meshed point cloud plane, a third table T3 and a fourth table T4 with the same dimensions as the first table T1 are first established. The third table T3 is used to store the cluster identifier (clusterid) of the object, and the fourth table T4 is used to store the access flag. The two-dimensional clustered object search process is as follows:

[0090] Step S501: Initialization settings: Set cluster identifier clusterid=0, starting coordinates x=0, y=0, first threshold to KN, neighborhood range value to N, and third table T3 and fourth table T4 to be empty tables;

[0091] Step S502: Search the first table T1. If T1[x][y] is 0, continue searching for the next point. If T1[x][y] is 1, proceed to the next step.

[0092] Step S503: Check if the [x][y] access flags in the fourth table T4 are marked. If they are marked, it means that the point has been searched, then proceed to step S502; otherwise, proceed to the next step.

[0093] Step S504: Perform a neighborhood search on (x, y) with a neighborhood range value N, obtain the number of grids with the corresponding indicator marked as 1 in the neighboring grids, and compare the number of grids with the first threshold KN. If the number of grids obtained is less than the first threshold KN, then jump to step S502 to continue the search; otherwise, proceed to the next step.

[0094] Step S505: After incrementing the cluster identifier (i.e., clusterid++), proceed to the next step;

[0095] Step S506: Write the neighboring points of (x, y) that meet the search conditions to the FIFO queue in an indexed manner; based on the searched objects, set the third table T3(x, y) to the value of the current clusterid, and mark the access flag of the fourth table T4(x, y), and proceed to the next step;

[0096] Step S507: If a point (xi, yi) can be read from the FIFO queue, proceed to step S506 and perform the above settings on the corresponding (x, y) in the third table T3 and the fourth table T4; otherwise, proceed to step S502.

[0097] Step S508: Continue until the first table T1 has been traversed to complete the two-dimensional clustering object search.

[0098] After the point cloud data is converted from 3D to 2D plane, the above steps enable object search in the 2D table, simplifying the complexity of 3D point cloud data clustering calculation, thereby simplifying the calculation, improving the calculation speed, and achieving more stable perception results in practical applications.

[0099] In this embodiment, the clustering stage S600 involves enumerating cluster identifiers and clustering objects with the same cluster identifier to complete the clustering of point cloud data, including:

[0100] Enumerate the value of each cluster identifier in the third table, and cluster the point cloud data stored in the second table containing objects with the same cluster identifier value to complete the three-dimensional clustering of the point cloud data.

[0101] Specifically, after the object search phase S500 ends, the third table T3 stores the cluster identifiers of the objects. That is, each clusterid in T3 represents an object. By enumerating the values ​​of the same clusterid in T3, point cloud clustering can be completed. For example, enumerating the objects with clusterid=1 in T3 and placing the point cloud data stored in the second table T2 together constitutes the clustering of one of the objects, thus achieving three-dimensional clustering of the point cloud data.

[0102] Example 2

[0103] like Figure 6 As shown, this embodiment provides a point cloud clustering device using HDL, including:

[0104] The point cloud clustering processing stage division module 21 is used to process the point cloud data generated by the vehicle-mounted LiDAR in stages, including the parsing stage, calibration stage, elimination stage, two-dimensional meshing stage, object search stage and clustering stage, to transform the point cloud data from three-dimensional to two-dimensional plane until the point cloud clustering is completed.

[0105] HDL parallel computing module 22 is used to perform HDL parallel computing at each stage, and the data transmission between stages uses a FIFO buffer to achieve point cloud clustering in a parallel pipeline manner in the hardware circuit supported by HDL.

[0106] In this embodiment, the point cloud clustering processing stage division module 21 specifically includes: a parsing stage submodule, used to parse the point cloud data generated by the vehicle-mounted LiDAR; a calibration stage submodule, used to convert the point cloud data to a three-dimensional vehicle coordinate system for calibration; a elimination stage submodule, used to eliminate point cloud data that does not meet preset conditions in the three-dimensional vehicle coordinate system; a two-dimensional meshing stage submodule, used to map the point cloud data to a two-dimensional plane and mesh it to generate a two-dimensional meshed point cloud plane; an object search stage submodule, used to perform two-dimensional clustering object search in the two-dimensional meshed point cloud plane and store the clustering identifiers corresponding to different objects; and a clustering stage submodule, used to enumerate the clustering identifiers and cluster objects with the same clustering identifier to complete the clustering of the point cloud data.

[0107] Furthermore, the elimination stage submodule specifically includes: an origin setting unit, used to set the middle position of the vehicle's head as the origin in the three-dimensional vehicle coordinate system; an axis direction setting unit, used to set the positive direction of the X-axis as the vehicle's head from left to right, the positive direction of the Y-axis as the vehicle's driving direction, and the positive direction of the Z-axis as the road's top; and a point cloud data range setting unit, used to set the range in the X-axis, Y-axis, and Z-axis directions with preset first, second, and third lengths respectively, and delete point cloud data that does not meet the range settings, that is, delete point cloud data outside the road, so that the point cloud data is concentrated in the road in the vehicle's driving direction.

[0108] The 2D meshing stage submodule specifically includes: a meshing model building unit, used to map point cloud data within the set range to a 2D plane formed by the X and Y axes, and to build a meshing model based on a preset mesh; a dimension determination unit, used to determine the dimensions in the X and Y axis directions based on the size of the mesh; and a table building unit, used to build a first table and a second table with the same dimensions based on the dimensions. For each mesh, the first table uses a boolean value to store whether point cloud data exists in the mesh, and the second table uses an array to store the point cloud data corresponding to the mesh. After traversing all meshes, a 2D meshed point cloud plane is generated in the form of a 2D table.

[0109] The object search phase submodule specifically includes: an initialization unit, used for initialization settings: setting the cluster identifier clusterid=0, starting coordinates x=0, y=0, first threshold KN, neighborhood range value N, and the third table T3 and the fourth table T4 as empty tables; a first search unit, used to search the first table T1. If T1[x][y] is 0, then continue searching for the next point; if T1[x][y] is 1, then enter the second search unit; a second search unit, used to check whether the access flag [x][y] of the fourth table T4 is marked. If it is marked, it means that the point has been searched, then jump to the first search unit to continue searching; otherwise, enter the neighbor point search unit; a neighbor point search unit, used to perform a neighbor point search on (x, y) with a neighborhood range value N, and to find the nearest neighbor point with the nearest neighbor value N. After adding the number of objects with the same auto-incremental identifier value (clusterid), if the result is less than or equal to the threshold KN, the search jumps to the first search unit to continue searching; otherwise, it enters the cluster identifier increment unit. The cluster identifier increment unit is used for cluster identifier increment processing. The write unit is used to write the neighboring points of (x, y) that meet the search conditions to the FIFO queue in an indexed manner. The first setting unit is used to set the value of the third table T3(x, y) to the current clusterid value based on the searched objects. The second setting unit is used to mark the access flag of the fourth table T4(x, y) based on the searched objects. The read unit is used to jump to the first setting unit and the second setting unit to perform the above settings if a point (xi, yi) can be read from the FIFO queue.

[0110] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method. Therefore, relevant parts can be referred to in the description of the method embodiment, and will not be repeated here.

[0111] The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division. In actual implementation, there may be other division methods. In the embodiments, each functional module can be integrated into a processor, or each module can be a separate device, or two or more modules can be integrated into a device. Each functional module in each embodiment can be implemented in hardware or in the form of hardware plus software functional units.

[0112] Example 3

[0113] like Figure 7As shown, this embodiment provides an electronic device, which includes a processor 310, a communication interface 320, a memory 330, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions stored in the memory 330. The processor 310 executes a point cloud clustering method using HDL as described in the above method embodiment. This method includes:

[0114] The point cloud data generated by the vehicle-mounted LiDAR is processed in stages, including parsing, calibration, elimination, 2D meshing, object search, and clustering, transforming the point cloud data from three dimensions to a two-dimensional plane until point cloud clustering is completed.

[0115] Each stage employs HDL parallel computation, and data transfer between stages uses a FIFO buffer to achieve point cloud clustering in a parallel pipeline manner within the hardware circuit supported by HDL.

[0116] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute a point cloud clustering method using HDL as described in the above method embodiments, the method comprising:

[0118] The point cloud data generated by the vehicle-mounted LiDAR is processed in stages, including parsing, calibration, elimination, 2D meshing, object search, and clustering, transforming the point cloud data from three dimensions to a two-dimensional plane until point cloud clustering is completed.

[0119] Each stage employs HDL parallel computation, and data transfer between stages uses a FIFO buffer to achieve point cloud clustering in a parallel pipeline manner within the hardware circuit supported by HDL.

[0120] Example 4

[0121] This embodiment provides a vehicle capable of assisted driving and / or autonomous driving, the vehicle including the electronic devices described above.

[0122] Example 5

[0123] This embodiment provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a point cloud clustering method using HDL as described in the above method embodiment. The method includes:

[0124] The point cloud data generated by the vehicle-mounted LiDAR is processed in stages, including parsing, calibration, elimination, 2D meshing, object search, and clustering, transforming the point cloud data from three dimensions to a two-dimensional plane until point cloud clustering is completed.

[0125] Each stage employs HDL parallel computation, and data transfer between stages uses a FIFO buffer to achieve point cloud clustering in a parallel pipeline manner within the hardware circuit supported by HDL.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A point cloud clustering method using HDL, characterized in that, include: The point cloud data generated by the vehicle-mounted LiDAR is processed in stages, including parsing, calibration, elimination, 2D meshing, object search, and clustering, transforming the point cloud data from three dimensions to a two-dimensional plane until point cloud clustering is completed. Each stage is computed in parallel using HDL, and data transfer between stages is buffered using a FIFO buffer, so that point cloud clustering can be implemented in a parallel pipeline manner in the hardware circuit supported by HDL. The point cloud data within the range setting is mapped to a two-dimensional plane formed by the X and Y axes, and a meshed model is established based on a preset grid. The dimensions in the X and Y axes are determined based on the size of the grid. Based on the dimension, a first table and a second table of the same dimension are established. For each grid, the first table stores an indicator of whether the point cloud data exists in the grid, and the second table stores the point cloud data corresponding to the grid. After traversing all the grids, the two-dimensional gridded point cloud plane is generated in the form of a two-dimensional table. Based on the two-dimensional meshed point cloud plane, a third table and a fourth table with the same dimensions as the first table are established. The third table is used to store the clustering identifier of the object, and the fourth table is used to store the access flag. After setting the search conditions, the first table is traversed to search for the grid where the point cloud data exists. During the traversal, for any grid containing the point cloud data, the point cloud data that meets the search conditions is taken as the searched object, an incremental identifier is used as the cluster identifier for different objects, and the neighboring points that meet the search conditions are written to the FIFO queue in an indexed manner. Retrieve index information from the FIFO queue and use whether the corresponding point cloud data can be read as a judgment condition. If so, store the clustering identifier stored in the third table with the current value of the auto-increment identifier, and mark the access flag stored in the fourth table as accessed. The above steps are repeated based on the values ​​of the access flag and the corresponding auto-incrementing identifier for each grid until the first table is traversed to complete the two-dimensional clustering object search.

2. The point cloud clustering method using HDL according to claim 1, characterized in that, The point cloud data generated by the vehicle-mounted LiDAR is processed in stages, including parsing, calibration, elimination, 2D meshing, object search, and clustering, transforming the point cloud data from three dimensions to a two-dimensional plane until point cloud clustering is completed. Analysis phase: Analyzing the point cloud data generated by the vehicle-mounted LiDAR; Calibration phase: The point cloud data is converted to a three-dimensional vehicle coordinate system for calibration; Elimination stage: Eliminate point cloud data that do not meet the preset conditions in the three-dimensional vehicle coordinate system; Two-dimensional meshing stage: The point cloud data is mapped to a two-dimensional plane and meshed to generate a two-dimensional meshed point cloud plane; Object search phase: Two-dimensional clustering object search is performed in the two-dimensional gridded point cloud plane, and the clustering identifiers corresponding to different objects are stored; Clustering stage: The clustering identifiers are enumerated, and objects with the same clustering identifier are clustered together to complete the clustering of the point cloud data.

3. The point cloud clustering method using HDL according to claim 2, characterized in that, The step of removing point cloud data that does not meet the preset conditions in the three-dimensional vehicle coordinate system includes: In the three-dimensional vehicle coordinate system, the origin is set at the middle position of the vehicle's front. The positive direction of the X-axis is set from left to right for the front of the vehicle, the positive direction of the vehicle's travel direction is set as the positive direction of the Y-axis, and the positive direction of the road is set as the positive direction of the Z-axis. The X, Y, and Z axes are respectively set with a preset first length, a second length, and a third length. Point cloud data that does not meet the set range is deleted, that is, point cloud data outside the road is deleted, so that the point cloud data is concentrated on the road in the direction of vehicle travel.

4. The point cloud clustering method using HDL according to claim 1, characterized in that, The search conditions include a preset first threshold and a neighborhood range value. Accordingly, for any given grid, a neighboring point search is performed using the neighborhood range value. The number of grids containing point cloud data in the neighboring grids is compared with the first threshold, and the number of grids greater than or equal to the first threshold is considered to satisfy the search conditions.

5. The point cloud clustering method using HDL according to claim 2, characterized in that, The step of enumerating the clustering identifiers and clustering objects with the same clustering identifier to complete the clustering of the point cloud data includes: Enumerate the value of each cluster identifier in the third table, and cluster the point cloud data stored in the second table where objects with the same cluster identifier value are located, so as to complete the three-dimensional clustering of the point cloud data.

6. A point cloud clustering device using HDL, characterized in that, include: The point cloud clustering processing stage division module is used to process the point cloud data generated by the vehicle-mounted LiDAR in stages, including the parsing stage, calibration stage, elimination stage, two-dimensional meshing stage, object search stage and clustering stage, to transform the point cloud data from three-dimensional to two-dimensional plane until the point cloud clustering is completed. The HDL parallel computing module is used to perform HDL parallel computing at each stage. Data transmission between stages uses a FIFO buffer to achieve point cloud clustering in a parallel pipeline manner in the hardware circuit supported by HDL. The point cloud clustering processing stage division module includes a two-dimensional meshing stage sub-module and an object search stage sub-module; The two-dimensional meshing stage submodule includes: The gridded model building unit is used to map the point cloud data within the range setting to a two-dimensional plane formed by the X-axis and Y-axis, and to build a gridded model based on a preset grid. A dimension determination unit is used to determine the dimensions in the X-axis and Y-axis directions based on the size of the grid, respectively; The table creation unit is used to create a first table and a second table with the same dimension based on the dimension. For each grid, the first table stores an indicator of whether the point cloud data exists in the grid, and the second table stores the point cloud data corresponding to the grid. After traversing all the grids, the two-dimensional gridded point cloud plane is generated in the form of a two-dimensional table. The object search phase submodule is used for: Based on the two-dimensional meshed point cloud plane, a third table and a fourth table with the same dimensions as the first table are established. The third table is used to store the clustering identifier of the object, and the fourth table is used to store the access flag. After setting the search conditions, the first table is traversed to search for the grid where the point cloud data exists. During the traversal, for any grid containing the point cloud data, the point cloud data that meets the search conditions is taken as the searched object, an incremental identifier is used as the cluster identifier for different objects, and the neighboring points that meet the search conditions are written to the FIFO queue in an indexed manner. Retrieve index information from the FIFO queue and use whether the corresponding point cloud data can be read as a judgment condition. If so, store the clustering identifier stored in the third table with the current value of the auto-increment identifier, and mark the access flag stored in the fourth table as accessed. The above steps are repeated based on the values ​​of the access flag and the corresponding auto-incrementing identifier for each grid until the first table is traversed to complete the two-dimensional clustering object search.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the point cloud clustering method using HDL as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the point cloud clustering method using HDL as described in any one of claims 1-5.

Images