Vehicle point cloud automatic acquisition method and system for specific position and angle

Abstract

The invention discloses a vehicle point cloud automatic acquisition method and system for a specific position and angle. The method comprises the following steps: acquiring point cloud data of a frame when there is no vehicle; performing RANSAC fitting on the preprocessed point cloud data to obtain an earth surface plane; calculating a three-dimensional transformation matrix based on an earth surface plane, and performing three-dimensional calibration on the preprocessed point cloud data; range filtering is carried out on the calibrated point cloud, a point cloud PCmasked is obtained, RANSAC fitting is carried out, a rotation matrix is obtained, and three-dimensional calibration is carried out on the point cloud PCmasked; performing histogram statistics on the calibrated point cloud to obtain an overall mask; obtaining an overall transformation matrix based on the three-dimensional transformation matrix and the rotation matrix; and based on the overall transformation matrix and the overall mask, carrying out three-dimensional calibration and range filtering on the point cloud collected in real time, then carrying out 3D grid segmentation, and selecting the point cloud collection opportunity based on the change conditions of the centroid space coordinates and the average reflectivity of the grid. According to the invention, the opportunity of collecting the approaching vehicle point cloud can be adaptively selected, the collection of empty lane information and vehicle position repetition information is avoided, and the storage space and the subsequent data cleaning time are saved.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data acquisition, and particularly to an automatic vehicle point cloud acquisition method, system, terminal device and computer-readable storage medium for specific positions and angles. Background Art

[0002] Currently, publicly available road surface point cloud datasets such as KITTI, SynthCity, nuScenes, and Waymo on the market cannot meet specific scenario requirements. The typical KITTI dataset is collected by multiple sensors on a self-driving vehicle, which does not match the height and angle of a specific scenario. In particular, there is too little data on various large vehicles, trucks, etc., and the data of different types of vehicles is unbalanced. Most of the other publicly available datasets are only applicable to autonomous driving and cannot meet specific scenario requirements.

[0003] Currently, several typical schemes for collecting point cloud data are as follows: (1) Three-dimensionally model the objects in the scenario, and then sample the vertex patches to obtain the point cloud, such as patent documents CN 112562067A and CN119091079A; (2) In view of the possible insufficient information of the radar scanning target, use a preset neural network for enhancement and then sampling, such as patent document CN112396067B; (3) Use multiple acquisition devices to collect block data of the target locally from different angles, and then perform point cloud fusion, such as patent documents CN116359942A and CN118898544A. For specific scenario requirements, for real-time acquisition with a single radar and a single industrial control device, considering the data acquisition cost and operation complexity, the software and hardware environments for data acquisition of the above schemes are not applicable.

[0004] Considering that the installation of the radar in specific scenario requirements is limited by the actual height of the on-site bracket, such as the differences in the building heights of different toll stations on highways, etc., and the hardware installation techniques of different technicians are different, only a general radar installation direction can be ensured. The non-uniformity of the installation position and angle adjustment will lead to differences in the coordinate system. The position and direction of the same object in different datasets will first be mislabeled due to this difference; secondly, the model cannot correctly learn the spatial relationship during training, resulting in problems in algorithm evaluation. At the same time, it is also necessary to calibrate the ground in the height direction to reduce data complexity and improve detection accuracy. Generally, if manual parameter adjustment is performed for the installation of the radar after on-site installation, the work efficiency is very low.

[0005] In the actual data acquisition process, since it is not necessary to collect all the data frames sent by the radar when the vehicle approaches, usually only the acquisition interval can be set manually, and it is impossible to avoid collecting empty lane information where there is no vehicle or the vehicle is stationary and information with repeated vehicle positions. Excessive point cloud data will not only burden some industrial control computers with insufficient storage space, but also greatly increase the workload of data cleaning and reduce the efficiency of data preparation and annotation. Summary of the Invention

[0006] To solve at least one of the above-mentioned prior art problems, the present invention provides a method, system, terminal device and computer-readable storage medium for automatic acquisition of vehicle point clouds at specific positions and angles. By fitting and positioning the ground detection area based on the Reuse Random Sample Consensus (RANSAC), the approximate equations of the ground surface plane and the straight lines on both sides of the lane are automatically obtained; by automatically selecting the point cloud acquisition timing based on the comprehensive changes of the centroid spatial coordinates and average reflectivity of the 3D segmentation grid, automatic acquisition of point cloud data is realized; and the scale consistency of the data collected by radars installed at different locations is maintained, effectively improving the debugging efficiency and data cleaning efficiency.

[0007] The first object of the present invention is to provide a method for automatic acquisition of vehicle point clouds at specific positions and angles.

[0008] The second object of the present invention is to provide a system for automatic acquisition of vehicle point clouds at specific positions and angles.

[0009] The third object of the present invention is to provide a terminal device.

[0010] The fourth object of the present invention is to provide a computer-readable storage medium.

[0011] The first object of the present invention can be achieved by adopting the following technical solutions:

[0012] A method for automatic acquisition of vehicle point clouds at specific positions and angles, the method comprising:

[0013] Collecting a frame of point cloud data when there is no vehicle as the initial point cloud data;

[0014] Performing RANSAC fitting on the preprocessed initial point cloud data to obtain the ground surface plane and the corresponding point cloud set;

[0015] Based on the ground surface plane, calculating a three-dimensional transformation matrix; using the three-dimensional transformation matrix to perform three-dimensional calibration on the preprocessed initial point cloud data;

[0016] Based on the point cloud set, performing range filtering on the three-dimensionally calibrated point cloud to obtain point cloud PC masked ; for point cloud PC maskedPerform RANSAC fitting to obtain the straight line of the road surface channel direction;

[0017] Based on the straight line of the road surface channel direction, calculate the rotation matrix; use the rotation matrix to perform three-dimensional calibration on the point cloud PC masked to obtain the corrected point cloud;

[0018] Perform histogram statistics on the corrected point cloud to calculate the lane position; based on the lane position, obtain the overall mask;

[0019] Based on the three-dimensional transformation matrix and the rotation matrix, obtain the overall transformation matrix;

[0020] Collect point clouds in real time and perform preprocessing; based on the overall transformation matrix and the overall mask, perform three-dimensional calibration and range filtering on the preprocessed point cloud again; perform 3D mesh segmentation on the point cloud after range filtering, and select the point cloud acquisition timing based on the change of the centroid spatial coordinates and average reflectivity of the 3D mesh to achieve automatic acquisition of vehicle point cloud data.

[0021] Further, before collecting point clouds in real time, it also includes:

[0022] Continuously collect point cloud data of n frames when there is no vehicle and perform preprocessing; then, based on the overall transformation matrix and the overall mask, perform three-dimensional calibration and range filtering on the preprocessed point cloud data; then, perform 3D mesh segmentation on the point cloud after range filtering along the vehicle driving direction, and calculate the average z coordinate z k0 and average x coordinate x k0 of the point cloud in each 3D mesh, as well as the average maximum and minimum reflectivities R kmin and R kmax as the lane parameters without vehicle; where k represents the grid number, n is a positive integer greater than or equal to 2 and less than or equal to ran x / 3, and ran x is the effective distance of radar scanning.

[0023] Further, the collecting point clouds in real time and performing preprocessing; based on the overall transformation matrix and the overall mask, performing three-dimensional calibration and range filtering on the preprocessed point cloud again; performing 3D mesh segmentation on the point cloud after range filtering, and selecting the point cloud acquisition timing based on the change of the centroid spatial coordinates and average reflectivity of the 3D mesh includes:

[0024] Collect a frame of point cloud data and perform preprocessing; then, based on the overall transformation matrix and the overall mask, perform three-dimensional calibration and range filtering on the preprocessed point cloud data; then, perform 3D mesh segmentation on the point cloud after range filtering along the vehicle driving direction, and calculate the average z coordinate height z k , average x coordinate depth x k and average reflectivity R k ;

[0025] Calculate the acquisition condition boolean quantity

[0026] If Collect is 0, return to acquire a frame of point cloud data, preprocess it, and continue to execute subsequent operations; if Collect is 1, set x k0 = x k , save the current point cloud frame, return to acquire a frame of point cloud data, preprocess it, and continue to execute subsequent operations.

[0027] Furthermore, the histogram statistics of the corrected point cloud and the calculation of the lane position include:

[0028] Perform histogram statistics on the corrected point cloud in the y-axis direction: Set the total number of bins for segmentation, and the coordinate range corresponding to each segmentation area is (E i , E i+1 ), and count the number of point clouds belonging to this coordinate range;

[0029] Find the index i of the maximum number of point clouds in the histogram statistics max ;

[0030] If i max ≤ bin / 2, the y coordinate on the left side of the lane is E imax+1 - w, and the y coordinate on the right side of the lane is E imax+1 ; if i max > bin / 2, the y coordinate on the left side of the lane is E imax , and the y coordinate on the right side of the lane is E imax + w; where w is the standard toll station exit width.

[0031] Furthermore, the equation of the ground plane is A 0 x + B 0 y + C 0 z + D 0 = 0; where x, y, and z represent the x, y, and z coordinates respectively, and A 0 , B 0 , C 0 , D 0 are all plane equation parameters;

[0032] The calculation of the three-dimensional transformation matrix based on the ground plane includes:

[0033] Convert the ground plane into a plane equation in the form of z = a 0 x + a 1 y + a 2 as follows:

[0034]

[0035] According to the plane equation, the rotation matrix R and the translation matrix T are obtained:

[0036]

[0037] Among them, h is the height from the ground when the radar collects the initial point cloud data;

[0038] According to the rotation matrix R and the translation matrix T, the three-dimensional transformation matrix is obtained

[0039] Furthermore, calculating the rotation matrix based on the straight line in the road surface channel direction includes:

[0040] Let the projection of the straight line in the road surface channel direction on the xoy plane be y = ax + a 3 x + a 4 ;

[0041] To transform the straight line y = ax + a 3 x + a 4 into y = 0, calculate the rotation angle θ = arctan(a 3 );

[0042] According to the rotation angle, the corresponding rotation matrix is obtained

[0043] Furthermore, performing RANSAC fitting on the preprocessed initial point cloud data to obtain the ground surface plane and the corresponding point cloud set includes:

[0044] Randomly select 3 non-collinear points from the preprocessed initial point cloud data PC dn to uniquely determine the plane M; among them, the equation of the plane M is Ax + By + Cz + D = 0, where x, y, and z represent the x, y, and z coordinates respectively, and A, B, C, and D are all plane equation parameters;

[0045] Traverse all points in the point cloud PC dn and calculate the distance from any point to the plane M:

[0046]

[0047] Count the number n of points that satisfy d m ≤ t m ; t 1 is the set plane distance threshold; m After iterating a specified number of times, take the plane corresponding to the maximum n

[0048] as the ground surface plane, and the set of all points with a distance to the ground surface plane ≤ t 1 is called the point cloud set. m

[0049] Furthermore, for the point cloud PC masked perform RANSAC fitting to obtain the straight line in the road surface channel direction, including:

[0050] Randomly select 2 points from the point cloud data PC masked with the difference in z coordinates less than a specified value to determine the straight line L; where the equation of the straight line L is (x - x 0 ) / a = (y - y 0 ) / b = (z - z 0 ) / c, and x 0 , y 0 , z 0 , a, b, and c are all parameters of the straight line equation;

[0051] Traverse all points in the point cloud PC masked and calculate the distance from any point to the straight line L:

[0052]

[0053] Count the number n l of points satisfying d l ≤ t 2 ; t l is the specified straight line distance threshold;

[0054] Iterate a specified number of times, and take the straight line corresponding to the maximum n 2 as the straight line in the road surface channel direction.

[0055] Furthermore, the preprocessing includes:

[0056] Set the neighborhood radius ε and the minimum number of points minPTS;

[0057] Traverse all points in the point cloud, and calculate the number of points PTS within the ε neighborhood of each point; if PTS ≥ minPTS, then this point is a core point, otherwise it is temporarily marked as a noise point;

[0058] Output all core points and all points within their ε neighborhoods as the denoised point cloud, and remove the remaining points as noise points. The second object of the present invention can be achieved by adopting the following technical solutions:

[0059] A vehicle point cloud automatic acquisition system for a specific position and angle, the system includes:

[0060] An initial acquisition module for acquiring a frame of point cloud data without a vehicle as the initial point cloud data;

[0061] A first fitting module for performing RANSAC fitting on the preprocessed initial point cloud data to obtain the ground surface plane and the corresponding point cloud set;

[0062] The first calibration module is used to calculate a three-dimensional transformation matrix based on the ground plane; and perform three-dimensional calibration on the preprocessed initial point cloud data by using the three-dimensional transformation matrix.

[0063] The second fitting module is used to perform range filtering on the point cloud after three-dimensional calibration based on the point cloud set to obtain point cloud PC masked ; perform RANSAC fitting on point cloud PC masked to obtain the straight line of the road surface channel direction.

[0064] The second calibration module is used to calculate a rotation matrix based on the straight line of the road surface channel direction; and perform three-dimensional calibration on point cloud PC masked by using the rotation matrix to obtain the corrected point cloud.

[0065] The calculation module is used to perform histogram statistics on the corrected point cloud to calculate the lane position; obtain the overall mask according to the lane position; and obtain the overall transformation matrix based on the three-dimensional transformation matrix and the rotation matrix.

[0066] The real-time acquisition module is used to acquire point cloud in real time and perform preprocessing; perform three-dimensional calibration and range filtering on the preprocessed point cloud again based on the overall transformation matrix and the overall mask; perform 3D grid segmentation on the point cloud after range filtering, and select the point cloud acquisition timing based on the change of the centroid spatial coordinates and the average reflectivity of the 3D grid, so as to realize the automatic acquisition of vehicle point cloud data.

[0067] The third object of the present invention can be achieved by adopting the following technical solutions:

[0068] A terminal device includes a processor and a memory for storing programs executable by the processor. When the processor executes the programs stored in the memory, the method for automatically acquiring vehicle point cloud at a specific position and angle as described above is realized.

[0069] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0070] A computer-readable storage medium stores a program, and when the program is executed by a processor, the method for automatically acquiring vehicle point cloud at a specific position and angle as described above is realized.

[0071] The present invention has the following beneficial effects compared with the prior art:

[0072] 1. The present invention can automatically locate the road surface area and perform automatic three-dimensional calibration, without the need for technicians to manually adjust parameters such as the rotation angle and displacement distance in the spatial coordinates, greatly improving the work efficiency and ensuring the detection accuracy.

[0073] 2. The present invention can adaptively select the timing of collecting the point cloud of approaching vehicles, avoiding the collection of information on empty lanes where there are no vehicles or the vehicles are stationary and information with repeated vehicle positions; in the case of no manual control, it greatly saves the storage space of the device and also saves the subsequent data cleaning time. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0075] Figure 1 It is a simple flowchart of the method for automatically collecting the vehicle point cloud at a specific position and angle in Embodiment 1 of the present invention.

[0076] Figure 2 It is a detailed flowchart of the method for automatically collecting the vehicle point cloud at a specific position and angle in Embodiment 1 of the present invention.

[0077] Figure 3 It is an effect diagram of using RANSAC to generate a fitted ground plane in Embodiment 1 of the present invention.

[0078] Figure 4 It is an effect diagram of using RANSAC to generate a fitted road channel in Embodiment 1 of the present invention.

[0079] Figure 5 Among them, (a), (b), and (c) are three consecutive point cloud images automatically collected in Embodiment 1 of the present invention.

[0080] Figure 6 It is a structural block diagram of the system for automatically collecting the vehicle point cloud at a specific position and angle in Embodiment 2 of the present invention.

[0081] Figure 7 It is a structural block diagram of the terminal device in Embodiment 3 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. It should be understood that the specific embodiments described are only used to explain the present application and are not used to limit the present application.

[0083] Example 1:

[0084] As Figure 1 , 2 shown, this example provides an automatic vehicle point cloud acquisition method for specific positions and angles, including the following steps:

[0085] S101. Install the radar and collect the initial point cloud data.

[0086] Furthermore, step S101 includes:

[0087] (1) Set up and fix the radar at the exit of the highway toll station, requiring the radar irradiation range to cover the entire toll station detection channel.

[0088] Fix the radar at a height of h meters from the ground and collect the point cloud data of approaching vehicles at a certain skew angle.

[0089] In this example, the value of h is approximately 4 meters.

[0090] (2) Use the radar to collect a frame of point cloud data PC when there is no vehicle. og .

[0091] S102. Preprocess the initial point cloud data to obtain the denoised point cloud.

[0092] In this example, based on the DBSCAN density clustering algorithm, the initial point cloud data is preprocessed.

[0093] Furthermore, step S102 includes:

[0094] (1) Set the neighborhood radius ε = 0.5 (the unit is meter, and all length units involved below are in meters), and the minimum number of points minPTS = 10.

[0095] (2) Traverse all points, calculate the number of points PTS within the ε neighborhood of each point. If PTS ≥ minPTS, then this point is a core point; otherwise, it is temporarily marked as a noise point.

[0096] (3) Take all core points and all points within their ε neighborhoods as the denoised point cloud PC dn output, and remove the remaining points as noise points.

[0097] S103. Perform RANSAC fitting on the denoised point cloud to obtain the ground surface plane and the corresponding point cloud set.

[0098] Furthermore, step S103 includes:

[0099] (1) From the point cloud data PC dnRandomly select 3 non - collinear points to uniquely determine a plane M, whose equation is Ax + By + Cz + D = 0; where x, y, and z represent the x, y, and z coordinates respectively, and A, B, C, and D are the plane equation parameters.

[0100] (2) Traverse the point cloud PC dn All points in it, and calculate its distance to the plane M:

[0101]

[0102] (3) Count the number n m ≤t m Of points that satisfy the condition. 1 ; t m Is the plane distance threshold, and the value in this embodiment is 0.1.

[0103] (4) Repeat the above steps (1) - (3) for 2000 iterations, and output the plane with the largest n 1 As the ground plane, and the set of all points whose distance to this plane satisfies d 1 ≤t 1 Is called P m , as Figure 3 Shown.

[0104] S104. Based on the ground plane, perform three - dimensional calibration on the denoised point cloud.

[0105] Perform the first three - dimensional calibration on the point cloud, rotate the ground plane to be parallel to the horizontal plane of the coordinate system, and translate it to z = - 4, that is, convert the ground plane equation Ax + By + Cz + D = 0 obtained in step S103 into the target plane equation.

[0106] Further, step S104 includes:

[0107] (1) Convert the ground plane into a plane equation in the form of z = a 0 x + a 1 y + a 2 The conversion is as follows:

[0108]

[0109] (2) Calculate the three - dimensional transformation matrix T m .

[0110] Calculate the three - dimensional transformation matrix T m , T m Can be decomposed into a rotation matrix and a translation matrix:

[0111]

[0112] Among them, the rotation matrix R and the translation matrix T are as follows:

[0113]

[0114] Among them, the parameter is calculated as and

[0115] (3) Use the three-dimensional transformation matrix T m to perform three-dimensional transformation on the denoised point cloud PC dn .

[0116] Use the three-dimensional transformation matrix T m to rotate and translate the denoised point cloud PC dn so that the ground position is basically flush with the target ground z = -4, and obtain the point cloud PC rotated .

[0117] S105. Based on the three-dimensionally calibrated point cloud and the point cloud set, obtain the point cloud PC masked ; perform RANSAC fitting on the point cloud PC masked to obtain the straight line in the road surface channel direction.

[0118] Furthermore, step S105 specifically includes:

[0119] (1) Remove the point cloud within the range of rotate from the point cloud PC , and then remove the corresponding point cloud of P m to obtain the point cloud PC masked .

[0120] (2) Randomly select 2 points from the point cloud data PC masked whose z-coordinate differences are less than 0.1 to determine a straight line L, and its equation is (x - x 0 ) / a = (y - y 0 ) / b = (z - z 0 ) / c, where x 0 , y 0 , z 0 , a, b, and c are all parameters of the straight line equation.

[0121] (3) Traverse all points in the point cloud PC masked and calculate its distance to the straight line L:

[0122]

[0123] (4) Count the number n l of points that satisfy d l ≤ t 2 ; t l is the straight line distance threshold, and the value in this embodiment is 0.1.

[0124] (5) Repeat the above steps (1) to (4) for 2000 iterations, and set n 2 The longest straight line is output as the straight line of the road surface channel direction, and the result is as Figure 4 shown.

[0125] S106. Calculate the rotation matrix based on the straight line of the road surface channel direction; use the rotation matrix to perform three-dimensional calibration on the point cloud PC masked to obtain the corrected point cloud.

[0126] Perform a second three-dimensional calibration on the point cloud to make the lane direction become the x-axis direction, that is, make the vehicle driving direction the x-axis direction.

[0127] Further, step S106 specifically includes:

[0128] (1) Calculate the projection of the straight line of the road surface channel direction on the xoy plane: y = a 3 x + a 4 .

[0129] (2) To make the fitted straight line y = a 3 x + a 4 transform into y = 0, calculate the rotation angle θ = arctan(a 3 ).

[0130] (3) Use the rotation matrix R corresponding to the rotation angle 2 to perform rotation correction on the point cloud PC masked to obtain the corrected point cloud PC rotated2 , and the matrix is as follows:[[]]

[0131]

[0132] S107. Based on the corrected point cloud, obtain the lane position and save it as a mask.

[0133] In this step, the y coordinates on both sides of the calibrated lane are obtained, and then the lane position is saved as a mask.

[0134] Further, step S107 includes:[[]]

[0135] (1) Perform a histogram statistics on the point cloud PC rotated2 along the y-axis direction. The total number of bins for segmentation is bin = 100, and the coordinate range corresponding to each segmentation region is (E i , E i+1 ), and count the number of point clouds belonging to this coordinate range.

[0136] (2) Find the index i of the maximum number of point clouds in the histogram statistics max .

[0137] (3) If i max ≤ bin / 2, then the y - coordinate on the left side of the lane is E imax+1 - w, and the y - coordinate on the right side is E imax+1 ; If i max > bin / 2, then the y - coordinate on the left side of the lane is E imax , and the y - coordinate on the right side is E imax + w; where w is the width of the highway toll station exit;

[0138] In this embodiment, the value of w is 4.5 meters.

[0139] (3) Save the lane position as a mask y ∈ (y 0 , y 1 ).

[0140] S108. Obtain the parameter of the lane without vehicles based on the mask.

[0141] Further, step S108 includes:

[0142] (1) Set the overall transformation matrix T total :

[0143]

[0144] (2) Set the overall mask M total :

[0145] z ∈ (- 4,00 ∩ x ∈ (0, ran x ) ∩ y ∈ (y 0 , y 1 ); where ran x is the effective radar scanning distance, and the value in this embodiment is 80.

[0146] (3) Obtain the parameter of the lane without vehicles based on the overall transformation matrix and the overall mask.

[0147] Specifically, step (3) includes:

[0148] (3 - 1) Continuously collect 10 frames of point cloud data when there is no vehicle. Pre - process the point cloud, and the method is the same as step S102. Use the overall transformation matrix T total to perform three - dimensional transformation on the pre - processed point cloud data, and remove the point cloud outside the range of the overall mask M total .

[0149] (3 - 2) Along the vehicle driving direction, that is, along x ∈ (0,80), divide a 3D grid every 10m, and generate a total of 8 grid regions.

[0150] (3 - 3) Calculate the average z - coordinate z k0 of the point cloud in each grid within 10 frames, and the average x - coordinate xk0 and the average maximum and minimum reflectivities R kmin 、R kmax , as the no-vehicle lane parameters, where k represents the grid number.

[0151] S109. Collect vehicle point cloud data in real time based on the no-vehicle lane parameters.

[0152] Further, step S109 includes:

[0153] (1) Collect a frame of point cloud data; preprocess the point cloud, and the method is the same as that in step S102; use the overall transformation matrix T total to perform three-dimensional transformation on the preprocessed point cloud data and remove the point cloud outside the range of the overall mask M total .

[0154] (2) Segment the point cloud, and the segmentation method is the same as that in (3-2) of step S108, and calculate the average z-coordinate height z k of the point cloud in each grid, the average x-coordinate depth x k and the average reflectivity R k , and calculate the acquisition condition boolean quantity:

[0155]

[0156] (3) If Collect is 0, that is, the acquisition condition is not satisfied, then no changes and processing are performed and return to step (1) to continue execution; if Collect is 1, that is, the acquisition condition is satisfied, let x k0 = x k , save the current point cloud frame and return to step (1) to continue execution.

[0157] Partial acquisition results of the vehicle point cloud data are as Figure 5 shown.

[0158] Those skilled in the art can understand that all or part of the steps in the method of implementing the above embodiments can be completed by instructing relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium.

[0159] It should be noted that although the method operations of the above embodiments are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the described steps can be changed in the execution order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0160] Embodiment 2:

[0161] AsFigure 6 As shown in Figure 6 , this embodiment provides a vehicle point cloud automatic acquisition system for a specific position and angle. The system includes an initial acquisition module 601, a first fitting module 602, a first calibration module 603, a second fitting module 604, a second calibration module 605, a calculation module 606, and a real-time acquisition module 607, where:

[0162] The initial acquisition module 601 is used to acquire a frame of point cloud data without a vehicle as the initial point cloud data;

[0163] The first fitting module 602 is used to perform RANSAC fitting on the preprocessed initial point cloud data to obtain the ground surface plane and the corresponding point cloud set;

[0164] The first calibration module 603 is used to calculate a three-dimensional transformation matrix based on the ground surface plane; use the three-dimensional transformation matrix to perform three-dimensional calibration on the preprocessed initial point cloud data;

[0165] The second fitting module 604 is used to perform range filtering on the three-dimensionally calibrated point cloud based on the point cloud set to obtain point cloud PC masked ; perform RANSAC fitting on point cloud PC masked to obtain the straight line of the road surface channel direction;

[0166] The second calibration module 605 is used to calculate a rotation matrix based on the straight line of the road surface channel direction; use the rotation matrix to perform three-dimensional calibration on point cloud PC masked to obtain the corrected point cloud;

[0167] The calculation module 606 is used to perform histogram statistics on the corrected point cloud, calculate the lane position; obtain the overall mask according to the lane position; obtain the overall transformation matrix based on the three-dimensional transformation matrix and the rotation matrix;

[0168] The real-time acquisition module 607 is used to acquire point clouds in real time; perform three-dimensional calibration and range filtering on the preprocessed point clouds based on the overall transformation matrix and the overall mask; perform 3D mesh segmentation on the point clouds after range filtering, and select the point cloud acquisition timing based on the change of the centroid spatial coordinates and the average reflectivity of the 3D mesh to achieve automatic acquisition of vehicle point cloud data.

[0169] For the specific implementation of each module in this embodiment, reference can be made to the above-mentioned Embodiment 1, which will not be elaborated here one by one; it should be noted that the system provided in this embodiment is only illustrated by the above-mentioned division of each functional module. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0170] Embodiment 3:

[0171] This embodiment provides a terminal device, which can be a computer, such as Figure 7 shown, which is connected to a processor 702, a memory, an input device 703, a display 704, and a network interface 705 through a system bus 701. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium 706 and an internal memory 707. The non-volatile storage medium 706 stores an operating system, a computer program, and a database. The internal memory 707 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor 702 executes the computer program stored in the memory, it implements the method for automatically collecting vehicle point clouds for a specific position and angle in the above-mentioned Embodiment 1, as follows:

[0172] Collect a frame of point cloud data when there is no vehicle as the initial point cloud data;

[0173] Perform RANSAC fitting on the preprocessed initial point cloud data to obtain the ground plane and the corresponding point cloud set;

[0174] Based on the ground plane, calculate the three-dimensional transformation matrix; use the three-dimensional transformation matrix to perform three-dimensional calibration on the preprocessed initial point cloud data;

[0175] Based on the point cloud set, perform range filtering on the three-dimensionally calibrated point cloud to obtain point cloud PC masked ; perform RANSAC fitting on point cloud PC masked to obtain the straight line of the road surface channel direction;

[0176] Based on the straight line of the road surface channel direction, calculate the rotation matrix; use the rotation matrix to perform three-dimensional calibration on point cloud PC masked to obtain the corrected point cloud;

[0177] Perform histogram statistics on the corrected point cloud, calculate the lane position; according to the lane position, obtain the overall mask;

[0178] Based on the three-dimensional transformation matrix and the rotation matrix, obtain the overall transformation matrix;

[0179] Collect point clouds in real time; perform three-dimensional calibration and range filtering on the preprocessed point clouds based on the overall transformation matrix and the overall mask; perform 3D mesh segmentation on the point clouds after range filtering, and select the point cloud acquisition timing based on the change of the centroid spatial coordinates and the average reflectivity of the 3D mesh to achieve the automatic acquisition of vehicle point cloud data.

[0180] Embodiment 4:

[0181] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for automatically collecting vehicle point clouds at specific positions and angles in Embodiment 1 above, as follows:

[0182] Collect a frame of point cloud data when there is no vehicle as the initial point cloud data;

[0183] Perform RANSAC fitting on the preprocessed initial point cloud data to obtain the ground plane and the corresponding point cloud set;

[0184] Based on the ground plane, calculate the three-dimensional transformation matrix; use the three-dimensional transformation matrix to perform three-dimensional calibration on the preprocessed initial point cloud data;

[0185] Based on the point cloud set, perform range filtering on the three-dimensionally calibrated point cloud to obtain point cloud PC masked ; For point cloud PC masked Perform RANSAC fitting to obtain the straight line in the road surface channel direction;

[0186] Based on the straight line in the road surface channel direction, calculate the rotation matrix; use the rotation matrix to perform three-dimensional calibration on point cloud PC masked to obtain the corrected point cloud;

[0187] Perform histogram statistics on the corrected point cloud, calculate the lane position; based on the lane position, obtain the overall mask;

[0188] Based on the three-dimensional transformation matrix and the rotation matrix, obtain the overall transformation matrix;

[0189] Collect point clouds in real time and; based on the overall transformation matrix and the overall mask, perform three-dimensional calibration and range filtering on the preprocessed point cloud again; perform 3D mesh segmentation on the point cloud after range filtering, and select the point cloud collection timing based on the change of the centroid spatial coordinates and average reflectivity of the 3D mesh to achieve automatic collection of vehicle point cloud data.

[0190] It should be noted that the computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0191] As mentioned above, the above is only a preferred embodiment of the present invention patent, but the protection scope of the present invention patent is not limited thereto. Any person skilled in the art within the scope disclosed by the present invention patent, according to the technical solution and inventive concept of the present invention patent, makes equivalent substitutions or changes, all of which belong to the protection scope of the present invention patent.

Claims

1. A method for automatically collecting vehicle point clouds at specific positions and angles, characterized in that: The method comprises: Collect a frame of point cloud data without a car as the initial point cloud data; Perform RANSAC fitting on the preprocessed initial point cloud data to obtain the surface plane and the corresponding point cloud set; Based on the ground plane, the three-dimensional transformation matrix is ​​calculated; the three-dimensional transformation matrix is ​​used to perform three-dimensional calibration on the pre-processed initial point cloud data; Based on the point cloud set, the point cloud after 3D calibration is range filtered to obtain the point cloud PC masked ; For point cloud PC masked Perform RANSAC fitting to obtain the road channel direction straight line; Based on the straight line of the road channel direction, calculate the rotation matrix; use the rotation matrix to PC the point cloud masked Perform three-dimensional calibration to obtain the corrected point cloud; Perform histogram statistics on the corrected point cloud and calculate the lane position; based on the lane position, obtain the overall mask; Based on the three-dimensional transformation matrix and the rotation matrix, the overall transformation matrix is ​​obtained; Collect point cloud in real time; perform three-dimensional calibration and range filtering on the pre-processed point cloud based on the overall transformation matrix and overall mask; perform 3D mesh segmentation on the point cloud after range filtering, and select the time to collect point cloud based on the changes in the centroid spatial coordinates and average reflectivity of the 3D mesh to achieve automatic collection of vehicle point cloud data.

2. The method for automatic vehicle point cloud acquisition according to claim 1, characterized in that: Before real-time point cloud acquisition, it also includes: Continuously collect n frames of point cloud data without a vehicle and perform preprocessing; then perform three-dimensional calibration and range filtering on the preprocessed point cloud data based on the overall transformation matrix and the overall mask; then, perform 3D grid segmentation on the point cloud after range filtering along the vehicle's driving direction, and calculate the average z coordinate z of the point cloud in each 3D grid of n frames k0 , the average x coordinate x k0 And the average maximum and minimum reflectivity R kmin , R kmax ; where k represents the grid number, and n is greater than or equal to 2 and less than or equal to ran x / 3 positive integer, ran x The effective distance of the radar scan.

3. The method for automatic vehicle point cloud acquisition according to claim 2, characterized in that: The real-time point cloud acquisition; performing three-dimensional calibration and range filtering on the pre-processed point cloud based on the overall transformation matrix and the overall mask; performing 3D grid segmentation on the point cloud after range filtering, and selecting the point cloud acquisition time based on the changes in the centroid space coordinates and average reflectivity of the 3D grid, including: Collect a frame of point cloud data and preprocess it; then perform three-dimensional calibration and range filtering on the preprocessed point cloud data based on the overall transformation matrix and the overall mask; then, perform 3D grid segmentation on the point cloud after range filtering along the vehicle's driving direction, and calculate the average z coordinate height z of the point cloud in each 3D grid k , the average x-coordinate depth x k and the average reflectivity R k ; Calculate the Boolean value of the acquisition condition If Collect is 0, return to collect a frame of point cloud data and perform preprocessing and continue to perform subsequent operations; if Collect is 1, let x k0 =x k , save the current point cloud frame, return to collect a frame of point cloud data, perform preprocessing and continue to perform subsequent operations.

4. The method for automatically collecting vehicle point clouds according to any one of claims 1 to 3, characterized in that: The above-mentioned histogram statistics are performed on the corrected point cloud to calculate the lane position, including: Perform histogram statistics on the corrected point cloud along the y-axis direction: Set the total number of segmentation bins, and the coordinate range corresponding to each segmentation area is (E i ,E i+1 ), count the number of point clouds within the coordinate range; Find the maximum point cloud index i in the histogram statistics max ; If i max ≤bin / 2, the y coordinate of the left side of the lane is E imax+1 -w, the y coordinate of the right side of the lane is E imax+1 ; if i max >bin / 2, the y coordinate of the left lane is E imax , the y coordinate of the right side of the lane is E imax +w; where w is the standard toll station exit width.

5. The method for automatically collecting vehicle point clouds according to any one of claims 1 to 3, characterized in that: The equation of the surface plane is A0x+B0y+C0z+D0=0; wherein x, y, z represent x, y, z coordinates respectively, and A0, B0, C0, D0 are all plane equation parameters; The method of calculating a three-dimensional transformation matrix based on the ground surface plane includes: The surface plane is converted into a plane equation in the form of z = a0x + a1y + a2. The conversion is as follows: According to the plane equation, we get the rotation matrix R and translation matrix T: in, h is the height from the ground when the radar collects the initial point cloud data; According to the rotation matrix R and translation matrix T, we get the three-dimensional transformation matrix 6. The method for automatically collecting vehicle point clouds according to any one of claims 1 to 3, characterized in that: The calculation of the rotation matrix based on the road channel direction straight line includes: Assume that the projection of the road channel direction straight line on the xoy plane is y=a3x+a4; To transform the straight line y=a3x+a4 to y=0, calculate the rotation angle θ=arctan(a3); According to the rotation angle, the corresponding rotation matrix is ​​obtained 7. The method for automatically collecting vehicle point clouds according to any one of claims 1 to 3, characterized in that: The RANSAC fitting is performed on the preprocessed initial point cloud data to obtain a surface plane and a corresponding point cloud set, including: From the preprocessed initial point cloud data PC dn Randomly select three non-collinear points to uniquely determine the plane M; the equation of the plane M is Ax+By+Cz+D=0, x, y, z represent the x, y, z coordinates respectively, and A, B, C, D are the parameters of the plane equation; Traversing Point Cloud PC dn For all points in , calculate the distance from any point to plane M: Statistics meet d m ≤t m The number of points n1; t m is the set plane distance threshold; After the specified number of iterations, the plane corresponding to the maximum n1 is taken as the ground plane, and the distance to the ground plane is ≤ t m The set of all points is called a point cloud set.

8. The method for automatically collecting vehicle point clouds according to any one of claims 1 to 3, characterized in that: Point Cloud PC masked Perform RANSAC fitting to obtain the road channel direction straight line, including: From point cloud data to PC masked Randomly select two points whose z coordinate difference is less than the specified value to determine the straight line L; the equation of the straight line L is (x-x0) / a=(y-y0) / b=(z-z0) / c, where x0, y0, z0, a, b, c are all parameters of the straight line equation; Traversing Point Cloud PC masked For all points in , calculate the distance from any point to the line L: Statistics meet d l ≤t l The number of points n2; t l is the specified straight-line distance threshold; Iterate a specified number of times and take the straight line corresponding to the maximum n2 as the straight line in the direction of the road channel.

9. The method for automatically collecting vehicle point clouds according to any one of claims 1 to 3, characterized in that: The preprocessing comprises: Set the neighborhood radius ε and the minimum number of points minPTS; Traverse all points in the point cloud and calculate the number of points PTS in the ε neighborhood of each point; if PTS ≥ minPTS, the point is a core point, otherwise it is temporarily marked as a noise point; All core points and all points in their ε neighborhood are output as denoised point clouds, and the remaining points are removed as noise points.

10. A vehicle point cloud automatic acquisition system for specific positions and angles, characterized in that: The system comprises: The initial acquisition module is used to acquire a frame of point cloud data without a vehicle as the initial point cloud data; The first fitting module is used to perform RANSAC fitting on the preprocessed initial point cloud data to obtain the surface plane and the corresponding point cloud set; The first calibration module is used to calculate a three-dimensional transformation matrix based on a ground plane; and to perform three-dimensional calibration on the pre-processed initial point cloud data using the three-dimensional transformation matrix; The second fitting module is used to filter the point cloud after 3D calibration based on the point cloud set to obtain the point cloud PC. masked ; For point cloud PC masked Perform RANSAC fitting to obtain the road channel direction straight line; The second calibration module is used to calculate the rotation matrix based on the straight line of the road channel direction; the rotation matrix is ​​used to calibrate the point cloud PC masked Perform three-dimensional calibration to obtain the corrected point cloud; The calculation module is used to perform histogram statistics on the corrected point cloud and calculate the lane position; obtain the overall mask based on the lane position; and obtain the overall transformation matrix based on the three-dimensional transformation matrix and the rotation matrix; The real-time acquisition module is used to acquire point clouds in real time; the pre-processed point clouds are calibrated in three dimensions and range filtered based on the overall transformation matrix and the overall mask; the point clouds after range filtering are segmented into 3D grids, and the point cloud acquisition timing is selected based on the changes in the spatial coordinates of the centroid and the average reflectivity of the 3D grid, so as to realize the automatic acquisition of vehicle point cloud data.

Images