Point cloud data processing method and device, computer device and storage medium

Abstract

The application relates to a point cloud data processing method and device, computer equipment and a storage medium. The method comprises the following steps: image matching is performed on preceding point cloud data and subsequent point cloud data to obtain a background motion amount and a first target motion amount of a laser radar compared with a detection target between the two groups of point cloud data; point cloud boundary detection is performed on the preceding point cloud data and the subsequent point cloud data to obtain a second target motion amount of the laser radar compared with the detection target between the two groups of point cloud data; and finally, compensation is performed on the subsequent point cloud data based on the background motion amount, the first target motion amount and the second target motion amount. After the point cloud data is compensated, the motion distortion of the point cloud can be effectively corrected.

Description

Technical Field

[0001] This application relates to the field of lidar technology, and in particular to a point cloud data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] LiDAR is one of the main perception sensors currently and in the future serving autonomous driving. Its measurement accuracy is an important indicator of radar performance, reflected in the accuracy of the output point cloud data. The accuracy of point cloud data does not depend entirely on hardware.

[0003] In normal driving conditions, radar travels at relatively high speeds, causing motion distortion in the scanned point cloud. This is because the point cloud is not generated simultaneously during radar scanning. In some radar imaging principles, a frame of point cloud is stitched together from point clouds scanned by two or more overlapping fields of view. This creates a time delay between different points, especially those in different fields of view. During this time, the radar's movement causes changes in the relative position between the radar and the target, resulting in the scanned points of the same target being generated in different coordinate systems. However, the radar still treats all points as being in a single coordinate system and outputs them, leading to motion distortion in the point cloud. Motion distortion reduces the radar's measurement accuracy, increasing the difficulty of target recognition and decreasing recognition accuracy in autonomous driving perception. Summary of the Invention

[0004] Therefore, it is necessary to provide a point cloud data processing method, apparatus, computer equipment, and computer-readable storage medium that can correct motion distortion of point clouds to address the above-mentioned technical problems.

[0005] Firstly, this application provides a point cloud data processing method, the method comprising:

[0006] Based on two sets of point cloud data, obtain the point cloud data in the foreground and the point cloud data in the background.

[0007] By performing image matching on the front point cloud data and the rear point cloud data, the background motion and first target motion of the lidar relative to the detected target are obtained between the two sets of point cloud data.

[0008] By performing point cloud boundary detection on the front point cloud data and the rear point cloud data, the second target motion of the lidar relative to the detected target between the two sets of point cloud data is obtained.

[0009] The background motion, the first target motion, and the second target motion are used to compensate the subsequent point cloud data.

[0010] In one embodiment, acquiring the preceding point cloud data and the following point cloud data based on two sets of point cloud data includes:

[0011] Delete the corresponding point cloud data of all ground points in the two sets of point cloud data to obtain the corresponding remaining point cloud data of each of the two sets of point cloud data.

[0012] The point cloud data that is located within the preset non-background range and was collected earlier in the two sets of remaining point cloud data is determined as the earlier point cloud data, and the point cloud data collected later is determined as the later point cloud data.

[0013] In one embodiment, image matching is performed between the preceding point cloud data and the following point cloud data to obtain the background motion of the lidar relative to the detected target between the two sets of point cloud data, including:

[0014] The front view is divided into grids, and the front grids are selected from all the grids obtained. The front view is obtained by mapping the front point cloud data onto a preset plane. The preset plane is a plane formed based on the horizontal and vertical directions. The vertical direction is the movement direction of the lidar parallel to the ground, and the horizontal direction is perpendicular to the vertical direction and parallel to the ground.

[0015] The front grid is mapped onto the rear top view to obtain the corresponding rear grid; the rear top view is obtained by mapping the rear point cloud data onto the preset plane;

[0016] For each preceding grid, the corresponding following grid is moved at least once. Based on the degree of difference between the pixels in the following adjusted grid and the corresponding pixels in the preceding grid, the target following adjusted grid for each preceding grid is determined.

[0017] By statistically analyzing the movement of each target in the subsequent grid adjustment, the background motion of the LiDAR relative to the detected target is obtained between the two sets of point cloud data.

[0018] In one embodiment, obtaining a first target motion relative to the detected target by the lidar between the two sets of point cloud data by performing image matching on the preceding point cloud data and the following point cloud data includes:

[0019] The point cloud data obtained by merging the front point cloud data and the rear point cloud data is mapped onto a preset plane to obtain a merged top view;

[0020] Clustering is performed on all pixels in the merged top view to obtain a target pixel set, and the minimum bounding box of all pixels in the target pixel set is determined.

[0021] The minimum bounding box is mapped to the front top view and the rear top view respectively to obtain the corresponding front bounding region and the rear bounding region; the front top view and the rear top view are obtained by mapping the front point cloud data and the rear point cloud data to a preset plane respectively.

[0022] Either the region in the front outer region or the region in the rear outer region is designated as the immovable first region, and the remaining region is designated as the movable second region.

[0023] The second region is moved at least once. Based on the degree of difference between the pixels in the second adjusted region obtained after the movement and the pixels in the first region, the corresponding target second adjusted region of the first region is determined. Based on the degree of movement of the target second adjusted region, the first target motion of the lidar relative to the detected target between the two sets of point cloud data is determined.

[0024] In one embodiment, obtaining the second target motion relative to the detected target by the lidar between the two sets of point cloud data by performing point cloud boundary detection on the preceding point cloud data and the following point cloud data includes:

[0025] Determine at least one row of front data points in the front point cloud data and at least one row of back data points in the back point cloud data;

[0026] Based on the degree of positional difference between the boundary points of the preceding data row and the boundary points of the following data row, the second target motion of the lidar relative to the detected target between the two sets of point cloud data is obtained.

[0027] In one embodiment, the first target motion includes a first lateral target motion and a first longitudinal target motion, and the second target motion includes a second lateral target motion and a second longitudinal target motion; the step of compensating the subsequent point cloud data based on the background motion, the first target motion, and the second target motion includes:

[0028] Determine whether the length of the first side and the length of the second side of the minimum bounding box are greater than a first preset length. If at least one of the first side and the second side is greater than the first preset length, then the target type of the corresponding detection target of the minimum bounding box is determined as a background target. The first side and the second side are the length and width of the minimum bounding box on a plane parallel to the preset plane, respectively.

[0029] If the lengths of the first side and the second side are both not greater than the first preset length, then obtain the first motion difference and the second motion difference between the first vertical target motion and the background motion, respectively.

[0030] If both the first motion difference and the second motion difference are less than the first motion threshold, and both the first lateral target motion and the second lateral target motion are less than the second motion threshold, then the target type of the corresponding detected target of the minimum bounding box is determined as a background target.

[0031] The target type of the remaining detected targets is determined to be a moving target.

[0032] In one embodiment, the method further includes:

[0033] For the moving target, it is determined whether the length of the third side of the minimum bounding box is greater than the second preset length. If it is greater, then point cloud expansion is performed on the corresponding front point cloud data and the corresponding back point cloud data along the direction of the third side length until the length of the third side of the expanded minimum bounding box is equal to the third preset length. The third side length is the side length of the minimum bounding box that is perpendicular to the preset plane.

[0034] In one embodiment, the compensation of the subsequent point cloud data based on the background motion, the first target motion, and the second target motion includes:

[0035] For the background target and the moving target after point cloud expansion, longitudinal compensation is performed on the subsequent point cloud data based on the degree of difference between each two of the background motion amount, the first vertical target motion amount and the second vertical target motion amount.

[0036] For the moving target after point cloud expansion, lateral compensation is performed on the subsequent point cloud data based on the degree of difference between the first lateral target motion and the second lateral target motion.

[0037] In one embodiment, the step of performing longitudinal compensation on the subsequent point cloud data for the background target and the moving target after point cloud expansion, based on the degree of difference between each two of the background motion amount, the first longitudinal target motion amount, and the second longitudinal target motion amount, includes:

[0038] If the difference between the first longitudinal target motion and the second longitudinal target motion is less than a preset longitudinal difference, the second longitudinal target motion is determined as the longitudinal target motion.

[0039] If the difference in longitudinal motion is not less than the preset longitudinal difference, the minimum difference in motion between the first difference in motion and the second difference in motion is obtained, and the minimum value between the longitudinal motion corresponding to the minimum difference in motion and the background motion is determined as the longitudinal target motion.

[0040] Based on the longitudinal target motion and the time difference between the two sets of point cloud data, the longitudinal velocity of the lidar relative to the detected target between the two sets of point cloud data is obtained.

[0041] Based on the longitudinal velocity and the transmission time interval between different data points in the subsequent point cloud data, longitudinal compensation is performed on the subsequent point cloud data.

[0042] In one embodiment, the lateral compensation of the subsequent point cloud data for the moving target after point cloud expansion, based on the degree of difference between the first lateral target motion and the second lateral target motion, includes:

[0043] If the difference in lateral motion between the first lateral target motion and the second lateral target motion is less than a preset lateral difference, the second lateral target motion is determined as the lateral target motion.

[0044] If the difference in lateral movement is not less than the preset lateral difference, the minimum value among the first lateral target movement and the second lateral target movement shall be determined as the lateral target movement.

[0045] Based on the lateral target motion and the time difference between the two sets of point cloud data, the lateral velocity of the lidar relative to the detected target between the two sets of point cloud data is obtained.

[0046] Lateral compensation is performed on the subsequent point cloud data based on the lateral velocity and the transmission time interval between different data points in the subsequent point cloud data.

[0047] In one embodiment, the method further includes:

[0048] For the second set of remaining point cloud data in the two sets of remaining point cloud data, determine the background point cloud data that is not in the preset non-background range in the second set of remaining point cloud data;

[0049] Based on the background motion data, longitudinal compensation is performed on the background point cloud data.

[0050] Secondly, this application also provides a point cloud data processing apparatus, the apparatus comprising:

[0051] The acquisition module is used to acquire the first and second set of point cloud data based on two sets of point cloud data.

[0052] The image matching module is used to obtain the background motion and first target motion of the lidar relative to the detected target between the two sets of point cloud data by performing image matching between the foreground point cloud data and the rear point cloud data.

[0053] The boundary detection module is used to obtain the second target motion of the lidar relative to the detected target between the two sets of point cloud data by performing point cloud boundary detection on the front point cloud data and the rear point cloud data.

[0054] The compensation module is used to compensate the back point cloud data based on the background motion, the first target motion, and the second target motion.

[0055] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method in any of the above embodiments.

[0056] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the methods in any of the above embodiments.

[0057] The aforementioned point cloud data processing method, apparatus, computer equipment, and storage medium obtain the background motion and first target motion of the lidar relative to the detected target between the two sets of point cloud data by performing image matching on the preceding and subsequent point cloud data. Simultaneously, by performing point cloud boundary detection on the preceding and subsequent point cloud data, the second target motion of the lidar relative to the detected target between the two sets of point cloud data is obtained. Finally, based on the background motion, the first target motion, and the second target motion, the subsequent point cloud data is compensated. After compensating the point cloud data, the motion distortion of the point cloud can be effectively corrected. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is an application environment diagram of a point cloud data processing method in one embodiment;

[0060] Figure 2This is a flowchart illustrating a point cloud data processing method in one embodiment;

[0061] Figure 3 This is a flowchart illustrating a method for determining front and back point cloud data in one embodiment.

[0062] Figure 4 This is a schematic diagram of a front top view and a rear top view in one embodiment;

[0063] Figure 5 A histogram of the movement position and the number of grid cells adjusted after the target in one embodiment;

[0064] Figure 6 This is a top view of the first region and the second region in one embodiment;

[0065] Figure 7 In one embodiment, the data point rows are shown in the pre-merge view and the top-merge view.

[0066] Figure 8 Here is a boundary difference histogram in one embodiment;

[0067] Figure 9 Here is a flowchart of the longitudinal compensation process in another embodiment;

[0068] Figure 10 Here is a flowchart of the lateral compensation process in another embodiment;

[0069] Figure 11 This is a structural block diagram of a point cloud data processing device in one embodiment;

[0070] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0072] The point cloud data processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the lidar 102 scans surrounding objects by emitting lasers to determine the point cloud of the surrounding objects. The vehicle-mounted controller 104 acquires multi-frame point cloud data of each surrounding object scanned by the lidar 102, and calculates the corresponding background motion, first target motion, and second target motion based on this point cloud data. Finally, it compensates for the point cloud data of the corresponding surrounding objects based on these three motion values. It should be noted that the point cloud data processing method can also be performed by the lidar itself, as long as its computing power meets the requirements.

[0073] In one exemplary embodiment, such as Figure 2 As shown, a point cloud data processing method is provided, which can be applied to... Figure 1 Taking the vehicle-side controller as an example, the explanation includes the following steps 202 to 208. Wherein:

[0074] S202. Based on two sets of point cloud data, obtain the front point cloud data and the back point cloud data.

[0075] Among them, the two sets of point cloud data are two frames of point cloud data of the detected target obtained after scanning by the same lidar. These two frames of point cloud data can be two adjacent frames of point cloud data or two non-adjacent frames of point cloud data. This application embodiment does not make a specific limitation on this. The first point cloud data and the second point cloud data are point cloud data obtained after preprocessing the two sets of point cloud data.

[0076] S204. By performing image matching on the front point cloud data and the rear point cloud data, the background motion and the first target motion of the lidar relative to the detected target are obtained between the two sets of point cloud data.

[0077] The detection target can be any object other than the ground scanned by the lidar. The process of obtaining the background motion is based on the assumption that the vehicle only moves longitudinally during driving and has no or only a small amount of lateral movement. Therefore, image matching is only performed in the longitudinal direction. Before obtaining the motion of the first target, the point cloud data of different detection targets can be distinguished based on two sets of point cloud data. Then, image matching is performed for each detection target. For example, clustering methods can be used to obtain the minimum bounding box of each detection target, thereby distinguishing the detection targets.

[0078] S206. By performing point cloud boundary detection on the front point cloud data and the rear point cloud data, the second target motion of the lidar relative to the detected target is obtained between the two sets of point cloud data.

[0079] Point cloud boundary detection refers to obtaining the row identifier of the data point row where each point is located in two sets of point cloud data, determining at least one pair of corresponding data point rows in the two sets of point cloud data based on the row identifier, determining the boundary difference of each pair of corresponding data point rows, and determining the motion of the second target based on the boundary difference.

[0080] S208. Based on the background motion, the first target motion, and the second target motion, compensate the subsequent point cloud data.

[0081] Before compensating the back point cloud data, the detected targets are first classified. Based on the classification results, different compensation methods are used. For example, the detected targets can be divided into background targets and moving targets. For background targets, only vertical compensation is required, while for moving targets, both horizontal and vertical compensation are required. In other embodiments, other classification methods can also be used to classify the detected targets. This application does not specifically limit this method.

[0082] In the above point cloud data processing method, image matching is performed on the preceding and following point cloud data to obtain the background motion and first target motion of the LiDAR relative to the detected target between the two sets of point cloud data. At the same time, point cloud boundary detection is performed on the preceding and following point cloud data to obtain the second target motion of the LiDAR relative to the detected target between the two sets of point cloud data. Finally, based on the background motion, the first target motion, and the second target motion, the following point cloud data is compensated. After compensating the point cloud data, the motion distortion of the point cloud can be effectively corrected.

[0083] In some embodiments, such as Figure 3 As shown, based on two sets of point cloud data, the preceding point cloud data and the following point cloud data are obtained, including:

[0084] S302. Delete the corresponding point cloud data of all ground points in the two sets of point cloud data to obtain the corresponding remaining point cloud data of each of the two sets of point cloud data.

[0085] S304. Among the two sets of remaining point cloud data, the point cloud data located within the preset non-background range and collected earlier is determined as the earlier point cloud data, and the point cloud data collected later is determined as the later point cloud data.

[0086] Since the detection target is any object other than the ground scanned by the lidar, the corresponding point cloud data of the ground point is considered noise in this embodiment. Before subsequent processing, the corresponding point cloud data of the ground point needs to be deleted. The remaining point cloud data located within the preset non-background range is determined as non-background point cloud data, and the remaining point cloud data not located within the preset non-background range is determined as background point cloud data. For non-background point cloud data, the corresponding background motion, the first target motion, and the second target motion are used to compensate the corresponding subsequent point cloud data. For background point cloud data, the corresponding background motion is directly used to compensate the corresponding subsequent point cloud data. In one embodiment, the preset non-background range can be represented in the lidar point cloud coordinate system as x∈(-30m, 30m), y∈(0, 60m), z∈(-∞, 0.7m).

[0087] In this embodiment, by deleting the corresponding point cloud data of the ground points, it is possible to avoid interference from the corresponding point cloud data of the ground points in the subsequent motion calculation process; by using different methods to compensate for non-background point cloud data and background point cloud data, the compensation of point cloud data is made more accurate.

[0088] In some embodiments, obtaining the background motion of the LiDAR relative to the detected target between the two sets of point cloud data by performing image matching on the foreground point cloud data and the background point cloud data includes: dividing the foreground top view into grids, and selecting the foreground grids from all the grids obtained; the foreground top view is obtained by mapping the foreground point cloud data onto a preset plane, the preset plane being a plane formed based on the horizontal and vertical directions, the vertical direction being the movement direction of the LiDAR parallel to the ground, and the horizontal and vertical directions being perpendicular to and parallel to the ground; mapping the foreground grids onto the rear top view to obtain the corresponding rear grids; the rear top view is obtained by mapping the rear point cloud data onto the preset plane; moving the corresponding rear grid of each foreground grid at least once, and determining the target rear adjustment grid corresponding to each foreground grid based on the degree of difference between the pixels in the rear adjustment grid obtained after the movement and the pixels in the corresponding foreground grid; and obtaining the background motion of the LiDAR relative to the detected target between the two sets of point cloud data by statistically analyzing the degree of movement of each target rear adjustment grid.

[0089] Specifically, such as Figure 4 As shown, the front point cloud data and the rear point cloud data are mapped onto a preset plane to obtain a front top view and a rear top view, respectively. Figure 4 (a) is the front top view. Figure 4 (b) is the rear top view. For example... Figure 4As shown in (a), multiple initial grids are obtained by dividing the front top view into grids, the number of point clouds included in each initial grid is obtained, and the initial grid with the corresponding number of point clouds greater than the first point cloud number threshold is determined as the front grid.

[0090] For each front grid, a movement range is determined along the longitudinal direction in the rear top view. The corresponding rear grid is controlled to move at least once within this movement range. The movement range may cover the original position of the rear grid's center. The movement distance can be measured by the number of pixels passed through during the movement. The distance of each movement may be the same or different; this embodiment does not specifically limit this. The number of point clouds included in each pixel of the front grid is obtained. Pixels with a point cloud number greater than a second point cloud number threshold are determined as target pixels. The sum of the point cloud numbers S_sum included in all target pixels is calculated, and the target pixels are sorted in descending order of their point cloud numbers. The number of point clouds included in each pixel of each rear adjustment grid is obtained. The rear adjustment grid includes the rear grid before the movement. For each subsequent adjustment grid, according to the sorting of target pixels, calculate the absolute value |Δs| of the difference between the number of point clouds included in each target pixel and the number of point clouds included in the corresponding pixel in the subsequent adjustment grid. After each calculation, |Δs| is accumulated, and the accumulated value is denoted as S_l, and the corresponding accumulation count is denoted as count. The accumulation process ends when S_l is greater than a preset accumulation value, and the accumulation count at this time is recorded as the target accumulation count. For example, the preset accumulation value can be S_sum*0.6. After determining the target accumulation count for all subsequent adjustment grids, the subsequent adjustment grid with the largest target accumulation count is determined as the target subsequent adjustment grid of the corresponding preceding grid.

[0091] like Figure 5 As shown, Figure 5 The horizontal axis represents the movement position, and the vertical axis represents the number of target adjustment grids corresponding to each movement position. After determining the target adjustment grids corresponding to all preceding grids, the number of target adjustment grids corresponding to each movement position within the movement position range is counted. The movement position with the highest number of adjustment grids is determined as the first target movement position. The first target movement position and the two movement positions adjacent to the first target movement position are determined as the second target movement positions. The weight of each second target movement position is determined based on the number of target adjustment grids corresponding to each second target movement position. The product of the movement distance and the corresponding weight for each second target movement position is calculated. The sum of the products of all second target movement positions is determined as the background motion.

[0092] In this embodiment, the target's corresponding rear adjustment grid is determined based on the degree of difference between the pixels in the rear adjustment grid and the corresponding pixels in the front grid, making the background motion determined based on the degree of movement of the target in the rear adjustment grid more accurate.

[0093] In some embodiments, by performing image matching on the foreground point cloud data and the background point cloud data, the first target motion of the LiDAR relative to the detected target between the two sets of point cloud data is obtained, including: mapping the point cloud data obtained by merging the foreground and background point cloud data onto a preset plane to obtain a merged top view; performing clustering processing on all pixels in the merged top view to obtain a target pixel set, and determining the minimum bounding box of all pixels in the target pixel set; mapping the minimum bounding box onto the foreground top view and the background top view respectively to obtain the corresponding foreground bounding region and background bounding region; The front top view and the rear top view are obtained by mapping the front point cloud data and the rear point cloud data onto a preset plane, respectively. Either the front outer region or the rear outer region is designated as a first immovable region, and the remaining region is designated as a second movable region. The second region is moved at least once. Based on the degree of difference between the pixels in the second adjustment region obtained after the movement and the pixels in the first region, the corresponding target second adjustment region of the first region is determined. Based on the corresponding degree of movement of the target second adjustment region, the first target motion of the lidar relative to the detected target between the two sets of point cloud data is determined.

[0094] Specifically, for pixels with non-zero values ​​in the merged top view, select one pixel as the current seed point and add it to the initially empty seed point set. The pixel value represents the number of points in the corresponding pixel cloud. Based on the current seed point, determine a region. Add any non-zero pixels in the region that have not been selected to the seed point set, and perform deduplication on the seed point set. Select any unselected seed points from the seed point set and use them as the new current seed point. Return to the step of determining a region based on the current seed point and continue execution until all seed points in the seed point set have been selected, obtaining the final seed point set. Use this final seed point set as the target pixel set.

[0095] like Figure 6 As shown, Figure 6 (a) and Figure 6 (b) are two top views, Figure 6 The rectangle in (a) represents the first region. Figure 6(b) The rectangle represents the second region. The second region is moved along a preset direction in the corresponding top view. This movement range represents the movable position of the region's center point and covers its original position. The preset direction includes horizontal and vertical directions. For each movable position within the movement range, the center point of the second region is moved to that position, resulting in the corresponding adjusted second region. For each adjusted second region, the pixel values ​​of each pixel in the first region are sorted according to a preset sorting method to obtain a sorting result. A first pixel is selected in the first region according to the sorting result. For each selected first pixel, a second pixel in the adjusted second region whose relative position matches the first pixel is determined, and the pixel value difference between the first and second pixels is obtained. The pixel value differences of the selected first pixels are accumulated, and the process of selecting the first pixel in the first region according to the sorting result is returned and continued until the accumulated sum exceeds a preset accumulation threshold, obtaining the corresponding accumulation count for the adjusted second region. From all adjusted second regions, the adjusted second region with the maximum accumulation count is determined as the target adjusted second region for the first region.

[0096] For the second adjustment area of ​​the target obtained by moving laterally, based on the movable position corresponding to the second adjustment area of ​​the target, the first lateral target motion of the lidar relative to the detected target between two frames of point cloud data is obtained; for the second adjustment area of ​​the target obtained by moving longitudinally, based on the movable position corresponding to the second adjustment area of ​​the target, the first longitudinal target motion of the lidar relative to the detected target between two frames of point cloud data is obtained.

[0097] In this embodiment, the minimum bounding box is determined by clustering, so that the determined minimum bounding box is more closely matched with the size of the corresponding detection target; the target second adjustment region corresponding to the first region is determined according to the degree of difference between the pixels in the second adjustment region and the pixels in the first region, so that the motion amount of the first target determined based on the corresponding motion degree of the target second adjustment region is more accurate.

[0098] In some embodiments, by performing point cloud boundary detection on the foreground and background point cloud data, a second target motion amount between the two sets of point cloud data relative to the detected target by the lidar is obtained, including: determining at least one foreground data point row in the foreground point cloud data and at least one background data point row in the background point cloud data; and obtaining the second target motion amount between the two sets of point cloud data relative to the detected target by the lidar based on the degree of positional difference between the boundary points of the foreground data point row and the boundary points of the background data row.

[0099] Specifically, the relative positions between transmitting units in the lidar's transmitting array are compared with the relative positions between data points in the preceding point cloud data and the relative positions between data points in the following point cloud data to obtain at least one row of data points in the preceding point cloud data and at least one row of data points in the following point cloud data. The difference between the coordinates of the boundary points of the preceding and following data point rows is obtained as the boundary difference. The variance and average of multiple boundary differences are obtained, and the range of boundary differences is determined based on the variance and average. Based on the boundary differences that fall within the range of the boundary differences, the second target motion of the lidar relative to the detected target between the two frames of point cloud data is determined.

[0100] Specifically, the second target motion includes the second lateral target motion and the second longitudinal target motion. For example... Figure 7 As shown, Figure 7 (a) is the front view of the merged point cloud data sets. Δx in the figure represents the difference in the lateral boundary. Figure 7 (b) is a top view of the merged point cloud data, where Δy represents the longitudinal boundary difference. Calculate the mean ξ1 and variance δ1 among all lateral boundary differences, and determine the lateral boundary differences within the range [ξ1-δ1, ξ1+δ1] as the target lateral boundary difference. Calculate the average of all target lateral boundary differences, and determine this average as the second lateral target motion. Calculate the mean ξ2 and variance δ2 among all longitudinal boundary differences, and determine the longitudinal boundary differences within the range [ξ2-δ2, ξ2+δ2] as the target longitudinal boundary difference. Calculate the average of all target longitudinal boundary differences, and determine this average as the second longitudinal target motion. This can be based on... Figure 8 The histogram in the graph calculates the mean ξ1, variance δ1, mean ξ2, and variance δ2. The horizontal axis in the graph represents the corresponding boundary difference.

[0101] In this embodiment, the second target motion is obtained based on the positional difference between the boundary points of the preceding data point row and the boundary points of the following data point row. This can effectively improve the detection accuracy of the target motion of the corresponding detection target, making the subsequent compensation process more accurate.

[0102] In some embodiments, the first target motion quantity includes a first horizontal target motion quantity and a first vertical target motion quantity, and the second target motion quantity includes a second horizontal target motion quantity and a second vertical target motion quantity. Based on the background motion quantity, the first target motion quantity, and the second target motion quantity, before compensating the subsequent point cloud data, the process includes: determining whether the length of the first side length and the length of the second side length of the minimum bounding box are greater than a first preset length; if at least one of the first side length and the second side length is greater than the first preset length, then the target type of the detected target corresponding to the minimum bounding box is determined as a background target, where the first side length and the second side length are the length and width on a plane parallel to the preset plane, respectively; if neither the length of the first side length nor the length of the second side length is greater than the first preset length, then the first motion quantity difference and the second motion quantity difference between the first vertical target motion quantity and the second vertical target motion quantity and the background motion quantity are obtained; if both the first motion quantity difference and the second motion quantity difference are less than a first motion quantity threshold, and both the first horizontal target motion quantity and the second horizontal target motion quantity are less than a second motion quantity threshold, then the target type of the detected target corresponding to the minimum bounding box is determined as a background target; and the target type of the remaining detected targets is determined as a moving target.

[0103] Among them, the background target can be considered as a relatively stationary target. For the background target, only the corresponding back point cloud data needs to be compensated vertically, while the moving target needs to be compensated horizontally and vertically at the same time.

[0104] In this embodiment, the background target and the moving target are distinguished based on the size of the minimum bounding box, the motion of the first target, the motion of the second target, and the motion of the background, making the distinction more accurate and thus making the subsequent compensation process more accurate.

[0105] In some embodiments, the above method further includes: for the moving target, determining whether the length of the third side of the minimum bounding box is greater than the second preset length; if it is greater, then expanding the corresponding front point cloud data and the corresponding rear point cloud data along the direction of the third side length until the length of the third side of the expanded minimum bounding box is equal to the third preset length, wherein the third side length is the side length of the minimum bounding box perpendicular to the preset plane.

[0106] Point cloud expansion involves acquiring point cloud data of the corresponding detection target within a cuboid composed of a first side length, a second side length, and a preset line segment. The preset line segment refers to the line segment difference between the line segment corresponding to the second preset length and the line segment corresponding to the third preset length. In one embodiment, the second preset length is 0.5m and the third preset length is 2.5m.

[0107] In this embodiment, by expanding the point cloud, the data points of the corresponding detection target can be obtained as much as possible, so that the point cloud data of the detection target can be compensated as much as possible in the future, thereby eliminating motion distortion to the greatest extent.

[0108] In some embodiments, compensation is performed on the subsequent point cloud data based on the background motion, the first target motion, and the second target motion, including: for the background target and the moving target after point cloud expansion, longitudinal compensation is performed on the subsequent point cloud data based on the degree of difference between each two of the background motion, the first vertical target motion, and the second vertical target motion; for the moving target after point cloud expansion, lateral compensation is performed on the subsequent point cloud data based on the degree of difference between the first horizontal target motion and the second horizontal target motion.

[0109] Among them, the corresponding compensation speed can be obtained based on the degree of difference, and the corresponding compensation can be performed based on the compensation speed and the transmission time interval between different data points in the post-point cloud data.

[0110] In this embodiment, longitudinal compensation is performed by comprehensively considering the background motion, the first longitudinal target motion, and the second longitudinal target motion, and lateral compensation is performed by comprehensively considering the first lateral target motion and the second lateral target motion, so that the final compensation result is more accurate.

[0111] In some embodiments, for a background target and a moving target after point cloud expansion, longitudinal compensation is performed on the subsequent point cloud data based on the degree of difference between each two of the background motion amount, the first longitudinal target motion amount, and the second longitudinal target motion amount. This includes: if the difference in longitudinal motion amount between the first and second longitudinal target motion amounts is less than a preset longitudinal difference, the second longitudinal target motion amount is determined as the longitudinal target motion amount; if the difference in longitudinal motion amount is not less than the preset longitudinal difference, the minimum motion amount difference between the first and second motion amount differences is obtained, and the minimum value between the longitudinal motion amount corresponding to the minimum motion amount difference and the background motion amount is determined as the longitudinal target motion amount; based on the longitudinal target motion amount and the time difference between the two sets of point cloud data, the longitudinal velocity of the lidar relative to the detected target between the two sets of point cloud data is obtained; and longitudinal compensation is performed on the subsequent point cloud data based on the longitudinal velocity and the transmission time interval between different data points in the subsequent point cloud data.

[0112] Among them, the minimum value between the vertical motion amount and the background motion amount is the minimum value of the corresponding absolute value; since the boundary detection process is more accurate than the image matching process, the second vertical target motion amount is determined as the vertical target motion amount when the difference in vertical motion amount is less than the preset vertical difference value.

[0113] In this embodiment, when the difference in longitudinal motion is less than a preset longitudinal difference, the second longitudinal target motion is determined as the longitudinal target motion, making the determined longitudinal target motion more accurate; based on the longitudinal target motion and the time difference between the two sets of point cloud data, the corresponding longitudinal velocity is obtained, making the subsequent compensation process based on the longitudinal velocity more accurate.

[0114] In some embodiments, for a moving target after point cloud expansion, lateral compensation is performed on subsequent point cloud data based on the degree of difference between the first lateral target motion and the second lateral target motion, including: if the difference in lateral motion between the first and second lateral target motions is less than a preset lateral difference, the second lateral target motion is determined as the lateral target motion; if the difference in lateral motion is not less than the preset lateral difference, the minimum value between the first and second lateral target motions is determined as the lateral target motion; based on the lateral target motion and the time difference between the two sets of point cloud data, the lateral velocity of the lidar relative to the detected target between the two sets of point cloud data is obtained; and lateral compensation is performed on subsequent point cloud data based on the lateral velocity and the transmission time interval between different data points in the subsequent point cloud data.

[0115] Among them, the minimum of the first lateral target motion amount and the second lateral target motion amount is the minimum of the corresponding absolute values; since the boundary detection process is more accurate than the image matching process, the second lateral target motion amount is determined as the lateral target motion amount when the difference in lateral motion amount is less than the preset lateral difference.

[0116] In this embodiment, when the difference in lateral motion is less than a preset lateral difference, the second lateral target motion is determined as the lateral target motion, making the determined lateral target motion more accurate; based on the lateral target motion and the time difference between the two sets of point cloud data, the corresponding lateral velocity is obtained, making the subsequent compensation process based on the lateral velocity more accurate.

[0117] In some embodiments, the method further includes: for the latter set of remaining point cloud data in the two sets of remaining point cloud data, determining the background point cloud data that is not in a preset non-background range in the latter set of remaining point cloud data; and performing longitudinal compensation on the background point cloud data based on the background motion amount.

[0118] Specifically, based on the background motion and the time difference between the two sets of point cloud data, the longitudinal velocity of the lidar relative to the detected target between the two sets of point cloud data is obtained; based on the longitudinal velocity and the transmission time interval between different data points in the background point cloud data, longitudinal compensation is performed on the background point cloud data.

[0119] In this embodiment, after compensating the subsequent point cloud data located within the preset non-background range, further compensation is provided for the background point cloud data not located within the preset non-background range, making the compensation for the remaining point cloud data of the next set more comprehensive.

[0120] In one embodiment, another point cloud data processing method is provided, which includes the following:

[0121] (1) Image matching detection of background motion

[0122] First, the point cloud within the range of x∈(-30m, 30m), y∈(0, 60m), z∈(-∞, 0.7m) is extracted and divided into the k-th frame point cloud and the k+1-th frame point cloud, which are mapped to a 600×600 top view, denoted as the k-th frame top view and the k+1-th frame top view, respectively. The k-th frame top view is then divided into multiple grid windows.

[0123] Then, a window in the k-th frame top view is selected and slid in the (k+1)-th frame top view. The sliding range is [-15, 15] pixels vertically. During the sliding process, SSDA calculation is performed for each slide, and the cumulative count H of the sub-image corresponding to the window is recorded. After the sliding ends, the sub-image with the highest cumulative count is selected as the matching image, and the sliding distance corresponding to this sub-image is the motion amount of the point cloud under that window. Here, it is assumed that the vehicle only moves vertically during the driving process, and there is no or only a small lateral movement, which does not affect the matching of the vertical image. Therefore, sliding matching is only performed in the vertical direction. Setting the sliding range of [-15, 15] pixels can detect targets with a relative speed within ±144 km / h.

[0124] Finally, a histogram is formed by statistically analyzing the motion of all windows in the top view of the k-th frame. The motion with the largest number of windows and its adjacent motions are weighted and summed, and the result is used as the background motion.

[0125] (2) Clustering

[0126] For each point in the point cloud of frame k and frame (k+1), label them k and k+1 respectively. Merge the two point clouds and extract points within the ranges x∈(-30m, 30m), y∈(0, 60m), z∈(-∞, 0.7m) to map them into a top view. Record the correspondence between the point cloud and pixels. Use the region growing method to cluster the top view. Region growing is a commonly used image segmentation algorithm. First, select a pixel with a non-zero value as the starting seed point of a cluster and mark its state as queried. Then, query its neighborhood (taking a 48-neighborhood as an example, with a window size of 7×7 and a query radius of 3) to see if there are any pixels with a non-zero value that have not been queried. Mark the pixels that meet the condition as belonging to the same cluster and add them to the seed point queue. Repeat the same query steps until the number of seed points in the queue is zero, at which point the clustering of one point cluster is complete. Then, find the next unqueried pixel with a non-zero value as the starting seed point of a new cluster and repeat the above query steps. Clustering ends when all pixels have been queried. Finally, points are assigned to their corresponding clusters based on the correspondence between points and pixels, and the boundaries and center points of each cluster are calculated to obtain the minimum bounding box of the target.

[0127] (3) Obtain the target motion quantity 1 based on image matching detection

[0128] First, based on the clustering results, the target point clusters and their minimum bounding boxes are known. Points in the k-th frame and the (k+1)-th frame are distinguished from the target point clusters based on point markers, and projected as top-down views of the point cloud in the k-th and (k+1)-th frames, respectively. Then, using the length, width, and center point of the target's minimum bounding box as a window, the top-down view in the k-th frame is cropped as a template. Horizontal and vertical sliding is performed at the corresponding positions in the (k+1)-th frame's top-down view. During the sliding process, an image matching algorithm is used to detect sub-images that match the template. The relative displacement (horizontal and vertical displacement) between the matched sub-image and the original window position is recorded as a set of motion quantities for the target, denoted as target motion quantity 1. The motion quantities of each target are calculated using the same operation.

[0129] (4) The target motion quantity 2 is calculated based on the point cloud boundary.

[0130] First, based on the clustering results, the target point clusters are known. From these clusters, points in frame k and frame (k+1) are distinguished by their point labels, and the line sequence of each point is determined by its point ID. Then, two lines with the same line sequence are selected, and their minimum and maximum difference in horizontal boundary values ​​(collectively referred to as horizontal boundary differences) and minimum difference in vertical boundary values ​​(referred to as vertical boundary differences) are calculated. Multiple sets of lines with the same line sequence may exist for the same target, resulting in multiple sets of horizontal and vertical boundary differences. These are statistically analyzed to obtain horizontal and vertical boundary difference histograms. The mean ξ and variance δ of the histograms are calculated. Values ​​within the range [ξ-δ, ξ+δ] of the histogram are selected, and the mean within that range is calculated to obtain the mean difference in horizontal and vertical boundary values. These are treated as a set of target motion quantities, denoted as target motion quantity 2.

[0131] (5) Perform point cloud compensation by combining background motion volume, target motion volume 1, and target motion volume 2.

[0132] The compensation process includes vertical compensation and horizontal compensation. Vertical compensation includes, for example, vertical compensation... Figure 9 As shown, the longitudinal compensation process is as follows:

[0133] Determine whether the length of the first side and the second side of the minimum bounding box are greater than a preset length. If at least one of the first and second side lengths is greater than the preset length, then the target type of the corresponding detected target of the minimum bounding box is determined as a background target. The first and second side lengths are the length and width of the minimum bounding box on a plane parallel to the preset plane, respectively. If neither the first nor the second side length is greater than the preset length, then obtain the first motion difference and the second motion difference between the vertical motion of target motion 1 and the vertical motion of target motion 2 and the background motion, respectively. If both the first and second motion difference are less than the first motion threshold, and both the lateral motion of target motion 1 and the lateral motion of target motion 2 are less than the second motion threshold, then the target type of the corresponding detected target of the minimum bounding box is determined as a background target. The target type of the remaining detected targets is determined as a dynamic target. Among them, according to the national standard for the outer contour dimensions of automobiles, trailers and automobile trains, the length of the vehicle shall not exceed 20m, and a margin of 5m is generally reserved. Therefore, the preset length can be 25m. The preset plane is a plane formed by the horizontal and vertical directions. The vertical direction is the direction of movement of the lidar parallel to the ground, and the horizontal direction is perpendicular to the vertical direction and parallel to the ground. The first motion threshold and the second motion threshold can both be 6cm.

[0134] Both target motion detected by image matching (1) and target motion detected by boundary (2) have their drawbacks. Boundary detection has higher accuracy but is prone to errors. Image matching is relatively stable but occasionally produces errors, and its lower resolution leads to lower accuracy in motion detection. Therefore, a selection strategy is set according to the principles of prioritizing accuracy and minimizing compensation, selecting the optimal value for compensation. The strategy settings are as follows:

[0135] Secondly, for dynamic targets, when the difference between the boundary detection motion and the image detection motion is less than 6cm, the boundary detection motion is considered accurate, and the boundary detection motion with higher accuracy is selected as the compensation value for correction. When the difference between the boundary detection motion and the image detection motion is greater than 6cm, it is considered that there may be an erroneous value between the two, and in this case, the motion value that is closer to the background motion is selected for compensation and correction.

[0136] Then, for the background target, if the difference between the boundary detection motion and the image detection motion is less than 6cm, the edge detection motion is selected for compensation. If the difference is greater than 6cm, the motion that is closer to the background motion is selected and compared with the background motion, and the motion with the smallest absolute value is selected for compensation.

[0137] Finally, point clouds outside the clustering range, i.e., those outside the range of x∈(-30m, 30m), y∈(0, 60m), and z∈(-∞, 0.7m), are considered background and directly compensated and corrected according to the background velocity. Targets identified as dynamic in the clustering results within x∈(-30m, 30m), y∈(0, 60m), and z∈(-∞, 0.7m) are expanded upwards to 2.5m, and the expanded points are corrected according to the motion of the corresponding dynamic target. The expansion method is as follows: targets marked as dynamic and whose maximum z-axis bounding value exceeds 0.5m participate in the expansion. The point clouds outside the clustering range are traversed to determine whether they are within the x and y bounding boundaries of the dynamic target; if so, they are assigned to the corresponding target cluster.

[0138] Lateral compensation such as Figure 10 As shown, the process of lateral compensation is as follows:

[0139] The overall lateral compensation amount does not need to take into account the background exercise amount. It only needs to compare whether the difference between the two sets of exercise amounts is less than 6cm. If it is less, the boundary value is selected; if it is greater, the minimum absolute value of the two is selected for compensation.

[0140] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0141] Based on the same inventive concept, this application also provides a point cloud data processing apparatus for implementing the point cloud data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more point cloud data processing apparatus embodiments provided below can be found in the limitations of the point cloud data processing method described above, and will not be repeated here.

[0142] In one exemplary embodiment, such as Figure 11 As shown, a point cloud data processing device 1100 is provided, including: an acquisition module 1101, an image matching module 1102, a boundary detection module 1103, and a compensation module 1104, wherein:

[0143] The acquisition module 1101 is used to acquire the front point cloud data and the back point cloud data based on two sets of point cloud data.

[0144] The image matching module 1102 is used to obtain the background motion and first target motion of the lidar relative to the detected target between the two sets of point cloud data by performing image matching between the front point cloud data and the rear point cloud data.

[0145] The boundary detection module 1103 is used to obtain the second target motion of the lidar relative to the detected target between the two sets of point cloud data by performing point cloud boundary detection on the front point cloud data and the rear point cloud data.

[0146] The compensation module 1104 is used to compensate the back point cloud data based on the background motion amount, the first target motion amount, and the second target motion amount.

[0147] In some embodiments, the acquisition module 1101 is further configured to delete the corresponding point cloud data of all ground points in the two sets of point cloud data to obtain the corresponding remaining point cloud data of each of the two sets of point cloud data; and to determine the point cloud data located in the preset non-background range and collected earlier in the two sets of remaining point cloud data as the preceding point cloud data, and the point cloud data collected later as the following point cloud data.

[0148] In some embodiments, the image matching module 1102 is further configured to perform grid division on the forward top view, and filter out the front grid from all the grids obtained; the forward top view is obtained by mapping the front point cloud data onto a preset plane, the preset plane being a plane formed based on the horizontal and vertical directions, the vertical direction being the movement direction of the lidar parallel to the ground, and the horizontal direction being perpendicular to the vertical direction and parallel to the ground; the front grid is mapped onto the rear top view to obtain the corresponding rear grid; the rear top view is obtained by mapping the rear point cloud data onto the preset plane; each front grid is moved at least once to the corresponding rear grid, and the target rear adjustment grid corresponding to each front grid is determined based on the degree of difference between the pixels in the rear adjustment grid obtained after the movement and the pixels in the corresponding front grid; by statistically analyzing the degree of movement of each target rear adjustment grid, the background motion of the lidar relative to the detected target between the two sets of point cloud data is obtained.

[0149] In some embodiments, the image matching module 1102 is further configured to map the point cloud data obtained by merging the foreground point cloud data and the background point cloud data onto a preset plane to obtain a merged top view; perform clustering processing on all pixels in the merged top view to obtain a target pixel set, and determine the minimum bounding box of all pixels in the target pixel set; map the minimum bounding box onto the foreground top view and the background top view respectively to obtain corresponding foreground and background bounding regions; the foreground top view and the background top view are obtained by merging the foreground point cloud data and the background point cloud data. The point cloud data is mapped onto a preset plane. Either the front outer region or the rear outer region is designated as a first immovable region, and the remaining region is designated as a second movable region. The second region is moved at least once. Based on the difference between the pixels in the second adjusted region and the pixels in the first region, a corresponding target second adjusted region is determined for the first region. Based on the degree of movement of the target second adjusted region, the first target motion of the lidar relative to the detected target between the two sets of point cloud data is determined.

[0150] In some embodiments, the boundary detection module 1103 is further configured to determine at least one front data point row in the front point cloud data and at least one rear data point row in the rear point cloud data; and based on the degree of positional difference between the boundary points of the front data point row and the boundary points of the rear data point row, to obtain the second target motion of the lidar relative to the detected target between the two sets of point cloud data.

[0151] In some embodiments, the first target motion quantity includes a first lateral target motion quantity and a first longitudinal target motion quantity, and the second target motion quantity includes a second lateral target motion quantity and a second longitudinal target motion quantity; the point cloud data processing device 1100 is specifically used to determine whether the length of the first side length and the length of the second side length of the minimum bounding box are greater than a first preset length; if at least one of the first side length and the second side length is greater than the first preset length, then the target type of the corresponding detected target of the minimum bounding box is determined as a background target, wherein the first side length and the second side length are respectively the length and width of the minimum bounding box on a plane parallel to a preset plane; if the length of the first side length and the length of the second side length are both not greater than the first preset length, then the first motion quantity difference and the second motion quantity difference between the first longitudinal target motion quantity and the second longitudinal target motion quantity and the background motion quantity are obtained; if the first motion quantity difference and the second motion quantity difference are both less than a first motion quantity threshold, and the first lateral target motion quantity and the second lateral target motion quantity are both less than a second motion quantity threshold, then the target type of the corresponding detected target of the minimum bounding box is determined as a background target; and the target type of the remaining detected targets is determined as a moving target.

[0152] In some embodiments, the point cloud data processing device 1100 is further configured to determine, for the moving target, whether the length of the third side of the minimum bounding box is greater than the second preset length. If it is greater, then point cloud expansion is performed on the corresponding front point cloud data and the corresponding rear point cloud data along the direction of the third side length until the length of the third side of the expanded minimum bounding box is equal to the third preset length, wherein the third side length is the side length of the minimum bounding box perpendicular to the preset plane.

[0153] In some embodiments, the compensation module 1104 includes:

[0154] The longitudinal compensation unit is used to perform longitudinal compensation on the subsequent point cloud data based on the degree of difference between each two of the background target, the first longitudinal target's motion, and the second longitudinal target's motion, for the background target and the moving target after point cloud expansion.

[0155] The lateral compensation unit is used to perform lateral compensation on the subsequent point cloud data for the moving target after point cloud expansion, based on the degree of difference between the first lateral target motion amount and the second lateral target motion amount.

[0156] In some embodiments, the longitudinal compensation unit is further configured to: determine the second longitudinal target motion as the longitudinal target motion when the difference between the first and second longitudinal target motions is less than a preset longitudinal difference; obtain the minimum motion difference between the first and second motion differences when the difference is not less than the preset longitudinal difference, and determine the minimum value between the longitudinal motion corresponding to the minimum motion difference and the background motion as the longitudinal target motion; obtain the longitudinal velocity of the lidar relative to the detected target between the two sets of point cloud data based on the longitudinal target motion and the time difference between the two sets of point cloud data; and perform longitudinal compensation on the subsequent point cloud data based on the longitudinal velocity and the transmission time interval between different data points in the subsequent point cloud data.

[0157] In some embodiments, the lateral compensation unit is further configured to: determine the second lateral target motion as the lateral target motion if the difference between the first and second lateral target motions is less than a preset lateral difference; determine the minimum value among the first and second lateral target motions as the lateral target motion if the difference is not less than the preset lateral difference; obtain the lateral velocity of the lidar relative to the detected target between the two sets of point cloud data based on the lateral target motion and the time difference between the two sets of point cloud data; and perform lateral compensation on the subsequent point cloud data based on the lateral velocity and the transmission time interval between different data points in the subsequent point cloud data.

[0158] In some embodiments, the point cloud data processing device 1100 is further configured to, for the latter set of remaining point cloud data in two sets of remaining point cloud data, determine the background point cloud data that is not in the preset non-background range in the latter set of remaining point cloud data; and perform longitudinal compensation on the background point cloud data based on the background motion amount.

[0159] Each module in the aforementioned point cloud data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0160] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a point cloud data processing method.

[0161] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0162] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0163] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0165] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A point cloud data processing method, characterized in that, The method includes: Based on two sets of point cloud data, obtain the point cloud data in the foreground and the point cloud data in the background. By performing image matching on the front point cloud data and the rear point cloud data, the background motion and first target motion of the lidar relative to the detected target are obtained between the two sets of point cloud data. By performing point cloud boundary detection on the front point cloud data and the rear point cloud data, the second target motion of the lidar relative to the detected target between the two sets of point cloud data is obtained. The background motion, the first target motion, and the second target motion are used to compensate the subsequent point cloud data.

2. The method according to claim 1, characterized in that, The process of acquiring the preceding and following point cloud data based on two sets of point cloud data includes: Delete the corresponding point cloud data of all ground points in the two sets of point cloud data to obtain the corresponding remaining point cloud data of each of the two sets of point cloud data. The point cloud data that is located within the preset non-background range and was collected earlier in the two sets of remaining point cloud data is determined as the earlier point cloud data, and the point cloud data collected later is determined as the later point cloud data.

3. The method according to claim 2, characterized in that, By performing image matching on the foreground point cloud data and the background point cloud data, the background motion of the lidar relative to the detected target between the two sets of point cloud data is obtained, including: The front view is divided into grids, and the front grids are selected from all the grids obtained. The front view is obtained by mapping the front point cloud data onto a preset plane. The preset plane is a plane formed based on the horizontal and vertical directions. The vertical direction is the movement direction of the lidar parallel to the ground, and the horizontal direction is perpendicular to the vertical direction and parallel to the ground. The front grid is mapped onto the rear top view to obtain the corresponding rear grid; the rear top view is obtained by mapping the rear point cloud data onto the preset plane; For each preceding grid, the corresponding following grid is moved at least once. Based on the degree of difference between the pixels in the following adjusted grid and the corresponding pixels in the preceding grid, the target following adjusted grid for each preceding grid is determined. By statistically analyzing the movement of each target in the subsequent grid adjustment, the background motion of the LiDAR relative to the detected target is obtained between the two sets of point cloud data.

4. The method according to claim 2, characterized in that, The step of obtaining the first target motion relative to the detected target by the lidar between the two sets of point cloud data by performing image matching on the preceding point cloud data and the following point cloud data includes: The point cloud data obtained by merging the front point cloud data and the rear point cloud data is mapped onto a preset plane to obtain a merged top view; Clustering is performed on all pixels in the merged top view to obtain a target pixel set, and the minimum bounding box of all pixels in the target pixel set is determined. The minimum bounding box is mapped to the front top view and the rear top view respectively to obtain the corresponding front bounding region and the rear bounding region; the front top view and the rear top view are obtained by mapping the front point cloud data and the rear point cloud data to a preset plane respectively. Either the region in the front outer region or the region in the rear outer region is designated as the immovable first region, and the remaining region is designated as the movable second region. The second region is moved at least once. Based on the degree of difference between the pixels in the second adjusted region obtained after the movement and the pixels in the first region, the corresponding target second adjusted region of the first region is determined. Based on the degree of movement of the target second adjusted region, the first target motion of the lidar relative to the detected target between the two sets of point cloud data is determined.

5. The method according to claim 2, characterized in that, The step of obtaining the second target motion relative to the detected target by the lidar between the two sets of point cloud data by performing point cloud boundary detection on the foreground point cloud data and the background point cloud data includes: Determine at least one row of front data points in the front point cloud data and at least one row of back data points in the back point cloud data; Based on the degree of positional difference between the boundary points of the preceding data row and the boundary points of the following data row, the second target motion of the lidar relative to the detected target between the two sets of point cloud data is obtained.

6. The method according to claim 1, characterized in that, The first target motion includes a first lateral target motion and a first longitudinal target motion, and the second target motion includes a second lateral target motion and a second longitudinal target motion; the step of compensating the subsequent point cloud data based on the background motion, the first target motion, and the second target motion includes: Determine whether the length of the first side and the length of the second side of the minimum bounding box are greater than a first preset length. If at least one of the first side and the second side is greater than the first preset length, then the target type of the corresponding detection target of the minimum bounding box is determined as a background target. The first side and the second side are the length and width of the minimum bounding box on a plane parallel to the preset plane, respectively. If the lengths of the first side and the second side are both not greater than the first preset length, then obtain the first motion difference and the second motion difference between the first vertical target motion and the background motion, respectively. If both the first motion difference and the second motion difference are less than the first motion threshold, and both the first lateral target motion and the second lateral target motion are less than the second motion threshold, then the target type of the corresponding detected target of the minimum bounding box is determined as a background target. The target type of the remaining detected targets is determined to be a moving target.

7. The method according to claim 6, characterized in that, The method further includes: For the moving target, it is determined whether the length of the third side of the minimum bounding box is greater than the second preset length. If it is greater, then point cloud expansion is performed on the corresponding front point cloud data and the corresponding rear point cloud data along the direction of the third side length until the length of the third side of the expanded minimum bounding box is equal to the third preset length. The third side length is the side length of the minimum bounding box that is perpendicular to the preset plane.

8. The method according to claim 7, characterized in that, The compensation of the subsequent point cloud data based on the background motion, the first target motion, and the second target motion includes: For the background target and the moving target after point cloud expansion, longitudinal compensation is performed on the subsequent point cloud data based on the degree of difference between each two of the background motion amount, the first vertical target motion amount and the second vertical target motion amount. For the moving target after point cloud expansion, lateral compensation is performed on the subsequent point cloud data based on the degree of difference between the first lateral target motion amount and the second lateral target motion amount.

9. The method according to claim 8, characterized in that, The step of performing longitudinal compensation on the subsequent point cloud data for the background target and the moving target after point cloud expansion, based on the degree of difference between every two items of the background motion, the first longitudinal target motion, and the second longitudinal target motion, includes: If the difference between the first longitudinal target motion and the second longitudinal target motion is less than a preset longitudinal difference, the second longitudinal target motion is determined as the longitudinal target motion. If the difference in longitudinal motion is not less than the preset longitudinal difference, the minimum difference in motion between the first difference in motion and the second difference in motion is obtained, and the minimum value between the longitudinal motion corresponding to the minimum difference in motion and the background motion is determined as the longitudinal target motion. Based on the longitudinal target motion and the time difference between the two sets of point cloud data, the longitudinal velocity of the lidar relative to the detected target between the two sets of point cloud data is obtained. Based on the longitudinal velocity and the transmission time interval between different data points in the subsequent point cloud data, longitudinal compensation is performed on the subsequent point cloud data.

10. The method according to claim 8, characterized in that, For the moving target after point cloud expansion, based on the degree of difference between the first lateral target motion and the second lateral target motion, lateral compensation is performed on the subsequent point cloud data, including: If the difference in lateral motion between the first lateral target motion and the second lateral target motion is less than a preset lateral difference, the second lateral target motion is determined as the lateral target motion. If the difference in lateral movement is not less than the preset lateral difference, the minimum value among the first lateral target movement and the second lateral target movement shall be determined as the lateral target movement. Based on the lateral target motion and the time difference between the two sets of point cloud data, the lateral velocity of the lidar relative to the detected target between the two sets of point cloud data is obtained. Lateral compensation is performed on the subsequent point cloud data based on the lateral velocity and the transmission time interval between different data points in the subsequent point cloud data.

11. The method according to claim 2, characterized in that, The method further includes: For the second set of remaining point cloud data in the two sets of remaining point cloud data, determine the background point cloud data that is not in the preset non-background range in the second set of remaining point cloud data; Based on the background motion data, longitudinal compensation is performed on the background point cloud data.

12. A point cloud data processing device, characterized in that, The device includes: The acquisition module is used to acquire the first and second set of point cloud data based on two sets of point cloud data. The image matching module is used to obtain the background motion and first target motion of the lidar relative to the detected target between the two sets of point cloud data by performing image matching between the foreground point cloud data and the rear point cloud data. The boundary detection module is used to obtain the second target motion of the lidar relative to the detected target between the two sets of point cloud data by performing point cloud boundary detection on the front point cloud data and the rear point cloud data. The compensation module is used to compensate the back point cloud data based on the background motion, the first target motion, and the second target motion.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images