Laser radar data processing method and system

Abstract

The invention discloses a laser radar data processing method and system. The method comprises the steps that at least one local data area is determined from single-frame point cloud data of a target scene, the target scene is scanned in sequence through at least one set of scanning lines of a laser radar according to a preset revisit time interval from at least part of different angles to obtain the single-frame point cloud data, the ratio of the view field range of each local data area to the view field range of the single-frame point cloud data is smaller than a preset local threshold value, and the acquisition time interval of at least two pieces of local point cloud data in each local data area is not smaller than the preset revisit time interval; and based on at least two pieces of local point cloud data in each local data area, generating a fitting point cloud at different angles with the local point cloud data, and / or obtaining the speed of a target object corresponding to the local area, and / or performing motion correction or clustering on the single-frame point cloud data.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of laser radar technology, and in particular to a method and system for processing laser radar data. Background Art

[0002] LiDAR (Light Detection and Ranging) is a radar system that detects targets by emitting a laser beam and receiving the reflected echo. LiDAR uses laser beam scanning to generate point cloud data within a certain range. Because each point in the point cloud contains three-dimensional coordinate information, LiDAR has been widely used in fields such as robotic navigation and autonomous driving. Summary of the Invention

[0003] According to a first aspect of the present application, a method for processing lidar data is provided, comprising:

[0004] Determining at least one local data region from a single frame of point cloud data of a target scene, wherein the single frame of point cloud data is obtained by sequentially scanning the target scene at at least partially different angles using at least one set of scan lines of a laser radar according to a preset revisit time interval, a ratio of a field of view range of each local data region to a field of view range of the single frame of point cloud data is less than a preset local threshold, and a time interval between acquisitions of at least two local point cloud data in each of the local data regions is no less than the preset revisit time interval; and

[0005] Based on at least two of the local point cloud data in each of the local data areas, a fitting point cloud at a different angle from the local point cloud data is generated, and / or the speed of the target object corresponding to the local data area is obtained, and / or motion correction or clustering is performed on the single-frame point cloud data.

[0006] According to a second aspect of the present application, a laser radar system is provided, comprising:

[0007] at least one processor; and,

[0008] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect.

[0009] According to a third aspect of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the method described in the first aspect is implemented.

[0010] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method described in the first aspect.

[0011] According to the laser radar data processing method and laser radar system of the present application, regular point cloud data can be generated by fitting the point cloud, reducing the number of points in the point cloud data obtained by direct measurement, reducing the energy of the laser emitter, generating new point cloud data or correcting the measured point cloud data, and realizing accurate fitting of the scanning trajectory, which can not only improve the accuracy of the scanning trajectory fitting, but also effectively restore the geometric shape of the object.

[0012] The contents described in this section are not intended to identify the key or important features of the embodiments of the present application, nor are they intended to limit the scope of the present application. Other features of the present application will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present application. Among them:

[0014] Figure 1 is a flowchart of a method for processing lidar data according to an embodiment of the present application;

[0015] Figure 2 Schematic diagram of the principle of the method for processing lidar data according to an embodiment of the present application;

[0016] Figure 3 is a schematic diagram of a scan line of a single frame of point cloud data according to an embodiment of the present application;

[0017] Figure 4 1 is a flow chart of a method for processing lidar data to generate a fitted point cloud according to an embodiment of the present application;

[0018] Figure 5 1 is a flow chart of a method for processing lidar data according to an embodiment of the present application for performing feature fusion on a fitted point cloud;

[0019] Figure 6 It is a schematic diagram of the composition structure of a laser radar data processing system suitable for implementing the implementation method of the present application. DETAILED DESCRIPTION

[0020] In the description of the embodiments of this application, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, the order in which the steps are described does not necessarily indicate the order in which they are performed, unless the context indicates that the steps are to be performed sequentially.

[0021] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] The embodiment of the present application provides a method for processing laser radar data. Figure 1 As shown, the laser radar data processing method 1000 may include the following steps:

[0024] S100, determining at least one local data area from a single-frame point cloud data of a target scene, wherein the single-frame point cloud data is obtained by sequentially scanning the target scene at at least partially different angles according to a preset revisit time interval through at least one group of scanning lines of a laser radar, the ratio of the field of view range of each local data area to the field of view range of the single-frame point cloud data is less than a preset local threshold, and the time interval for obtaining at least two local point cloud data in each local data area is not less than the preset revisit time interval.

[0025] S200, based on at least two local point cloud data in each local data area, generating a fitting point cloud at a different angle from the local point cloud data, and / or obtaining the speed of the target object corresponding to the local data area, and / or performing motion correction or clustering on the single frame point cloud data.

[0026] The laser radar data processing method of the embodiment of the present application can be executed by the laser radar system. Figure 2As shown, the laser radar system can obtain point cloud data of the target scene by scanning the target scene through the transceiver scanning module 400 according to the preset scanning rules. The single-frame point cloud data 410 in the point cloud data can be obtained by scanning the target scene at at least partially different angles at different times by at least one group of scanning lines of the transceiver scanning module 400. The time interval and angle of the target scene scanned by at least one group of scanning lines can be pre-set according to the preset scanning rules. The laser radar system can process the single-frame point cloud data in the obtained point cloud data: determine at least one local data area 411 in the single-frame point cloud data 410, and determine at least two local point cloud data P1 and P2 in each local data area 411, so that the ratio of the field of view range of each local data area 411 to the field of view range of the single-frame point cloud data is less than the preset local threshold, and the time interval ΔT between the two local point cloud data P1 and P2 is not less than the revisit time of the target scene scanned at at least partially different angles by at least one group of scanning lines pre-set in the preset scanning rules. interval; then, a fitting point cloud P3 with a different angle from the local point cloud data P1 and P2 can be generated based on at least two local point cloud data P1 and P2 in each local data area 411, and / or the speed of the local point cloud data P1 and P2, that is, the speed of the target object corresponding to the local data area 411, can be obtained based on at least two local point cloud data P1 and P2 in each local data area 411, and / or motion correction or clustering can be performed on at least part of the point cloud data in the single frame point cloud data 410 based on at least two local point cloud data P1 and P2 in each local data area 411.

[0027] As can be seen from the above, the embodiments of the present application create a time difference in the scanning time of scanning lines scanned at different angles in a single-frame point cloud data, and determine at least one local data area with a smaller range in the single-frame point cloud data, and obtain at least two local point cloud data with a larger time interval in each local area. It is possible to generate a fitting point cloud at a different angle from the local point cloud data for at least two local point cloud data in each local data area, and / or obtain the speed of the target object corresponding to the local data area, and / or perform motion correction or clustering on the single-frame point cloud data, so that regular point cloud data can be generated by fitting the point cloud, reducing the number of points in the point cloud data obtained by direct measurement, reducing the energy of the laser emitter, generating new point cloud data or correcting the measured point cloud data, and achieving accurate fitting of the scanning trajectory, which can not only improve the accuracy of the scanning trajectory fitting, but also effectively restore the geometric shape of the object.

[0028] It should be noted that the embodiments of the present application do not limit the composition structure of the laser radar and the method of scanning the target scene. For example, the laser radar may include a transmitting module, a receiving module, a scanning module and a control module, wherein the transmitting module may include a laser and an optical system, the receiving module may include an optical system and a detector, the scanning module may include a motor and a micro-resonant mirror MEMS that changes the projection direction of the laser beam, and the control module may control the transmitting module, the receiving module and the scanning module. For example, the scanning line formed by the laser beam of the laser radar can scan the target scene horizontally and vertically according to the spatial distribution of the horizontal and / or vertical angles to obtain a single frame of point cloud data of the target scene.

[0029] It should be noted that the embodiments of the present application do not limit the value of the preset revisit time interval. Generally, when the frame rate of the laser radar is determined, the preset revisit time interval can be determined based on the number of scan line groups of a single frame of point cloud data. Optionally, the ratio of the preset revisit time interval to the scan time of a single frame of point cloud data can include one of 1 / 3, 1 / 4, and 1 / 5. For example, if the frame rate of the laser radar is 10 Hz, the laser radar obtains a frame of point cloud data in 100 milliseconds, and the number of scan line groups of a single frame of point cloud data is 5, the preset revisit time interval can be 20 milliseconds.

[0030] It should be noted that the embodiments of the present application do not limit the value of the preset local threshold for determining the local data area in a single frame of point cloud data. Generally, the value of the preset local threshold can be pre-set to a smaller value, or it can also be determined based on the field of view of the lidar, that is, the horizontal field of view angle FOVv and the vertical field of view angle FOVh. For example, a local multiple n greater than 1 can be preset, and the preset local threshold can be the minimum value of FOVv / n and FOVh / n. Optionally, the preset local threshold can include one of 1 / 10, 1 / 100, 1 / 1000, and 1 / 10000.

[0031] The following is a detailed introduction to the various steps of the method for processing lidar data in the implementation manner of the present application.

[0032] Step S100

[0033] In step S100, at least one local data area is determined from the single-frame point cloud data of the target scene, wherein the single-frame point cloud data is obtained by sequentially scanning the target scene at at least partially different angles according to a preset revisit time interval through at least one group of scanning lines of the laser radar, the ratio of the field of view range of each local data area to the field of view range of the single-frame point cloud data is less than a preset local threshold, and the time interval for obtaining at least two local point cloud data in each local data area is not less than the preset revisit time interval.

[0034] In some optional embodiments, a laser radar (LiDAR) can be used to scan a target scene to obtain point cloud data of the target scene. The multiple laser beams emitted by the LiDAR can form at least one set of scan lines. The at least one set of scan lines formed by the multiple laser beams sequentially forms multiple scan line groups in the vertical or horizontal direction at at least partially different angles according to a preset revisit time interval, scanning the target scene horizontally and vertically to obtain a single frame of point cloud data of the target scene. The scanning angles of the scan lines in the multiple scan line groups are at least partially different, and the scan lines are arranged in a cyclical sequence in the vertical or horizontal direction according to the group number of each scan line group.

[0035] In some optional examples, a laser radar emits multiple laser beams to scan a target scene. The multiple laser beams form a group of scan lines. The scan lines formed by the multiple laser beams sequentially form N scan line groups in the vertical direction at at least partially different angles at a preset revisit time interval to vertically scan the target scene. Each scan line in the N scan line groups scans the target scene horizontally to obtain single-frame point cloud data of the target scene. The scanning angles of the N scan line groups are at least partially different, and the scan lines are arranged in a cyclical manner in the vertical direction according to the group numbers of the N scan line groups, where N is an integer greater than 2 and less than 100.

[0036] In other optional examples, a laser radar emits multiple laser beams to scan a target scene. The multiple laser beams form a group of scan lines. The scan lines formed by the multiple laser beams sequentially form M scan line groups in the horizontal direction at at least partially different angles at a preset revisit time interval to vertically scan the target scene. Each scan line in the M scan line groups vertically scans the target scene to obtain single-frame point cloud data of the target scene. The scan lines in the M scan line groups have at least partially different scanning angles, and the scan lines are arranged in a cyclical manner in the horizontal direction according to the group numbers of the M scan line groups, where M is an integer greater than 2 and less than 100.

[0037] For example, the multiple laser beams emitted by the laser radar form 24 scan lines, and the 24 scan lines are grouped as one. A group of scan lines of the laser radar forms 5 scan line groups in the vertical direction at different angles according to the preset revisit time interval. The 5 scan line groups can achieve 120-line scanning of the target scene and obtain a single frame of point cloud data of the target scene. Among them, the 120 scan lines are arranged in a cyclical manner in the vertical direction according to the group number of the scan line group, such as Figure 3As shown, the vertical sequence of the 120 scan lines from top to bottom is 12345, 12345, 12345, ..., 12345. Among them, 24 scan lines with the same group number are scanned simultaneously, and the five scan line groups are scanned sequentially according to the group number, completing the scanning of the current frame, and then restarting and entering the scanning of the next frame. That is, the 24 scan lines with group number 1 are scanned first. After completing the scanning after a ΔTm period, the 24 scan lines with group number 2 are scanned. After completing the scanning after a ΔTm period, the 24 scan lines with group number 3 are scanned. After completing the scanning after a ΔTm period, the 24 scan lines with group number 4 are scanned. After completing the scanning after a ΔTm period, the 24 scan lines with group number 5 are scanned. ΔTm is the preset revisit time interval between the scan lines in two scan line groups with increasing sequence numbers within each frame period. For example, ΔTm can be 20 milliseconds.

[0038] In some optional embodiments, step S100 may include: performing superpixel segmentation on a two-dimensional image projected from a certain viewing angle of a three-dimensional single-frame point cloud data to obtain a plurality of superpixel regions, and determining a local data region based on the ratio of the field of view range being less than a preset local threshold based on the plurality of superpixel regions. Superpixels are small blocks between pixels and image objects that are divided into segments. Superpixels of ordinary images are independent of time, while superpixels of 3D point cloud data projected onto any 2D plane have a certain relationship with time. Therefore, the local data region is a superpixel that spans time and space and can be called a spatiotemporal superpixel. By performing 3D superpixel segmentation similar to 2D superpixel segmentation on single-frame point cloud data and selecting the superpixel region obtained by superpixel segmentation as the local region, the correspondence between the local region and the target object in the single-frame point cloud can be effectively guaranteed.

[0039] It should be noted that superpixels are only one implementation method of determining local data areas in the embodiments of the present application. The embodiments of the present application can also determine local data areas by other methods. The embodiments of the present application do not make specific limitations on this, and local data areas that meet the conditions selected by other methods can also be called spatiotemporal superpixels, such as a 3x3 area formed by 9 simple points in the horizontal and vertical directions in point cloud data, or an area formed by continuous points with a distance difference of less than 3 cm in point cloud data, etc.

[0040] The local data area can be output as a spatiotemporal superpixel unit to the outside of the lidar system, or as an embedded data unit to the AI / computing module inside the lidar system for recalculating the spatiotemporal superpixel to perform calculations or processing such as data unit selection, distance distribution, signal-to-noise ratio distribution, speed, motion correction, and clustering.

[0041] In other optional embodiments, step S100 may include: based on the recognition results of the other frame point cloud data, determining in each local data region of the current frame point cloud data at least two local point cloud data with a time interval no less than a preset revisit time interval, wherein the recognition results include one of the type of superpixel in the other frame point cloud data and the type of the local part of the target object in the other frame point cloud data. In some optional examples, the lidar system may identify the previous frame point cloud data of the target scene and determine in the local data region of the current frame point cloud data at least two local point cloud data with a time interval no less than a preset revisit time interval. For example, when the lidar is a vehicle-mounted lidar, the lidar system may identify the previous frame point cloud data of the target scene to obtain the positions of the left and right rear lights of the preceding vehicle in the previous frame point cloud data, and then, based on the positions of the left and right rear lights of the preceding vehicle in the previous frame point cloud data, determine in the local data region of the current frame point cloud data the local point cloud data of the vehicle with a time interval no less than a preset revisit time interval.

[0042] In yet other optional embodiments, step S200 may include: determining, based on a preset local point cloud position determination rule, in each local data region of the current frame point cloud data, at least two local point cloud data having a time interval no less than a preset revisit time interval. In some optional examples, a local point cloud position determination rule may be preset in the lidar system, and based on the preset local point cloud position determination rule, determining, in each local data region of the current frame point cloud data, at least two local point cloud data having a time interval no less than a preset revisit time interval. For example, the local point cloud position determination rule preset in the lidar system may be preset when the lidar is shipped from the factory.

[0043] In some further optional embodiments, step S200 may include: based on the location information of the local point cloud input by the user, determining, in each local data region of the current frame of point cloud data, at least two local point cloud data items with a time interval no less than a preset revisit time interval. In some optional examples, the user may identify a single frame of point cloud data of the target scene and input the location information of the local point cloud data into the lidar system. The lidar system may then determine, in each local data region of the current frame of point cloud data, at least two local point cloud data items with a time interval no less than a preset revisit time interval based on the location information of the local point cloud data input by the user. For example, when the lidar is a vehicle-mounted lidar, the user may identify a single frame of point cloud data of the target scene to obtain the location of the left and right rear lights of the vehicle ahead, and input the location of the left and right rear lights of the vehicle ahead into the lidar system. The lidar system may determine, in the local data region of the current frame of point cloud data, the local point cloud data items for the left and right rear lights of the vehicle ahead, with a time interval no less than a preset revisit time interval based on the location information of the left and right rear lights of the vehicle ahead.

[0044] For example, Figure 3 As shown, the time interval of at least two local point cloud data in the local area can be greater than the time interval 4*ΔTm between the scan line with group number 1 and the scan line with group number 5, wherein T0 is the moment in the scanning period of the scan line with group number 1, and T1 is the moment in the scanning period of the scan line with group number 5. Based on the spatial angle relationship of the scan lines in the vertical direction, including the distance and relative position between the scan lines, and the continuity of the scan lines in the spatial angle, a local point cloud data can be determined at time T0 in the single frame point cloud data, and another local point cloud data can be determined at time T1 to complete the determination of the two local point cloud data, and obtain the following: Figure 3 The local point cloud data marked in area A. By ensuring that the time interval between at least two local point cloud data in the local data area is greater than the time interval between the scan line with the smallest group number and the scan line with the largest group number in the multiple scan line groups, it can be effectively ensured that at least two local point cloud data have the maximum time interval in a single frame of point cloud data.

[0045] Step S200

[0046] In step S200, a fitting point cloud at different angles relative to the local point cloud data is generated based on at least two local point cloud data in each local data region, and / or the velocity of the target object corresponding to the local data region is obtained, and / or motion correction or clustering is performed on the single-frame point cloud data. After obtaining the local point cloud data, the lidar system can obtain a lidar scanning trajectory by performing trajectory fitting on the at least two local point cloud data in each local data region. The lidar system can generate a fitting point cloud at different angles relative to the local point cloud data based on the obtained scanning trajectory.

[0047] In some optional embodiments, such as Figure 4 As shown, step S200 may include the following steps: S210, based on the spatial positions and relative acquisition times of at least two local point cloud data in each local data region, determining the motion parameters of the corresponding local point cloud data, where the motion parameters include displacement and velocity. S220, performing trajectory fitting based on the motion parameters of each local region to obtain a scanning trajectory for the target object corresponding to the local data region. S230, based on the scanning trajectory, determining the spatial coordinates of the target object corresponding to the target moment within the time period corresponding to the scanning trajectory, and generating a fitted point cloud based on the spatial coordinates.

[0048] In step S210, the lidar system can determine the spatial position and relative time of at least two local point cloud data in each local data region based on the order of scan lines, the grouping of scan lines, the distance and relative position between scan lines, the time difference between scan lines, etc. in the single frame point cloud data, and determine the vertical and / or horizontal motion parameters of the corresponding local point cloud data based on the spatial position and relative time of the at least two local point cloud data. For example, the motion parameters of the local point cloud data may include displacement and velocity, etc., which are not limited in the embodiments of the present application. The velocity of the local point cloud data can be output as the velocity of the target object corresponding to the local data region.

[0049] Optionally, step S210 may include determining the spatial position distribution of the target object corresponding to each local data region between at least two local point cloud data sets based on the vertical or horizontal angular distances and relative positions between adjacent scan lines in the plurality of scan line groups; and determining the velocity distribution of the target object corresponding to each local data region between at least two local point cloud data sets based on the time difference and spatial position distribution of adjacent scan lines between at least two local point cloud data sets, thereby obtaining a velocity curve for the target object. The time difference between adjacent scan lines between at least two local point cloud data sets for each local data region may be recorded by a lidar system. Step S220 may include performing trajectory fitting based on the spatial position distribution and velocity distribution of each local region, thereby obtaining a scanning trajectory for the target object corresponding to the local data region. By using the changes in the time difference, the target object's velocity at different time points can be accurately determined, thereby obtaining a velocity curve for the target object. A more accurate fitting trajectory can be obtained based on the obtained spatial position distribution and velocity curve.

[0050] It should be noted that the embodiments of the present application do not limit the implementation method of performing scanning trajectory fitting based on at least two local point cloud data in each local data area. For example, the scanning trajectory fitting can be performed using an existing mathematical model.

[0051] In some optional examples, step S220 may include: performing trajectory fitting on the corresponding local point cloud data based on the motion parameters obtained for each local data region using a data association algorithm and a Kalman filter algorithm to obtain a scanning trajectory of the target object corresponding to the local data region. The target trajectory can be obtained by performing target tracking using the association algorithm and the Kalman filter algorithm in a time series based on the position and velocity of the local point cloud data. During data association, data at times T0 and T1 corresponding to two local point cloud data can be considered to belong to the same object if the distance between them is close. Therefore, global data matching can be omitted, and only the displacement caused by the time interval between T0 and T1 needs to be processed. During Kalman filtering, due to the influence of motion or noise in the point cloud data between times T0 and T1, the point cloud data will fluctuate. Kalman filtering is introduced to predict the coordinates and velocity of the object's position based on the observation sequence of the object's position, that is, to update the predicted value based on the measured value to achieve a more accurate estimate. For example, the measured value can be combined with the predicted value of the uniform motion mathematical model to determine the optimal estimate of the target position.

[0052] In other optional examples, step S220 may include: performing trajectory fitting on the corresponding local point cloud data based on the motion parameters obtained for each local data area through a Bayesian filtering algorithm to obtain a scanning trajectory of the target object corresponding to the local data area. The Bayesian filter can be initialized based on the data of the two local point clouds at time T0 and T1, and state variables are set, such as the position and velocity of the target point cloud data including at least two local point cloud data, and the corresponding covariance matrix. State prediction can then be performed based on the velocity information obtained by the trajectory fitting algorithm, and state prediction can be performed through Bayesian filtering based on the motion model and system noise. An updated target state estimate is obtained through the Bayesian filter, including information such as position and velocity, which more accurately reflects the actual motion state of the target object. After the state prediction is performed through the Bayesian filter, the actual measurement data of the lidar can be compared with the predicted value, and the updated and corrected state estimate through the Bayesian filter can be updated according to the uncertainty of the sensor measurement. The Bayesian filtering algorithm can effectively handle the uncertainty in the scanning trajectory and improve the robustness of the fitting. By comprehensively considering the uncertainty factors, it can show higher stability in the scanning process and provide a reliable basis for the geometric restoration of point cloud data.

[0053] In yet other optional examples, step S220 may include performing trajectory fitting on the corresponding local point cloud data based on the motion parameters obtained for each local data region using a time series analysis algorithm to obtain a scanning trajectory for the target object corresponding to the local data region. This can be accomplished by performing time series feature extraction, time series analysis, and time trend analysis on at least the current frame's point cloud data, and integrating the obtained information with the results of the trajectory fitting algorithm. First, by recording consecutive single frames and analyzing the differences between frames, local points with large fluctuations can be extracted and used to match the point cloud data of the same target between different frames. Then, through autocorrelation analysis, sliding window analysis, and other methods, the motion trajectory and shape features of the local point cloud data are supplemented and corrected to compensate for the lack of information within the frame. Subsequently, possible matching motion models are analyzed based on changes over a longer period of time, and information such as acceleration and deceleration over time, as well as changes in shape features, is statistically analyzed to enhance the stability of the local point cloud data. The information obtained from this analysis is then integrated with the results of the trajectory fitting algorithm, including adjusting fitting parameters, optimizing trajectory estimation, and increasing point cloud data detail. Time series analysis, through modeling and analysis, reveals trends, seasonality, cycles, and irregular fluctuations in data. It can include models such as moving average (MA), autoregressive (AR), and autoregressive moving average (ARMA). Time series analysis algorithms can improve trajectory fitting accuracy, better adapt to motion characteristics at different time points, and effectively address issues that traditional methods may encounter in dynamic scenarios.

[0054] Optionally, the lidar data processing method of the embodiment of the present application may further include clustering the single-frame point cloud data to obtain aggregated data corresponding to the target object in the single-frame point cloud data. In step S200, trajectory fitting may be performed on at least two local point cloud data in the local data region contained in the aggregated data, and a fitted point cloud may be generated based on the scanning trajectory obtained by trajectory fitting. By clustering the single-frame point cloud data to obtain aggregated data corresponding to the target object in the single-frame point cloud data, and performing trajectory fitting based on the aggregated data corresponding to the target object obtained by clustering, the correspondence between the local data region and the target object can be verified.

[0055] Optionally, after obtaining the aggregated data corresponding to the target object in the single-frame point cloud data through clustering, the calculation of the velocity of at least two local point cloud data can be converted into the calculation of the velocity of the target object's center of gravity at the time corresponding to the at least two local point cloud data by obtaining the center of gravity of the target object. Therefore, performing trajectory fitting on at least two local point cloud data in the local data area contained in the aggregated data, and generating a fitted point cloud based on the scanning trajectory obtained by the trajectory fitting, can include: determining the center of gravity of the target object based on the aggregated data; determining the spatial position of the center of gravity of the target object based on the time corresponding to the at least two local point cloud data in the aggregated data; and determining the velocity of the target object as the velocity of the corresponding local point cloud data based on the relative time of the at least two local point cloud data in the aggregated data and the spatial position of the center of gravity of the target object.

[0056] Optionally, after generating the fitted point cloud in step S200, the features of at least two local point cloud data may be analyzed, such as at least one of the geometric contour of the fitted surface, the scan line shape, the normal vector, the point cloud density, the surface curvature, the slope, and the color (or reflectivity). The relevant features may be associated or matched by analyzing the correlation of the various features, and the fitted point cloud may be fused based on the association or matching results. This may make the resulting fitted point cloud after feature fusion more consistent with the appearance of the actual target or make the point cloud features more continuous, i.e., the error with the target point cloud is smaller, thereby improving the accuracy and authenticity of the fitted point cloud. The correlations may include: the correlation between reflectivity and normal direction, the correlation between surface curvature and reflectivity consistency, the correlation between point cloud density and slope, etc.

[0057] Therefore, if Figure 5As shown, method 1000 may further include the following steps: S240, performing point cloud feature extraction on at least three local point cloud data in each local data region to obtain point cloud features of the local data region. S250, based on the correlation between various features in the point cloud features of the local data region, correlating or matching the correlated features in the point cloud features of each local data region. S260, based on the results of correlating or matching the point cloud features of each local data region, performing feature fusion on the corresponding fitting point cloud to obtain a fitting point cloud that matches the local data region.

[0058] Among them, interrelated features can increase the dimension of the point cloud, enrich the details and characteristics of the point cloud, and effectively supplement the fitted point cloud. For example, color is associated with the normal direction: regions with similar colors may have a certain consistency in the normal direction. For example, on the surface of a vehicle, the normal direction may be related to the change in the color of the vehicle body, and the color change between the front and the body may be accompanied by a change in the normal direction. Curvature is associated with color consistency: curvature may affect the reflection of light, and color may show different consistency between regions with different curvatures. For example, the color of leaves may remain consistent between regions with similar curvature, while color changes may occur where the curvature changes. Point cloud density is associated with slope: changes in slope may affect the point cloud density of the target. For example, the point cloud density of the opposite wall is uniform, while the point cloud density of the road surface changes significantly from near to far.

[0059] In some optional examples, step S260 performs feature fusion, which can adopt a weighted fusion approach. By assigning weights to different features and comprehensively considering the contribution of information such as color and density, accurate matching and local matching of local point cloud data can be achieved. Therefore, step S360 can also include: determining the weight of each type of feature based on the contribution of each type of feature in the point cloud features of each local data area; performing weighted fusion on the corresponding fitting point cloud based on the results of the association or matching of the point cloud features of each local data area, to obtain a fitting point cloud that matches the local data area. Among them, accurate matching generally refers to a one-to-one matching of the feature points of two matching point cloud data, and the two matching point cloud data have similar shapes / curvatures / normal vector distributions. Local matching generally refers to matching only local feature points, and the matching parts of the two point cloud data have similar shapes / curvatures / normal vector distributions. For example, since each scan line of the vehicle body has similar right-angle characteristics, only the misaligned right angles in the vehicle body point cloud are corrected to be aligned. For another example, due to occlusion, the target object in the two frames of point cloud data has different local losses, so only the point cloud data of the target object in the two frames of point cloud data are matched to combine into more complete point cloud data.

[0060] In some optional embodiments, step S200 may include: searching for ground point cloud data that includes a local data area and contains a local minimum height, and performing three-dimensional ground surface fitting of the ground point cloud data; above a preset height threshold of the fitted three-dimensional ground surface, searching for point cloud data that has a distance less than a preset clustering threshold from any point cloud data in another local data area, and classifying it as the same cluster data as the other local data area; and repeatedly performing the search for point cloud data that has a distance less than a preset clustering threshold from any point cloud data in another local data area, and classifying it as the same cluster data as the other local data area, above a preset height threshold of the fitted three-dimensional ground surface, to obtain cluster data of the point cloud data of another local data area, as the result output of clustering the single-frame point cloud data based on at least two local point cloud data in each local data area.

[0061] In some optional embodiments, method 1000 may further include: scanning a target scene using at least one laser radar to obtain single-frame point cloud data; wherein at least one set of scan lines of each laser radar sequentially scans at least a portion of the target scene at at least partially different angles according to a preset revisit time interval to obtain at least a portion of the single-frame point cloud data. Each laser radar scans at least a portion of a field of view of a single-frame point cloud of the target scene.

[0062] For example, a LiDAR system might include three LiDARs. The first LiDAR's field of view primarily covers the left side of the target scene, the second LiDAR's field of view primarily covers the front of the target scene, and the third LiDAR's field of view primarily covers the right side of the target scene. When these three LiDARs scan synchronously, the point cloud data obtained by these three LiDARs is connected within a single frame to produce a single frame of point cloud data for the target scene.

[0063] Optionally, the lidar data processing method of the embodiments of the present application can also determine the ranging capability and ranging reliability of the lidar at each scanning angle based on the distance and intensity information of the lidar's echo signal, so as to determine the contamination level of the lidar window at the corresponding angle based on the ranging capability and ranging reliability. In some optional examples, the contamination level of the lidar window at each scanning angle can be determined based on the distance and intensity information of at least three non-primary lidar echo signals. This allows accurate judgment of the location and level of contamination on the lidar window through echo signals.

[0064] Optionally, the data processing method for the lidar according to the embodiments of the present application can also receive stray light signals reflected by objects within the lidar; based on the vertical or horizontal differences in the stray light signals, the vibration direction, phase, and amplitude of the device within the lidar used for scanning in the corresponding direction are determined. The device used for scanning in the corresponding direction includes a MEMS or prism that does not use a code disk or angle feedback. The lidar can achieve vertical or horizontal scanning using MEMS.

[0065] The embodiment of the present application also provides a system for processing laser radar data. Figure 6 , is a block diagram of a system for processing lidar data according to an embodiment of the present application. The system includes: one or more processors 501, a memory 502, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, each device providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 501 is taken as an example.

[0066] Memory 502 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor, causing the at least one processor to perform the lidar data processing method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the lidar data processing method provided in this application.

[0067] Memory 502, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the lidar data processing method in the embodiments of the present application. Processor 501 executes the non-transitory software programs, instructions, and modules stored in memory 502 to execute various server functional applications and data processing, thereby implementing the lidar data processing method in the above-mentioned method embodiment.

[0068] Memory 502 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function; the data storage area may store data generated based on the use of the blockchain-based information processing electronic device. Furthermore, memory 502 may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state memory device. In some embodiments, memory 502 may optionally include memory remotely located from processor 501. Such remote memory may be connected to the blockchain-based information processing electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0069] The laser radar data processing system may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0070] The input device 503 can receive input digital or character information and generate key signal input related to user settings and function control of blockchain-based information processing electronic devices, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, an indicator stick, one or more mouse buttons, a trackball, a joystick, and other input devices. The output device 504 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0071] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0072] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0073] Optionally, the laser radar data processing system may further include: at least one laser radar, each of the laser radars including a transceiver scanning module; wherein the transceiver scanning module may include a laser emitting component, a laser receiving component, and a laser scanning component;

[0074] The transceiver scanning module of the at least one laser radar is configured to scan the target scene to obtain the single-frame point cloud data;

[0075] Among them, the transceiver scanning module of each laser radar is configured to scan at least part of the target scene in sequence at at least some different angles according to the preset revisit time interval through at least one group of scanning lines to obtain at least part of the single-frame point cloud data.

[0076] An embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the method for processing lidar data as described in the embodiment of the present application.

[0077] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the lidar data processing method as described in the embodiment of the present application.

[0078] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0079] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0080] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0081] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0082] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for processing laser radar data, characterized in that: include: Determining at least one local data region from a single frame of point cloud data of a target scene, wherein the single frame of point cloud data is obtained by sequentially scanning the target scene at at least partially different angles using at least one set of scan lines of a laser radar according to a preset revisit time interval, a ratio of a field of view range of each local data region to a field of view range of the single frame of point cloud data is less than a preset local threshold, and a time interval between acquisitions of at least two local point cloud data in each of the local data regions is no less than the preset revisit time interval; and Based on at least two of the local point cloud data in each of the local data areas, a fitting point cloud at a different angle from the local point cloud data is generated, and / or the speed of the target object corresponding to the local data area is obtained, and / or motion correction or clustering is performed on the single-frame point cloud data.

2. The method according to claim 1, characterized in that The ratio of the preset revisit time interval to the scanning time of the single frame point cloud data includes one of 1 / 3, 1 / 4, and 1 / 5; The preset local threshold includes one of 1 / 10, 1 / 100, 1 / 1000, and 1 / 10000.

3. The method according to claim 1, characterized in that The determining of at least one local data region from a single frame of point cloud data of the target scene includes: Superpixel segmentation is performed on the two-dimensional image of the three-dimensional single-frame point cloud data projected at a certain perspective to obtain multiple superpixel areas, and based on the multiple superpixel areas, the local data area is determined according to the ratio of the field of view range being less than the preset local threshold.

4. The method according to any one of claims 1 to 3, characterized in that Based on the recognition results of other frame point cloud data, at least two of the local point cloud data with an acquisition time interval not less than the preset revisit time interval are determined in each of the local data areas of the current frame point cloud data, wherein the recognition results include one of the types of superpixels in the other frame point cloud data and the types of local parts of the target object in the other frame point cloud data.

5. The method according to any one of claims 1 to 3, characterized in that Based on a preset local point cloud position determination rule, at least two local point cloud data having an acquisition time interval not less than the preset revisit time interval are determined in each local data region of the current frame point cloud data.

6. The method according to any one of claims 1 to 3, characterized in that Based on the position information of the local point cloud input by the user, at least two local point cloud data with an acquisition time interval not less than the preset revisit time interval are determined in each local data area of ​​the current frame point cloud data.

7. The method according to any one of claims 1 to 6, characterized in that Generating a fitting point cloud having a different angle from the local point cloud data based on at least two local point cloud data in each local data area includes: Determining motion parameters of the corresponding local point cloud data based on the spatial positions and relative acquisition times of at least two local point cloud data in each local data region, wherein the motion parameters include displacement and velocity; Performing trajectory fitting based on the motion parameters of each of the local regions to obtain a scanning trajectory of the target object corresponding to the local data region; and Based on the scanning trajectory, the spatial coordinates of the target object corresponding to the target moment within the time period corresponding to the scanning trajectory are determined, and the fitting point cloud is generated based on the spatial coordinates.

8. The method according to claim 7, characterized in that Through the data association algorithm and the Kalman filter algorithm, the corresponding local point cloud data is subjected to trajectory fitting based on the motion parameters obtained in each local data area to obtain a scanning trajectory of the target object corresponding to the local data area.

9. The method according to claim 7, characterized in that By using a Bayesian filtering algorithm, trajectory fitting is performed on the corresponding local point cloud data based on the motion parameters obtained in each local data area to obtain a scanning trajectory of the target object corresponding to the local data area.

10. The method according to claim 7, characterized in that Performing trajectory fitting on the corresponding local point cloud data based on the motion parameters obtained for each local data region using a time series analysis algorithm to obtain a scanning trajectory of the target object corresponding to the local data region; The time series analysis algorithm includes one of a moving average model, an autoregressive model, and an autoregressive moving average model.

11. The method according to claim 7, characterized in that The laser mine emits multiple laser beams to scan the target scene, wherein the multiple laser beams form at least one group of scan lines, and the at least one group of scan lines sequentially forms multiple scan line groups in the vertical or horizontal direction at at least partially different angles according to the preset revisit time interval, and scans the target scene horizontally and vertically to obtain the single frame of point cloud data; The scanning angles of the scanning lines in the plurality of scanning line groups are at least partially different, and the scanning lines are cyclically arranged in sequence in the vertical or horizontal direction according to the group numbers of the scanning line groups.

12. The method according to claim 11, characterized in that The laser mine emits multiple laser beams to scan the target scene, wherein the multiple laser beams form a group of scan lines, and the group of scan lines sequentially forms N scan line groups in a vertical direction at at least partially different angles according to the preset revisit time interval to vertically scan the target scene, and each scan line in the N scan line groups scans the target scene horizontally to obtain the single-frame point cloud data; The scanning angles of the scanning lines in the N scanning line groups are at least partially different, and the scanning lines are arranged cyclically in sequence in the vertical direction according to the group numbers of the N scanning line groups; The N is an integer greater than 2 and less than 100.

13. The method according to claim 7, characterized in that The determining of the motion parameters of the corresponding local point cloud data based on the spatial positions and relative acquisition times of at least two local point cloud data in each local data region includes: Determining the spatial position distribution of the target object corresponding to each of the local data regions between at least two of the local point cloud data based on the angular distances and relative positions in the vertical or horizontal directions between adjacent scan lines in the plurality of scan line groups; Based on the time difference between adjacent scan lines and the spatial position distribution of the target object corresponding to each local data area between at least two of the local point cloud data, the velocity distribution of the corresponding target object between the at least two of the local point cloud data is determined to obtain a velocity curve of the target object.

14. The method according to claim 13, characterized in that The performing trajectory fitting based on the motion parameters of each of the local data regions to obtain a scanning trajectory of the target object corresponding to the local data region includes: Trajectory fitting is performed based on the spatial position distribution and the velocity distribution of each local area to obtain a scanning trajectory of the target object corresponding to the local data area.

15. The method according to claim 7, characterized in that Also includes: Performing point cloud feature extraction on at least three of the local point cloud data in each of the local data regions to obtain point cloud features of the local data region; Based on the correlation of various features in the point cloud features of the local data region, associating or matching the features with correlation in the point cloud features of each local data region; and Based on the result of associating or matching the point cloud features of each of the local data regions, feature fusion is performed on the corresponding fitting point cloud to obtain a fitting point cloud that matches the local data region.

16. The method according to claim 15, characterized in that The point cloud features include at least one of a geometric profile of a fitted surface, a scan line shape, a normal vector, a point cloud density, a surface curvature, a slope, and a reflectivity; The correlation includes: the correlation between reflectivity and normal direction, the correlation between surface curvature and reflectivity consistency, and the correlation between point cloud density and slope.

17. The method according to claim 15 or 16, characterized in that The step of performing feature fusion on the corresponding fitting point cloud based on the result of associating or matching the point cloud features of each local data region to obtain a fitting point cloud matching the local data region includes: Determining the weight of each type of feature based on the contribution of each type of feature in the point cloud features of each local data area; Based on the result of associating or matching the point cloud features of each of the local data regions, the corresponding fitting point clouds are weightedly fused to obtain a fitting point cloud that matches the local data region.

18. The method according to claim 1, wherein Also includes: Clustering the single-frame point cloud data to obtain aggregated data corresponding to the target object in the single-frame point cloud data; Generating a fitting point cloud having a different angle from the local point cloud data based on at least two local point cloud data in each local data area includes: Trajectory fitting is performed on at least two of the local point cloud data in the local data area included in the aggregated data, and the fitting point cloud is generated based on a scanning trajectory obtained by the trajectory fitting.

19. The method according to claim 18, characterized in that The performing trajectory fitting on at least two of the local point cloud data in the local data area included in the aggregated data, and generating the fitting point cloud based on a scanning trajectory obtained by the trajectory fitting, includes: determining a center of gravity of the target object based on the aggregated data; Determining the spatial position of the center of gravity of the target object based on the time corresponding to at least two of the local point cloud data in the aggregated data; and Based on the relative time of at least two of the local point cloud data in the aggregated data and the spatial position of the center of gravity of the target object, a speed of the target object is determined as the speed of the corresponding local point cloud data.

20. The method according to any one of claims 1 to 6, characterized in that: Clustering the single-frame point cloud data based on at least two of the local point cloud data in each of the local data regions comprises: Searching for ground point cloud data that includes the local data area and the local minimum height, and performing three-dimensional ground surface fitting on the ground point cloud data; Above a preset height threshold of the fitted three-dimensional ground surface, searching for point cloud data whose distance from any point cloud data in another local data region is less than a preset clustering distance threshold, and classifying the point cloud data and the point cloud data in the other local data region into the same cluster data; and Repeat the step of searching for point cloud data whose distance from any point cloud data in another local data area is less than a preset clustering distance threshold above a preset height threshold of the three-dimensional ground surface obtained by fitting, and classifying the point cloud data in the same cluster data as the other local data area to obtain cluster data of the point cloud data in the other local data area.

21. The method according to claim 1, wherein Also includes: Based on the distance information and intensity information of the laser radar's echo signal, the ranging capability and ranging reliability of each scanning angle of the laser radar are determined, so as to determine the contamination condition of the laser radar window at the corresponding angle based on the ranging capability and the ranging reliability.

22. The method according to claim 1, wherein Also includes: Based on the distance information and intensity information of at least three non-primary echo signals of the laser radar, the ranging capability and ranging reliability of each scanning angle of the laser radar are determined.

23. The method according to claim 1, wherein Also includes: receiving stray light signals reflected by objects inside the laser radar; Based on the difference in the vertical or horizontal direction of the stray light signal, the vibration direction, phase and amplitude of the device used for scanning in the corresponding direction in the laser radar are determined, wherein the device used for scanning in the corresponding direction includes a MEMS or prism that does not use a code disk or does not use angle feedback.

24. The method according to claim 1, wherein Also includes: Scanning the target scene by at least one laser radar to obtain the single-frame point cloud data; Among them, at least one group of scanning lines of each of the laser radars scans at least part of the target scene in sequence at at least some different angles according to the preset revisit time interval to obtain at least part of the single-frame point cloud data.

25. A laser radar data processing system, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 23.

26. The system according to claim 25, characterized in that Also includes: At least one laser radar, each of the laser radars comprising a transceiver scanning module; The transceiver scanning module of the at least one laser radar is configured to scan the target scene to obtain the single-frame point cloud data; Among them, the transceiver scanning module of each laser radar is configured to scan at least part of the target scene in sequence at at least some different angles according to the preset revisit time interval through at least one group of scanning lines to obtain at least part of the single-frame point cloud data.

27. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 23 is implemented.

28. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 23.

Images