Point cloud data processing method, device, electronic device, and storage medium

Abstract

The present application provides a point cloud data processing method, device, electronic device, and storage medium. The method can be applied to business scenarios such as ports, ports, road freight, urban distribution, mines, or airports. The method includes: obtaining point cloud data of each point cloud frame in multiple groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames; adding the point cloud data of the previous point cloud frame in a group of point cloud frames to the point cloud data of the adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame; obtaining the point cloud data of the first point cloud frame in each group of point cloud frames, and obtaining updated point cloud frames of each point cloud frame with added point cloud data in each group of point cloud frames to obtain a target point cloud data group. The method of the present application can solve the problem of how to increase the number of single-frame laser point clouds when using laser radar for detection, so as to improve the ability to obtain target information and increase the robustness of laser radar perception.

Description

Technical Field

[0001] The present application relates to autonomous driving image data processing technology, and in particular to a point cloud data processing method, device, electronic equipment, and storage medium, which can be applied to scenarios such as ports, highways, logistics, mines, closed parks, or urban transportation. Background Art

[0002] LiDAR is a radar system that emits laser beams to detect the position, speed and other characteristic quantities of a target. Its working principle is to emit a detection signal (laser beam) to the target, and then compare the received signal reflected from the target (target echo) with the transmitted signal. After appropriate processing, relevant information about the target can be obtained, such as target distance, direction, height, speed, attitude, and even shape parameters. With the development of autonomous driving technology, LiDAR is increasingly widely used in the field of autonomous driving.

[0003] Specifically, when using LiDAR for detection, relevant information about the target is obtained through the collected laser point cloud. However, the number of laser point clouds collected when using LiDAR for detection is currently very small. For example, when a 128-line LiDAR works at 10Hz, it can detect about 30 points of vehicle targets and about 15 points of pedestrian targets at 100 meters. Although the number of collected laser point clouds can also be used for target perception, it cannot be used to obtain more detailed content of the target.

[0004] Therefore, when using LiDAR for detection, how to increase the number of single-frame laser point clouds to improve the ability to obtain target information and increase the robustness of LiDAR perception still needs to be solved. Summary of the invention

[0005] The present application provides a point cloud data processing method, device, electronic device, and storage medium to solve the problem of how to increase the number of single-frame laser point clouds when using laser radar for detection, so as to improve the ability to obtain target information and increase the robustness of laser radar perception.

[0006] On the one hand, the present application provides a point cloud data processing method, comprising:

[0007] Acquire point cloud data of each point cloud frame in a plurality of groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2;

[0008] Adding point cloud data of a previous point cloud frame in a group of point cloud frames to point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame;

[0009] When obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, the point cloud data of the first point cloud frame in each group of point cloud frames is obtained, and the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames is obtained to obtain the target point cloud data group.

[0010] The point cloud data processing method provided in this embodiment adds part or all of the point cloud data of the previous point cloud frame to the point cloud data of the directly or indirectly adjacent subsequent point cloud frame, thereby enriching the point cloud data of the subsequent point cloud frame and increasing the number of laser point clouds and the richness of point cloud information in a single point cloud frame. When performing target detection based on the point cloud information (point cloud data) processed by the method provided in this embodiment, the ability to obtain target information can be improved, and the robustness of laser radar perception can be increased.

[0011] In one embodiment, the step of adding the point cloud data of a previous point cloud frame in a group of point cloud frames to the point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame includes:

[0012] Get the vector velocity information of the point cloud data of the previous point cloud frame;

[0013] Get the working frame rate of the LiDAR;

[0014] Processing the point cloud data of the previous point cloud frame according to the vector velocity information of the point cloud data of the previous point cloud frame and the working frame rate to obtain the point cloud data to be added;

[0015] The point cloud data to be added is added to the point cloud data of the subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame.

[0016] In one embodiment, the point cloud data of the previous point cloud frame is processed according to the vector velocity information of the point cloud data of the previous point cloud frame and the working frame rate to obtain the point cloud data to be added, including:

[0017] For each point cloud data in the plurality of point cloud data in the previous point cloud frame, determining a moving direction of the first point cloud data according to a speed direction in the vector speed information of the first point cloud data;

[0018] Determine a moving distance of the first point cloud data according to a speed value in the vector speed information of the first point cloud data and the working frame rate;

[0019] The first point cloud data is processed according to the moving direction and moving distance of the first point cloud data to obtain the first point cloud data to be added.

[0020] In one embodiment, the adjacent relationship between the previous point cloud frame and the adjacent subsequent point cloud frame is directly adjacent or indirectly adjacent;

[0021] When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, there are 0 point cloud frames formed;

[0022] When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirect, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, M point cloud frames are formed, where M is a natural number greater than zero.

[0023] In one embodiment, when the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, determining the moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate includes:

[0024] Determine the moving distance of the first point cloud data according to the formula h=v / f;

[0025] Among them, v represents the speed value in the vector speed information of the first point cloud data, f represents the working frame rate, and h represents the moving distance of the first point cloud data.

[0026] In one embodiment, when the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirect, determining the moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate includes:

[0027] Determine the moving distance of the first point cloud data according to the formula h=(M+1)v / f;

[0028] Among them, v represents the speed value in the vector speed information of the first point cloud data, f represents the working frame rate, and h represents the moving distance of the first point cloud data.

[0029] In one embodiment, the method further comprises:

[0030] Acquire data attributes of point cloud data of a previous point cloud frame, wherein the data attributes at least include an azimuth attribute, a pitch angle attribute, a distance attribute, and a speed attribute, and the attribute values ​​in the data attributes at least include an azimuth value, a pitch angle value, a distance value, and a speed value;

[0031] The step of adding the point cloud data of a previous point cloud frame in a group of point cloud frames to the point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame comprises:

[0032] When any attribute value of the data attributes of the second point cloud data of the current point cloud frame is different from each attribute value of the data attributes of each point cloud data of the adjacent subsequent point cloud frame, the second point cloud data is added to the point cloud data of the adjacent subsequent point cloud frame;

[0033] When each attribute value in the data attributes of the third point cloud data of the current point cloud frame is the same as each attribute value in the data attributes of the fourth point cloud data of the adjacent subsequent point cloud frame, no data adding action is performed, or the fourth point cloud data is updated to the third point cloud data.

[0034] The point cloud data processing method provided in this embodiment further filters out some point cloud data of the previous point cloud frame when adding the point cloud data of the previous point cloud frame in a group of point cloud frames to the point cloud data of the adjacent subsequent point cloud frame, thereby reducing the amount of calculation during point cloud data processing.

[0035] On the other hand, the present application provides a point cloud data processing device, comprising:

[0036] An acquisition module, used to acquire point cloud data of each point cloud frame in a plurality of groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2;

[0037] An updating module, used for adding the point cloud data of a previous point cloud frame in a group of point cloud frames to the point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame;

[0038] The acquisition module is also used to obtain the point cloud data of the first point cloud frame in each group of point cloud frames when obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, and to obtain the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames to obtain the target point cloud data group.

[0039] On the other hand, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0040] The memory stores computer-executable instructions;

[0041] The processor executes the computer-executable instructions stored in the memory to implement the point cloud data processing method as described in the first aspect.

[0042] On the other hand, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the instructions are executed, the computer executes the point cloud data processing method as described in the first aspect.

[0043] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the point cloud data processing method as described in the first aspect.

[0044] In summary, the method provided in the embodiment of the present application obtains point cloud data of each point cloud frame in multiple groups of point cloud frames when processing point cloud data, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2; the point cloud data of the previous point cloud frame in a group of point cloud frames is added to the point cloud data of the adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame; when obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, the point cloud data of the first point cloud frame in each group of point cloud frames is obtained, and the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames is obtained to obtain the target point cloud data group.

[0045] That is, part of the point cloud data or all of the point cloud data of the previous point cloud frame is added to the point cloud data of the directly or indirectly adjacent subsequent point cloud frame, thereby enriching the point cloud data of the subsequent point cloud frame and increasing the number of laser point clouds and the richness of point cloud information in a single point cloud frame. When performing target detection based on the point cloud information (point cloud data) processed by the method provided in this embodiment, the ability to obtain target information can be improved, and the robustness of laser radar perception can be increased. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of an application scenario of the point cloud data processing method provided in this application;

[0047] Figure 2 A schematic diagram of a process flow of a point cloud data processing method provided by one embodiment of the present application;

[0048] Figure 3 A schematic diagram of a laser point cloud provided for one embodiment of the present application;

[0049] Figure 4 Another schematic diagram of a laser point cloud provided for one embodiment of the present application;

[0050] Figure 5 A schematic diagram of a point cloud data processing device provided by one embodiment of the present application;

[0051] Figure 6 A schematic diagram of an electronic device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0053] In the description of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the feature. In the description of the present application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0054] LiDAR is a radar system that emits laser beams to detect the position, speed and other characteristic quantities of a target. Its working principle is to emit a detection signal (laser beam) to the target, and then compare the received signal reflected from the target (target echo) with the transmitted signal. After appropriate processing, relevant information about the target can be obtained, such as target distance, direction, height, speed, attitude, and even shape parameters. With the development of autonomous driving technology, LiDAR is increasingly widely used in the field of autonomous driving.

[0055] Specifically, when using LiDAR for detection, relevant information about the target is obtained through the collected laser point cloud. For driving targets on the road, the number of returned point clouds will have obvious changes in point cloud density depending on the size of the target and the distance of the target. However, when using LiDAR for detection, the number of collected laser point clouds is very small, especially compared with the image information collected by the camera, the point cloud density of LiDAR is relatively low. For example, when a 128-line LiDAR usually works at 10Hz, there are about 30 points of vehicle targets and about 15 points of pedestrian targets detected at 100 meters. Although the number of collected laser point clouds can also be used for target perception, it cannot be used to obtain more detailed content of the target.

[0056] Therefore, when using laser radar for detection, how to increase the number of laser point clouds in a single frame to improve the ability to obtain information about the target and increase the robustness of laser radar perception is still a problem that needs to be solved. Based on this, the present application provides a point cloud data processing method, device, electronic device, and storage medium. When processing point cloud data, the point cloud data processing method obtains point cloud data of each point cloud frame in multiple groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2; the point cloud data of the previous point cloud frame in a group of point cloud frames is added to the point cloud data of the adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame; when obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, the point cloud data of the first point cloud frame in each group of point cloud frames is obtained, and the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames is obtained to obtain the target point cloud data group. That is, part of the point cloud data or all of the point cloud data of the previous point cloud frame is added to the point cloud data of the directly or indirectly adjacent subsequent point cloud frame, thereby enriching the point cloud data of the subsequent point cloud frame and increasing the number of laser point clouds and the richness of point cloud information in a single point cloud frame. When performing target detection based on the point cloud information (point cloud data) processed by the method provided in this application, the ability to obtain target information can be improved, and the robustness of laser radar perception can be increased.

[0057] The point cloud data processing method provided in the present application is applied to electronic devices, such as computers, servers, image processing devices, etc. Figure 1 This is a schematic diagram of the application of the point cloud data processing method provided by the present application. In the figure, the electronic device obtains the point cloud data of each point cloud frame in multiple groups of point cloud frames generated by the laser radar, and adds the point cloud data of the previous point cloud frame in a group of point cloud frames to the point cloud data of the adjacent subsequent point cloud frame to obtain the updated point cloud data of the subsequent point cloud frame. When obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, the point cloud data of the first point cloud frame in each group of point cloud frames is obtained, and the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames is obtained to obtain the target point cloud data group.

[0058] See also Figure 2 , an embodiment of the present application provides a point cloud data processing method. The point cloud data processing method is applied to a laser radar, which is used to emit a laser beam and generate point cloud data of multiple point cloud frames based on the reflected signal wave. It should be noted that the laser radar has both speed measurement performance and distance measurement performance. For example, the FMCW 4D laser radar, the generated point cloud data has speed information, distance information, and angle information (direction angle, pitch angle). The point cloud data of a point cloud frame includes multiple point cloud data, and the information of the point cloud data is associated with the movement of the target. The information of each point cloud data may be the same or different.

[0059] The point cloud data processing method includes:

[0060] S210, acquiring point cloud data of each point cloud frame in a plurality of groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2.

[0061] Point cloud frames are continuous. When processing point cloud data, all point cloud frames collected by the LiDAR can be divided into multiple groups of point cloud frames in chronological order. The number of point cloud frames contained in each group of point cloud frames can be the same or different. For example, N is equal to 5, and a group of point cloud frames contains 5 continuous point cloud frames, which are named the first frame, the second frame, the third frame, the fourth frame, and the fifth frame in chronological order.

[0062] S220, adding the point cloud data of the previous point cloud frame in a group of point cloud frames to the point cloud data of the adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame.

[0063] The amount of point cloud data and the information contained in a single point cloud frame are limited. If the point cloud data of different point cloud frames can be superimposed, the amount of point cloud data in a single point cloud frame can be increased and the richness of the point cloud data can be improved.

[0064] The adjacent relationship between the previous point cloud frame and the adjacent subsequent point cloud frame can be directly adjacent or indirectly adjacent. When the adjacent relationship between the previous point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, 0 point cloud frames are formed. When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirectly adjacent, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, M point cloud frames are formed, and M is a natural number greater than zero. Correspondingly, when adding the point cloud data of the previous point cloud frame in a group of point cloud frames to the point cloud data of the adjacent subsequent point cloud frame, the point cloud data of the previous point cloud frame can be added to the point cloud data of the directly adjacent subsequent point cloud frame, or the point cloud data of the previous point cloud frame can be added to the point cloud data of the indirectly adjacent subsequent point cloud frame.

[0065] When adding the point cloud data of the previous point cloud frame to the point cloud data of the adjacent subsequent point cloud frame, it is necessary to know the moving direction and moving distance of the point cloud data of the previous point cloud frame. In an optional embodiment, the point cloud data of the previous point cloud frame can be processed according to the vector velocity information of the point cloud data of the previous point cloud frame and the working frame rate of the laser radar, and the point cloud data can be added after the moving direction and moving distance of the point cloud data of the previous point cloud frame are obtained.

[0066] Specifically, the vector velocity information of the point cloud data of the previous point cloud frame is obtained, and the working frame rate of the laser radar is obtained. According to the vector velocity information of the point cloud data of the previous point cloud frame and the working frame rate, the point cloud data of the previous point cloud frame is processed to obtain the point cloud data to be added. The point cloud data to be added is the point cloud data with the moving direction and the moving distance. The point cloud data to be added is added to the point cloud data of the next point cloud frame to obtain the updated point cloud data of the next point cloud frame.

[0067] The vector velocity information includes a velocity value and a velocity direction. Specifically, for each point cloud data in the plurality of point cloud data in the previous point cloud frame, the moving direction of the first point cloud data is determined according to the velocity direction in the vector velocity information of the first point cloud data. The moving distance of the first point cloud data is determined according to the velocity value in the vector velocity information of the first point cloud data and the working frame rate. The first point cloud data is processed according to the moving direction and moving distance of the first point cloud data to obtain the first point cloud data to be added.

[0068] Among them, the multiple point cloud data in the previous point cloud frame can be all point cloud data or part of the point cloud data. The part of the point cloud data can be some manually selected data, or it can be the point cloud data that needs to be added that is screened out through the data attributes of the point cloud data. Specifically, the data attributes of the point cloud data of the previous point cloud frame are obtained, and the data attributes at least include azimuth attributes, pitch angle attributes, distance attributes and speed attributes, and the attribute values ​​of the data attributes at least include azimuth values, pitch angle values, distance values ​​and speed values. When the data attributes of a point cloud data in the current point cloud frame are exactly the same as the data attributes of any point cloud data in the subsequent point cloud frame, the point cloud data in the previous point cloud frame belongs to the point cloud data that does not need to be added, and can be not added to the point cloud data in the subsequent point cloud frame. Except for the point cloud data that does not need to be added in the point cloud data of the previous point cloud frame, the other point cloud data are the point cloud data that need to be added, that is, the multiple point cloud data in the previous point cloud frame described above.

[0069] The first point cloud data represents any one of the multiple point cloud data in the previous point cloud frame. The moving direction of the first point cloud data is determined according to the speed direction in the vector speed information of the first point cloud data, that is, the moving direction of the first point cloud data is determined to be the speed direction in the vector speed information of the first point cloud data.

[0070] When determining the moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate, there are two cases:

[0071] The first case: as described above, the adjacent relationship between the previous point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, that is, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, there are 0 point cloud frames formed.

[0072] The adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is directly adjacent. When determining the moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate, specifically, the moving distance of the first point cloud data is determined according to the formula h=v / f, wherein v represents the speed value in the vector speed information of the first point cloud data, f represents the working frame rate, and h represents the moving distance of the first point cloud data. For example, when the first point cloud data is added to the point cloud data of the next frame, i.e., the second point cloud frame, the moving distance of the first point cloud data is h=v / f.

[0073] The second case: the adjacent relationship between the previous point cloud frame and the adjacent subsequent point cloud frame is indirect adjacent. After the previous point cloud frame is formed and before the subsequent point cloud frame is formed, M point cloud frames are formed, and M is a natural number greater than zero.

[0074] The adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirect adjacent. When determining the moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate, specifically, the moving distance of the first point cloud data is determined according to the formula h=(M+1)v / f, where v represents the speed value in the vector speed information of the first point cloud data, f represents the working frame rate, and h represents the moving distance of the first point cloud data. For example, when the first point cloud data is added to the point cloud data of the next next frame, that is, the third point cloud frame, the moving distance of the first point cloud data is

[0075] In an optional embodiment, in order to reduce the amount of calculation, when the point cloud data of the previous point cloud frame in a group of point cloud frames is added to the point cloud data of the adjacent subsequent point cloud frame to obtain the updated point cloud data of the subsequent point cloud frame, some point cloud data of the previous point cloud frame do not need to be added.

[0076] Specifically, the data attributes of the point cloud data of the previous point cloud frame are obtained, and the data attributes at least include azimuth attributes, pitch angle attributes, distance attributes and speed attributes, and the attribute values ​​in the data attributes at least include azimuth values, pitch angle values, distance values ​​and speed values. When any attribute value in the data attributes of the second point cloud data of the current point cloud frame is different from each attribute value in the data attributes of each point cloud data of the adjacent subsequent point cloud frame, the second point cloud data is added to the point cloud data of the adjacent subsequent point cloud frame. When each attribute value in the data attributes of the third point cloud data of the current point cloud frame is the same as each attribute value in the data attributes of the fourth point cloud data of the adjacent subsequent point cloud frame, the data adding action is not performed, or the fourth point cloud data is updated to the third point cloud data.

[0077] In particular, when no data addition action is performed, the workload of data addition is reduced, the processing speed of point cloud data is improved, and the workload of later using point cloud data to construct target information is reduced.

[0078] S230, when obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, obtain the point cloud data of the first point cloud frame in each group of point cloud frames, and obtain the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames to obtain the target point cloud data group.

[0079] The point cloud data of the first point cloud frame in each group of point cloud frames is not updated. In order to ensure data integrity, after all the point cloud data are updated, it is necessary not only to obtain the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames, but also to obtain the point cloud data of the first point cloud frame in each group of point cloud frames.

[0080] In an optional embodiment, it is not necessary to obtain the target point cloud data group when obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames. Instead, when obtaining the updated point cloud data of the Nth point cloud frame in L (L is a natural number greater than 0) groups of point cloud frames, the point cloud data of the first point cloud frame in each group of L groups of point cloud frames is obtained, and the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames is obtained to obtain the target point cloud data group.

[0081] Since the point cloud data in the multiple point cloud frames from which the target point cloud data group comes are all superimposed on the point cloud data of multiple previous point cloud frames, the amount and richness of the point cloud data have been greatly improved. Therefore, when constructing target information based on the target point cloud data group, the ability to obtain target information can be improved.

[0082] See also Figure 3 and Figure 4 . Figure 3The original point cloud data that has not been processed by the point cloud data processing method described in any of the above embodiments is shown, including lane line point cloud (parallelogram area pointed to by A) and target point cloud (rectangular area pointed to by B, the target is, for example, a car), and ground point cloud (trapezoidal area pointed to by C). The single point cloud frame of the original point cloud data is not superimposed, and the number and density of point clouds are both small. Figure 4 The target point cloud data set is obtained by processing the point cloud data processing method described in any of the above embodiments, including the lane line point cloud (the parallelogram area pointed to by A) and the target point cloud (the rectangular area pointed to by B, the target is, for example, a car), and also includes the ground point cloud (the trapezoidal area pointed to by C). Figure 3 and Figure 4 It can be seen that the target point cloud data set is larger than the original point cloud data in terms of point cloud quantity and point cloud density.

[0083] In summary, the method provided in the embodiment of the present application obtains point cloud data of each point cloud frame in multiple groups of point cloud frames when processing point cloud data, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2; the point cloud data of the previous point cloud frame in a group of point cloud frames is added to the point cloud data of the adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame; when obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, the point cloud data of the first point cloud frame in each group of point cloud frames is obtained, and the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames is obtained to obtain the target point cloud data group.

[0084] That is, part of the point cloud data or all of the point cloud data of the previous point cloud frame is added to the point cloud data of the directly or indirectly adjacent subsequent point cloud frame, thereby enriching the point cloud data of the subsequent point cloud frame and increasing the number of laser point clouds and the richness of point cloud information in a single point cloud frame. When performing target detection based on the point cloud information (point cloud data) processed by the method provided in this embodiment, the ability to obtain target information can be improved, and the robustness of laser radar perception can be increased.

[0085] See also Figure 5 One embodiment of the present application further provides a point cloud data processing device 10, which is applied to a laser radar, the laser radar is used to emit a laser beam and generate point cloud data of multiple point cloud frames based on the reflected signal wave. The point cloud data processing device 10 includes:

[0086] The acquisition module 11 is used to acquire point cloud data of each point cloud frame in multiple groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2.

[0087] The updating module 12 is used to add the point cloud data of the previous point cloud frame in a group of point cloud frames to the point cloud data of the adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame.

[0088] The acquisition module 11 is also used to obtain the point cloud data of the first point cloud frame in each group of point cloud frames when obtaining the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, and to obtain the updated point cloud frame of each point cloud frame with added point cloud data in each group of point cloud frames to obtain the target point cloud data group.

[0089] The update module 12 is specifically used to obtain the vector velocity information of the point cloud data of the previous point cloud frame; obtain the working frame rate of the laser radar; process the point cloud data of the previous point cloud frame according to the vector velocity information of the point cloud data of the previous point cloud frame and the working frame rate to obtain the point cloud data to be added; add the point cloud data to be added to the point cloud data of the subsequent point cloud frame to obtain the updated point cloud data of the subsequent point cloud frame.

[0090] The update module 12 is specifically used to determine, for each point cloud data in the multiple point cloud data in the previous point cloud frame, a moving direction of the first point cloud data according to the speed direction in the vector speed information of the first point cloud data; determine a moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate; and process the first point cloud data according to the moving direction and moving distance of the first point cloud data to obtain the first point cloud data to be added.

[0091] The adjacent relationship between the previous point cloud frame and the adjacent subsequent point cloud frame is directly adjacent or indirectly adjacent; when the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, 0 point cloud frames are formed; when the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirectly adjacent, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, M point cloud frames are formed, and M is a natural number greater than zero.

[0092] When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, the update module 12 is specifically used to determine the moving distance of the first point cloud data according to the formula h=v / f; wherein v represents the speed value in the vector speed information of the first point cloud data, f represents the working frame rate, and h represents the moving distance of the first point cloud data.

[0093] When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirect, the update module 12 is specifically used to determine the moving distance of the first point cloud data according to the formula h=(M+1)v / f; wherein v represents the speed value in the vector speed information of the first point cloud data, f represents the working frame rate, and h represents the moving distance of the first point cloud data.

[0094] The acquisition module 11 is also used to acquire data attributes of the point cloud data of the previous point cloud frame, and the data attributes include at least azimuth attribute, pitch angle attribute, distance attribute and speed attribute, and the attribute values ​​in the data attributes include at least azimuth value, pitch angle value, distance value and speed value. The update module 12 is specifically used to add the second point cloud data to the point cloud data of the adjacent subsequent point cloud frame when any attribute value in the data attributes of the second point cloud data of the current point cloud frame is different from each attribute value in the data attributes of each point cloud data of the adjacent subsequent point cloud frame; when each attribute value in the data attributes of the third point cloud data of the current point cloud frame is the same as each attribute value in the data attributes of the fourth point cloud data of the adjacent subsequent point cloud frame, no data addition action is performed, or the fourth point cloud data is updated to the third point cloud data.

[0095] See also Figure 6 An embodiment of the present application further provides an electronic device 20, comprising: a processor 21, and a memory 22 in communication with the processor 21. The memory 22 stores computer-executable instructions, and the processor 21 executes the computer-executable instructions stored in the memory 22 to implement the point cloud data processing method provided in any of the above embodiments.

[0096] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the instructions are executed, the computer executes the point cloud data processing method provided in any of the above embodiments.

[0097] One embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the point cloud data processing method provided in any of the above embodiments.

[0098] It should be noted that the computer-readable storage medium may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM), etc. It may also be various electronic devices including one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0099] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0100] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0102] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A point cloud data processing method, It is characterized in that include: Acquire point cloud data of each point cloud frame in a plurality of groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2; Adding point cloud data of a previous point cloud frame in a group of point cloud frames to point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame; When obtaining the updated point cloud data of the Nth point cloud frame in each set of point cloud frames, obtaining the point cloud data of the first point cloud frame in each set of point cloud frames, and obtaining the updated point cloud frame of each point cloud frame to which the point cloud data is added in each set of point cloud frames, to obtain the target point cloud data set; The method further comprises: Acquire data attributes of point cloud data of a previous point cloud frame, wherein the data attributes at least include an azimuth attribute, a pitch angle attribute, a distance attribute, and a speed attribute, and the attribute values ​​in the data attributes at least include an azimuth value, a pitch angle value, a distance value, and a speed value; The step of adding the point cloud data of a previous point cloud frame in a group of point cloud frames to the point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame comprises: When any attribute value of the data attributes of the second point cloud data of the current point cloud frame is different from each attribute value of the data attributes of each point cloud data of the adjacent subsequent point cloud frame, the second point cloud data is added to the point cloud data of the adjacent subsequent point cloud frame; When each attribute value in the data attributes of the third point cloud data of the current point cloud frame is the same as each attribute value in the data attributes of the fourth point cloud data of the adjacent subsequent point cloud frame, no data adding action is performed, or the fourth point cloud data is updated to the third point cloud data.

2. The method according to claim 1, It is characterized in that The step of adding the point cloud data of a previous point cloud frame in a group of point cloud frames to the point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame comprises: Get the vector velocity information of the point cloud data of the previous point cloud frame; Get the working frame rate of the LiDAR; Processing the point cloud data of the previous point cloud frame according to the vector velocity information of the point cloud data of the previous point cloud frame and the working frame rate to obtain the point cloud data to be added; The point cloud data to be added is added to the point cloud data of the subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame.

3. The method according to claim 2, It is characterized in that The step of processing the point cloud data of the previous point cloud frame according to the vector velocity information of the point cloud data of the previous point cloud frame and the working frame rate to obtain the point cloud data to be added includes: For each point cloud data in the plurality of point cloud data in the previous point cloud frame, determining a moving direction of the first point cloud data according to a speed direction in the vector speed information of the first point cloud data; Determine a moving distance of the first point cloud data according to a speed value in the vector speed information of the first point cloud data and the working frame rate; The first point cloud data is processed according to the moving direction and moving distance of the first point cloud data to obtain the first point cloud data to be added.

4. The method according to claim 3, It is characterized in that The adjacent relationship between the previous point cloud frame and the adjacent subsequent point cloud frame is direct adjacent or indirect adjacent; When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, there are 0 point cloud frames formed; When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirect, after the previous point cloud frame is formed and before the subsequent point cloud frame is formed, Point cloud frames, is a natural number greater than zero.

5. The method according to claim 4, It is characterized in that When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is directly adjacent, determining the moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate includes: According to the formula Determine the moving distance of the first point cloud data; in, Represents the velocity value in the vector velocity information of the first point cloud data, represents the working frame rate, Represents the moving distance of the first point cloud data.

6. The method according to claim 4 or 5, It is characterized in that When the adjacent relationship between the current point cloud frame and the adjacent subsequent point cloud frame is indirect adjacent, determining the moving distance of the first point cloud data according to the speed value in the vector speed information of the first point cloud data and the working frame rate includes: According to the formula Determine the moving distance of the first point cloud data; in, Represents the velocity value in the vector velocity information of the first point cloud data, represents the working frame rate, Represents the moving distance of the first point cloud data.

7. A point cloud data processing device, It is characterized in that include: An acquisition module, used to acquire point cloud data of each point cloud frame in a plurality of groups of point cloud frames, wherein a group of point cloud frames includes N consecutive point cloud frames, and N is a natural number greater than or equal to 2; An updating module, used for adding the point cloud data of a previous point cloud frame in a group of point cloud frames to the point cloud data of an adjacent subsequent point cloud frame to obtain updated point cloud data of the subsequent point cloud frame; The acquisition module is also used to acquire the point cloud data of the first point cloud frame in each group of point cloud frames when acquiring the updated point cloud data of the Nth point cloud frame in each group of point cloud frames, and acquire the updated point cloud frame of each point cloud frame to which the point cloud data is added in each group of point cloud frames, so as to obtain the target point cloud data group; The acquisition module is further used to acquire data attributes of the point cloud data of the previous point cloud frame, wherein the data attributes at least include an azimuth attribute, a pitch angle attribute, a distance attribute and a speed attribute, and the attribute values ​​in the data attributes at least include an azimuth value, a pitch angle value, a distance value and a speed value; The updating module is specifically used to add the second point cloud data to the point cloud data of the adjacent subsequent point cloud frame when any attribute value of the data attributes of the second point cloud data of the current point cloud frame is different from each attribute value of the data attributes of each point cloud data of the adjacent subsequent point cloud frame; When each attribute value in the data attributes of the third point cloud data of the current point cloud frame is the same as each attribute value in the data attributes of the fourth point cloud data of the adjacent subsequent point cloud frame, no data adding action is performed, or the fourth point cloud data is updated to the third point cloud data.

8. An electronic device, It is characterized in that include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the point cloud data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when the instructions are executed, the computer executes the point cloud data processing method according to any one of claims 1 to 6.

Images