Point cloud data processing method and device, computer device and storage medium

Abstract

The application relates to a point cloud data processing method and device, computer equipment and a storage medium. It belongs to the field of automatic driving, and the method comprises the following steps: acquiring original point cloud data of at least one laser radar; performing encapsulation processing on each piece of original point cloud data to obtain a standard point cloud data packet corresponding to each piece of original point cloud data; determining an associated transmission interface of each standard point cloud data packet in the depth processing equipment; and sending each standard point cloud data packet to the depth processing equipment through the associated transmission interface, so that the depth processing equipment performs depth processing on each standard point cloud data packet. The method encapsulates the acquired original point cloud data of each laser radar through an encapsulation unit, encapsulates the original point cloud data into a standard format point cloud data packet, and then transmits the point cloud data packet to the depth processing equipment through an associated transmission interface, thereby effectively reducing the processing burden of the depth processing equipment and improving the processing efficiency.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a point cloud data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] In autonomous driving verification systems, depth processing equipment is typically used to acquire and process point cloud data output by LiDAR. Since autonomous driving verification systems use LiDAR from multiple manufacturers, it is necessary to equip the depth processing equipment with driver software from each manufacturer, acquire point cloud data via the network, and then process the point cloud data.

[0003] Because the data formats output by LiDARs from different manufacturers are inconsistent, and because the amount of point cloud data transmitted by LiDARs to depth processing equipment (such as industrial control computers in autonomous driving verification systems) is large, this greatly increases the burden on depth processing equipment when processing point cloud data. Summary of the Invention

[0004] Therefore, it is necessary to provide a point cloud data processing method, apparatus, computer equipment, and storage medium that can reduce the data processing volume of deep processing devices in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a point cloud data processing method. The method includes:

[0006] Acquire raw point cloud data from at least one LiDAR source;

[0007] The raw point cloud data from each source is encapsulated to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0008] Determine the associated transmission interface for each standard point cloud data packet in the deep processing device;

[0009] The standard point cloud data packets are sent to the depth processing device via the associated transmission interface so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0010] In one embodiment, the raw point cloud data from each path is encapsulated to obtain a standard point cloud data package corresponding to each path of raw point cloud data, including:

[0011] The raw point cloud data from each source is packaged into a uniformly formatted raw point cloud data package according to preset rules.

[0012] Based on the identification information of the LiDAR that collected the raw point cloud data from each channel, an identity identifier is added to each raw point cloud data packet to obtain the standard point cloud data packet corresponding to each raw point cloud data.

[0013] In one embodiment, determining the associated transmission interface for each standard point cloud data packet in the depth processing device includes:

[0014] Based on the identity identifier carried by each standard point cloud data packet, the associated transmission interface of each standard point cloud data packet in the depth processing device is determined.

[0015] In one embodiment, the raw point cloud data from each path is encapsulated and processed, including:

[0016] The raw point cloud data from each path is preprocessed to obtain the target point cloud data; the preprocessing includes: angle compensation processing and / or coordinate system transformation processing;

[0017] The target point cloud data is encapsulated and processed.

[0018] In one embodiment, angle compensation processing is performed on each source of raw point cloud data, including:

[0019] Based on the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value in each source of raw point cloud data, horizontal angle compensation processing is performed on each source of raw point cloud data.

[0020] In one embodiment, it further includes:

[0021] Obtain the emission timing deviation and rotation speed of the lidar;

[0022] The emission time compensation value is determined based on the emission time deviation value and the rotation speed.

[0023] In one embodiment, the coordinate system transformation process includes at least one of: transformation from spherical coordinate system to vehicle coordinate system and transformation from spherical coordinate system to local geographic coordinate system; the original point cloud data are in spherical coordinate system.

[0024] In one embodiment, acquiring raw point cloud data from at least one LiDAR source includes:

[0025] Obtain the raw point cloud data packet from at least one LiDAR source;

[0026] Frame parsing is performed on the raw point cloud data packets of at least one LiDAR to obtain the raw point cloud data of at least one LiDAR.

[0027] Secondly, this application also provides a point cloud data processing apparatus. The apparatus includes:

[0028] The acquisition module acquires raw point cloud data from at least one LiDAR source.

[0029] The encapsulation module encapsulates the raw point cloud data from each source to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0030] The interface determination module determines the associated transmission interface of each standard point cloud data packet in the depth processing device.

[0031] The sending module sends each standard point cloud data packet to the depth processing device through the associated transmission interface, so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0032] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0033] Acquire raw point cloud data from at least one LiDAR source;

[0034] The raw point cloud data from each source is encapsulated to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0035] Determine the associated transmission interface for each standard point cloud data packet in the deep processing device;

[0036] The standard point cloud data packets are sent to the depth processing device via the associated transmission interface so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0037] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0038] Acquire raw point cloud data from at least one LiDAR source;

[0039] The raw point cloud data from each source is encapsulated to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0040] Determine the associated transmission interface for each standard point cloud data packet in the deep processing device;

[0041] The standard point cloud data packets are sent to the depth processing device via the associated transmission interface so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0042] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0043] Acquire raw point cloud data from at least one LiDAR source;

[0044] The raw point cloud data from each source is encapsulated to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0045] Determine the associated transmission interface for each standard point cloud data packet in the deep processing device;

[0046] The standard point cloud data packets are sent to the depth processing device via the associated transmission interface so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0047] The aforementioned point cloud data processing method, apparatus, computer equipment, and storage medium acquire raw point cloud data from one or more LiDAR sources, encapsulate the acquired raw point cloud data into standard point cloud data packets of a standard format, determine the associated transmission interface for each standard point cloud data packet in the depth processing device, and finally send each standard point cloud data packet to the depth processing device through the associated transmission interface for depth processing. This application encapsulates the acquired raw point cloud data from each LiDAR source into standard format point cloud data packets through an encapsulation unit, and then transmits them to the depth processing device through the associated transmission interface. This achieves integrated and standardized processing of raw point cloud data from different LiDAR sources. For the depth processing device, there is no need to increase the processing load due to different point cloud data formats, thus effectively reducing the processing burden of the depth processing device and improving processing efficiency. Attached Figure Description

[0048] Figure 1 This is an application environment diagram of the point cloud data processing method provided in this embodiment;

[0049] Figure 2 This is a flowchart illustrating the first point cloud data processing method provided in this embodiment;

[0050] Figure 3 This is a schematic diagram illustrating the process of encapsulating raw point cloud data provided in this embodiment;

[0051] Figure 4 This is a schematic diagram of the original unit data preprocessing process provided in this embodiment;

[0052] Figure 5 This is a schematic diagram of the process for angle compensation of the original unit data provided in this embodiment;

[0053] Figure 6 This is a schematic diagram of the process for obtaining raw point cloud data provided in this embodiment;

[0054] Figure 7 This is a flowchart illustrating the second point cloud data processing method provided in this embodiment;

[0055] Figure 8 This is a simplified flowchart of the point cloud data processing method provided in this embodiment;

[0056] Figure 9 This is a structural block diagram of the first point cloud data processing device provided in this embodiment;

[0057] Figure 10 This is a structural block diagram of the second type of point cloud data processing device provided in this embodiment;

[0058] Figure 11 This is a structural block diagram of the third point cloud data processing device provided in this embodiment;

[0059] Figure 12 This is a structural block diagram of the fourth point cloud data processing device provided in this embodiment;

[0060] Figure 13 This is an internal structural diagram of the computer device provided in this embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] The point cloud data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the LiDAR 102 communicates with the preprocessing device 104 via a communication network, and the preprocessing device 104 communicates with the depth processing device 106 via a communication network. Specifically, the LiDAR 102 sends the raw point cloud data obtained from scanning to the preprocessing device 104. The preprocessing device 104 acquires the raw point cloud data from each LiDAR and encapsulates it into standard point cloud data packets. After determining the associated transmission interface of each standard point cloud data packet in the depth processing device 106, the preprocessing device 104 sends each standard point cloud data packet to the depth processing device 106 through the associated transmission interface for depth processing. The preprocessing device uses, but is not limited to, an FPGA computing platform, and the depth processing device uses, but is not limited to, an industrial control computer.

[0063] In one embodiment, such as Figure 2 As shown, a point cloud data processing method is provided, which can be applied to... Figure 1 Taking the pretreatment equipment in the example, the following steps are included:

[0064] S201, acquire raw point cloud data from at least one LiDAR source.

[0065] Among them, lidar is a radar system that uses laser beams to detect the position, velocity and other characteristics of a target; raw point cloud data is point cloud data obtained by lidar scanning without encapsulation processing.

[0066] Optionally, in this embodiment, multiple acquisition channels can be configured. These channels acquire raw point cloud data from at least one LiDAR source via wired or wireless communication.

[0067] S202 encapsulates the raw point cloud data from each channel to obtain the standard point cloud data package corresponding to each raw point cloud data.

[0068] Among them, encapsulation processing refers to encapsulating the raw point cloud data into a standard format point cloud data packet; a standard point cloud data packet refers to a point cloud data packet encapsulated according to a certain standard format, which mainly encapsulates the raw point cloud data corresponding to each LiDAR into a unified format point cloud data packet.

[0069] Optionally, the encapsulation standard should be determined first, and the raw point cloud data of each channel should be encapsulated according to the encapsulation standard to obtain the standard point cloud data package corresponding to each raw point cloud data. The encapsulation standard can be a self-defined encapsulation rule or an industry standard rule.

[0070] This embodiment can acquire raw point cloud data from multiple LiDARs. The raw point cloud data formats of different LiDARs may be different. This application can encapsulate the raw point cloud data of each channel to obtain a standard point cloud data package with the same format.

[0071] S203, determine the associated transmission interface of each standard point cloud data packet in the depth processing device.

[0072] The associated transmission interface refers to the transmission interface that is associated with each standard point cloud data packet. The deep processing device is a device that further processes the point cloud data in the standard point cloud data packet, such as an industrial control computer.

[0073] In this embodiment, the depth processing device has multiple transmission interfaces. When determining the associated transmission interface for each standard point cloud data packet in the depth processing device, one possible implementation is to configure a fixed transmission interface as the associated transmission interface based on the identification information of the LiDAR. Therefore, each LiDAR has a corresponding associated transmission interface. Based on the LiDAR to which each standard point cloud data packet belongs, the associated transmission interface for each standard point cloud data packet in the depth processing device can be determined. Another possible implementation is to pre-determine the idle transmission interfaces in the depth processing device when transmitting standard point cloud data packets, and select at least one transmission interface from these idle interfaces as the associated transmission interface. For example, each LiDAR can be assigned one associated transmission interface.

[0074] S204 sends each standard point cloud data packet to the depth processing device through the associated transmission interface, so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0075] Optionally, the standard point cloud data packet is transmitted to the character device of the depth processing device through the associated transmission interface. When the depth processing device has processing requirements, it retrieves the standard point cloud data packet from the character device for depth processing. Each transmission interface has a corresponding connected character device, which is a device that transmits data in units of characters during I / O transmission.

[0076] In this embodiment, raw point cloud data from one or more LiDAR sources can be acquired. The acquired raw point cloud data is then encapsulated according to a unified encapsulation standard into standard point cloud data packets of a standard format. The associated transmission interface for each standard point cloud data packet in the depth processing device is determined. Finally, the standard point cloud data packets are sent to the depth processing device through the associated transmission interface for depth processing. This application encapsulates the acquired raw point cloud data from each LiDAR source into standard format point cloud data packets using an encapsulation unit, and then transmits them to the depth processing device through the associated transmission interface. For the depth processing device, there is no need to increase the processing load due to different point cloud data formats; therefore, the processing burden on the depth processing device is effectively reduced, and the processing efficiency is improved.

[0077] To encapsulate the raw point cloud data from various sources into a unified standard format, and to clearly identify the standard point cloud data package and the LiDAR to which it belongs, in one embodiment, such as... Figure 3 As shown, in S202, the raw point cloud data of each channel is encapsulated to obtain the standard point cloud data package corresponding to each raw point cloud data, which further includes:

[0078] S301 encapsulates the raw point cloud data from various sources into a uniformly formatted raw point cloud data package according to preset rules.

[0079] Among them, the preset rules are pre-defined encapsulation rules used to encapsulate the raw point cloud data from various sources into raw point cloud data packets with a uniform format.

[0080] Optionally, in this embodiment, the raw point cloud data from each path is encapsulated into a uniformly formatted raw point cloud data packet according to data transmission rules, for example, encapsulated into the raw point cloud data packet required by the QDMA (queue dma) data transmission rule.

[0081] S302, based on the identification information of the LiDAR that collects each source of raw point cloud data, adds an identity identifier to each source of raw point cloud data packet to obtain the standard point cloud data packet corresponding to each source of raw point cloud data.

[0082] The identification information refers to the information used to identify the LiDAR; the identity identifier is the identifier used to identify the identity of the original point cloud data packet. The identity identifier is associated with the identification information of the LiDAR, and the identification information of the LiDAR can be determined through the identity identifier.

[0083] Optionally, based on the identification information (such as production number, identification number, and custom unique identifier) ​​of the LiDAR that collected each source of raw point cloud data, an identification identifier is added to the header or body of each raw point cloud data packet. This identification identifier can be directly obtained by using the identification information of each LiDAR as the identification identifier of each raw point cloud data packet, or it can be obtained by encoding the identification information of each LiDAR according to preset rules. After adding the identification identifier to each raw point cloud data packet, the standard point cloud data packet corresponding to each source of raw point cloud data is obtained.

[0084] This embodiment can encapsulate each raw point cloud data in a unified format using preset rules, ensuring that the resulting standard point cloud data packets have the same format, effectively reducing the processing burden on the depth processing device. Each standard point cloud data packet has a corresponding identifier, which can be used to determine the identification information of the LiDAR, enabling the depth processing device to quickly identify the source of the standard point cloud data packet and update the corresponding LiDAR point cloud data more quickly.

[0085] In one embodiment, when an identity identifier is added to the original point cloud data packet, in order to achieve fast transmission of each standard point cloud data packet, in S203, the associated transmission interface of each standard point cloud data packet in the depth processing device is determined, further including:

[0086] Based on the identity identifier carried by each standard point cloud data packet, the associated transmission interface of each standard point cloud data packet in the depth processing device is determined.

[0087] In this embodiment, an optional implementation method is as follows: based on the identification information of the LiDAR, the corresponding associated transmission interface is pre-configured, and based on the identity identifier carried by each standard point cloud data packet, the identity identifier of the LiDAR to which it belongs can be determined, thereby determining the associated transmission interface of each standard point cloud data packet in the depth processing device.

[0088] In this embodiment, based on the structural design of the standard point cloud data packet, the associated transmission interface can be determined quickly and flexibly, and the data transmission pressure of the associated transmission interface can be effectively reduced.

[0089] To further alleviate the processing burden on deep processing devices, in one embodiment, such as Figure 4 As shown, in S202, the raw point cloud data from each path is encapsulated and processed, including:

[0090] S401, preprocess the raw point cloud data from each path to obtain the target point cloud data; the preprocessing includes: angle compensation processing and / or coordinate system transformation processing.

[0091] Among them, preprocessing is the process of processing the original point cloud data before encapsulation; angle compensation processing refers to calibrating the angle of the original point cloud data to make it closer to the actual scene; coordinate system transformation processing refers to transforming the coordinate system of the original point cloud data; and target point cloud data is the preprocessed original point cloud data.

[0092] In this embodiment, one optional implementation method is as follows: The horizontal angle and / or pitch angle of the point cloud in each source of raw point cloud data are obtained, along with the influencing factors affecting the horizontal angle and / or pitch angle. Based on these influencing factors, the deviation between the horizontal angle and / or pitch angle of the point cloud in the raw point cloud data and the actual horizontal angle and / or pitch angle of the point cloud is determined. Angle compensation processing is then performed on the horizontal angle and / or pitch angle of the point cloud in the raw point cloud data according to the deviation value to obtain the target point cloud data. Here, the influencing factors refer to the factors that cause the deviation between the horizontal angle and / or pitch angle of the point cloud in the raw point cloud data and the actual horizontal angle and / or pitch angle of the point cloud.

[0093] Another optional implementation involves performing coordinate system transformation on the raw point cloud data, including but not limited to at least one of the following: transformation from spherical coordinates to vehicle coordinates and transformation from spherical coordinates to local geographic coordinates; all raw point cloud data are in spherical coordinates. The transformation from spherical coordinates to vehicle coordinates is as follows: a reference point on the vehicle body (e.g., chassis center point, wheel center point, etc.) is selected as the origin, the vehicle's front direction is the X-axis, the vertical upward direction of the X-axis is the Z-axis, and the leftward direction of the vehicle is the Y-axis, establishing a three-dimensional coordinate system. Based on the installation position and attitude of the LiDAR and the raw point cloud data, the spherical coordinate system of the raw point cloud data is converted to the vehicle coordinate system. Alternatively, the converted vehicle coordinate system can be converted to a local geographic coordinate system to obtain the vehicle's attitude information. Based on the vehicle's attitude information, the tilt angle between the vehicle and the local geographic location is determined. The local geographic coordinate system is then obtained by converting the vehicle's tilt angle with the local geographic location and the converted vehicle coordinate system. The vehicle's attitude information can be obtained through inertial sensors.

[0094] In this embodiment, preferably, the point cloud data after the transformation of the three coordinate systems are used together as the target point cloud data.

[0095] Another optional implementation is to perform angle compensation processing and coordinate system transformation processing on each original point cloud data to obtain target point cloud data. It should be noted that the processing order of angle compensation processing and coordinate system transformation processing in this implementation is not important. Preferably, angle compensation processing is performed first, followed by coordinate system transformation processing. For specific processing methods, please refer to the above embodiments, which will not be repeated here.

[0096] S402 encapsulates the target point cloud data.

[0097] Optionally, the target point cloud data obtained after preprocessing the raw point cloud data from each source can be packaged into a standard point cloud data package in a unified format according to preset rules.

[0098] In this embodiment, before encapsulating the original unit data, preprocessing operations can be performed to perform angle compensation processing and / or coordinate system transformation processing on the original point cloud data. This not only makes the point cloud data obtained by the depth processing device more accurate, but also effectively reduces the processing burden of the depth processing device.

[0099] To compensate for the horizontal angle of the raw point cloud data, based on the above embodiment, in S401, angle compensation processing is performed on each path of the raw point cloud data, further including:

[0100] Based on the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value in each source of raw point cloud data, horizontal angle compensation processing is performed on each source of raw point cloud data.

[0101] Among them, the point cloud horizontal angle value refers to the horizontal angle of the collected point cloud data, the lidar horizontal angle offset value refers to the horizontal angle offset value of the laser installed inside the lidar, and the emission time compensation value is the compensation value of the laser emission time.

[0102] Optionally, the compensated horizontal angle is obtained by summing the point cloud horizontal angle value, the lidar horizontal angle offset value, and the compensation value at the emission time.

[0103] Based on the above embodiments, when the specific value of the compensation for the emission time is unknown, such as Figure 5 As shown, the method for obtaining the emission time compensation value includes:

[0104] S501, obtains the emission time deviation value and rotation speed of the lidar.

[0105] Among them, the emission timing deviation value refers to the deviation value of the emission timing of the laser inside the lidar, and the rotation speed refers to the rotation speed of the lidar.

[0106] Optionally, the emission timing deviation and rotation speed of the lidar can be obtained by detection or from the manufacturer.

[0107] S502, determine the light emission time compensation value based on the light emission time deviation value and the rotation speed.

[0108] Optionally, the compensation value for the emission time is obtained by multiplying the emission time deviation value and the rotation speed.

[0109] This embodiment provides a preferred method for determining the emission time compensation value when the specific magnitude of the emission time compensation value is unknown.

[0110] If the acquired data from at least one lidar is a raw point cloud data packet, in order to extract the raw point cloud data from the raw point cloud data packet, in one embodiment, such as Figure 6 As shown, in S201, acquiring raw point cloud data from at least one LiDAR source further includes:

[0111] S601, acquire the raw point cloud data packet of at least one LiDAR.

[0112] The raw point cloud data packet is the raw point cloud data that has not undergone frame parsing.

[0113] Optionally, multiple acquisition channels can be configured, each capable of acquiring the raw point cloud data packet of one LiDAR. Through these multiple acquisition channels, the raw point cloud data packet of at least one LiDAR can be acquired from at least one LiDAR using a wired or wireless network. This application can connect to LiDARs from various manufacturers via these multiple acquisition channels.

[0114] S602, perform frame parsing on the raw point cloud data packets of at least one LiDAR to obtain the raw point cloud data of at least one LiDAR.

[0115] Frame parsing refers to parsing the acquired point cloud data packets in the form of data frames.

[0116] Optionally, first determine the data transmission protocol, and then perform frame parsing on the raw point cloud data packets of at least one LiDAR source according to the data transmission protocol to obtain the raw point cloud data of at least one LiDAR source. For example, data transmitted using the UDP data transmission protocol needs to be parsed according to the UDP protocol rules.

[0117] In this embodiment, point cloud data in different formats output by lidar from different manufacturers can be flexibly processed. It can process point cloud data directly or in the form of data packets.

[0118] Optionally, based on the above embodiments, such as Figure 7 , Figure 8 As shown in the figure, this embodiment provides an optional method for point cloud data processing, including the following steps:

[0119] S701, acquire the raw point cloud data packet of at least one LiDAR.

[0120] S702 performs frame parsing on the raw point cloud data packets of at least one LiDAR to obtain the raw point cloud data of at least one LiDAR.

[0121] S703 acquires the emission timing deviation value and rotation speed of the lidar, and determines the emission timing compensation value based on the emission timing deviation value and rotation speed.

[0122] S704 performs horizontal angle compensation processing on each source of raw point cloud data based on the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value.

[0123] S705 performs coordinate system transformation on the original point cloud data after horizontal angle compensation processing to obtain the target point cloud data.

[0124] Optionally, coordinate transformation processing includes at least one of the following: transformation from spherical coordinate system to Eulerian coordinate system; transformation from spherical coordinate system to vehicle body coordinate system; and transformation from spherical coordinate system to local geographic coordinate system; all original point cloud data are in spherical coordinate system.

[0125] S706 encapsulates the target point cloud data into a target point cloud data packet with a uniform format according to preset rules.

[0126] S707 adds an identity identifier to each target point cloud data packet based on the identification information of the LiDAR that collects each source of raw point cloud data, thereby obtaining the standard point cloud data packet corresponding to each source of raw point cloud data.

[0127] S708 determines the associated transmission interface of each standard point cloud data packet in the depth processing device.

[0128] The S709 sends standard point cloud data packets to the depth processing device via the associated transmission interface, so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0129] This application encapsulates the raw point cloud data from various LiDAR sources into a standard-format point cloud data packet using an encapsulation unit. This packet is then transmitted to the depth processing device via a connection transmission interface. This eliminates the need for additional processing due to different point cloud data formats, effectively reducing the processing burden on the depth processing device and improving its efficiency. Furthermore, this application can preprocess the point cloud data, making the obtained point cloud data more accurate and further reducing the processing burden on the depth processing device.

[0130] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0131] Based on the same inventive concept, this application also provides a point cloud data processing apparatus for implementing the point cloud data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations of the point cloud data processing apparatus embodiment provided below can be found in the limitations of the point cloud data processing method described above, and will not be repeated here.

[0132] In one embodiment, such as Figure 9 As shown, a point cloud data processing device 1 is provided, comprising: an acquisition module 10, an encapsulation module 20, an interface determination module 30, and a transmission module 40, wherein:

[0133] The acquisition module 10 is used to acquire raw point cloud data from at least one LiDAR source.

[0134] The encapsulation module 20 is used to encapsulate the raw point cloud data from each channel to obtain the standard point cloud data packets corresponding to each channel of raw point cloud data.

[0135] Interface determination module 30 is used to determine the associated transmission interface of each standard point cloud data packet in the depth processing device;

[0136] The sending module 40 sends each standard point cloud data packet to the depth processing device through the associated transmission interface, so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0137] In one embodiment, in order to encapsulate the raw point cloud data from various sources into a unified standard format, and to clearly define the identity information of the standard point cloud data packet and the LiDAR to which it belongs, such as... Figure 10 As shown above, Figure 9 Based on this, the packaging module 20 further includes:

[0138] The first encapsulation unit 201 is used to encapsulate the raw point cloud data from each source into a raw point cloud data packet with a uniform format according to preset rules.

[0139] The identification adding unit 202 adds an identity identifier to each raw point cloud data packet based on the identification information of the LiDAR that collects each raw point cloud data, thereby obtaining the standard point cloud data packet corresponding to each raw point cloud data.

[0140] In one embodiment, the upper Figure 9 In this context, the interface determination module 30 is specifically used to determine the associated transmission interface of each standard point cloud data packet in the depth processing device based on the identity identifier carried by each standard point cloud data packet.

[0141] In one embodiment, to improve the accuracy of the point cloud data sent to the depth processing device, such as Figure 11 As shown above, Figure 9 Based on this, the packaging module 20 further includes:

[0142] The preprocessing unit 203 is used to preprocess the raw point cloud data from each path to obtain the target point cloud data; wherein, the preprocessing includes: angle compensation processing and / or coordinate system transformation processing;

[0143] The second encapsulation unit 204 is used to encapsulate the target point cloud data.

[0144] In one embodiment, in order to perform angle compensation and coordinate system transformation processing on the raw point cloud data, the preprocessing unit 203 is further configured to perform horizontal angle compensation on each raw point cloud data according to the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value in each raw point cloud data; and to perform the coordinate system transformation processing, including at least one of: spherical coordinate system to Euler coordinate system; spherical coordinate system to vehicle body coordinate system; and spherical coordinate system to local geographic coordinate system; wherein each raw point cloud data is in a spherical coordinate system.

[0145] In one embodiment, the angle compensation unit is further configured to acquire the emission timing deviation value and rotation speed of the lidar; and determine the emission timing compensation value based on the emission timing deviation value and the rotation speed.

[0146] In one embodiment, to improve the flexibility of point cloud data format processing, such as Figure 12 As shown above, Figure 9 Based on this, module 10 further includes:

[0147] The data packet acquisition unit 101 acquires at least one raw point cloud data packet from the lidar.

[0148] The frame parsing unit 102 performs frame parsing on the raw point cloud data packets of at least one LiDAR to obtain the raw point cloud data of at least one LiDAR.

[0149] Each module in the aforementioned point cloud data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0150] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores spectral feature data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a point cloud data processing method.

[0151] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0152] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0153] Acquire raw point cloud data from at least one LiDAR source;

[0154] The raw point cloud data from each source is encapsulated to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0155] Determine the associated transmission interface for each standard point cloud data packet in the deep processing device;

[0156] The standard point cloud data packets are sent to the depth processing device via the associated transmission interface so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0157] In one embodiment, when the processor executes the computer program, it further performs the following steps: encapsulating the raw point cloud data from each source to obtain a standard point cloud data packet corresponding to each source of raw point cloud data, including:

[0158] The raw point cloud data from each source is packaged into a uniformly formatted raw point cloud data package according to preset rules.

[0159] Based on the identification information of the LiDAR that collected the raw point cloud data from each channel, an identity identifier is added to each raw point cloud data packet to obtain the standard point cloud data packet corresponding to each raw point cloud data.

[0160] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the associated transmission interface of each standard point cloud data packet in the depth processing device, including:

[0161] Based on the identity identifier carried by each standard point cloud data packet, the associated transmission interface of each standard point cloud data packet in the depth processing device is determined.

[0162] In one embodiment, when the processor executes the computer program, it further performs the following steps: encapsulating and processing the raw point cloud data from each source, including:

[0163] The raw point cloud data from each path is preprocessed to obtain the target point cloud data; the preprocessing includes: angle compensation processing and / or coordinate system transformation processing;

[0164] The target point cloud data is encapsulated and processed.

[0165] In one embodiment, when the processor executes the computer program, it further performs the following steps: angle compensation processing on each stream of raw point cloud data, including:

[0166] Based on the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value in each source of raw point cloud data, horizontal angle compensation processing is performed on each source of raw point cloud data.

[0167] In one embodiment, the processor, when executing a computer program, further performs the following steps: (further details omitted)

[0168] Obtain the emission timing deviation and rotation speed of the lidar;

[0169] The emission time compensation value is determined based on the emission time deviation value and the rotation speed.

[0170] In one embodiment, when the processor executes the computer program, it further performs the following steps: the coordinate system transformation process includes: transformation from spherical coordinates to Euler coordinates;

[0171] At least one of the following transformations: from spherical coordinate system to vehicle coordinate system and from spherical coordinate system to local geographic coordinate system; all original point cloud data are in spherical coordinate system.

[0172] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring raw point cloud data from at least one LiDAR source, including:

[0173] Obtain the raw point cloud data packet from at least one LiDAR source;

[0174] Frame parsing is performed on the raw point cloud data packets of at least one LiDAR to obtain the raw point cloud data of at least one LiDAR.

[0175] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0176] Acquire raw point cloud data from at least one LiDAR source;

[0177] The raw point cloud data from each source is encapsulated to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0178] Determine the associated transmission interface for each standard point cloud data packet in the deep processing device;

[0179] The standard point cloud data packets are sent to the depth processing device via the associated transmission interface so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0180] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: encapsulating the raw point cloud data from each source to obtain a standard point cloud data packet corresponding to each source of raw point cloud data, including:

[0181] The raw point cloud data from each source is packaged into a uniformly formatted raw point cloud data package according to preset rules.

[0182] Based on the identification information of the LiDAR that collected the raw point cloud data from each channel, an identity identifier is added to each raw point cloud data packet to obtain the standard point cloud data packet corresponding to each raw point cloud data.

[0183] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: determining the associated transmission interface of each standard point cloud data packet in the depth processing device, including:

[0184] Based on the identity identifier carried by each standard point cloud data packet, the associated transmission interface of each standard point cloud data packet in the depth processing device is determined.

[0185] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: encapsulating and processing the raw point cloud data from each source, including:

[0186] The raw point cloud data from each path is preprocessed to obtain the target point cloud data; the preprocessing includes: angle compensation processing and / or coordinate system transformation processing;

[0187] The target point cloud data is encapsulated and processed.

[0188] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: performing angle compensation processing on the raw point cloud data from each path, including:

[0189] Based on the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value in each source of raw point cloud data, horizontal angle compensation processing is performed on each source of raw point cloud data.

[0190] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: It also includes:

[0191] Obtain the emission timing deviation and rotation speed of the lidar;

[0192] The emission time compensation value is determined based on the emission time deviation value and the rotation speed.

[0193] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: the coordinate system transformation process includes: a transformation from a spherical coordinate system to an Euler coordinate system;

[0194] At least one of the following transformations: from spherical coordinate system to vehicle coordinate system and from spherical coordinate system to local geographic coordinate system; all original point cloud data are in spherical coordinate system.

[0195] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: acquiring raw point cloud data from at least one LiDAR source, including:

[0196] Obtain the raw point cloud data packet from at least one LiDAR source;

[0197] Frame parsing is performed on the raw point cloud data packets of at least one LiDAR to obtain the raw point cloud data of at least one LiDAR.

[0198] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0199] Acquire raw point cloud data from at least one LiDAR source;

[0200] The raw point cloud data from each source is encapsulated to obtain the standard point cloud data package corresponding to each source of raw point cloud data.

[0201] Determine the associated transmission interface for each standard point cloud data packet in the deep processing device;

[0202] The standard point cloud data packets are sent to the depth processing device via the associated transmission interface so that the depth processing device can perform depth processing on each standard point cloud data packet.

[0203] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: encapsulating the raw point cloud data from each source to obtain a standard point cloud data packet corresponding to each source of raw point cloud data, including:

[0204] The raw point cloud data from each source is packaged into a uniformly formatted raw point cloud data package according to preset rules.

[0205] Based on the identification information of the LiDAR that collected the raw point cloud data from each channel, an identity identifier is added to each raw point cloud data packet to obtain the standard point cloud data packet corresponding to each raw point cloud data.

[0206] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: determining the associated transmission interface of each standard point cloud data packet in the depth processing device, including:

[0207] Based on the identity identifier carried by each standard point cloud data packet, the associated transmission interface of each standard point cloud data packet in the depth processing device is determined.

[0208] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: encapsulating and processing the raw point cloud data from each source, including:

[0209] The raw point cloud data from each path is preprocessed to obtain the target point cloud data; the preprocessing includes: angle compensation processing and / or coordinate system transformation processing;

[0210] The target point cloud data is encapsulated and processed.

[0211] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: performing angle compensation processing on the raw point cloud data from each path, including:

[0212] Based on the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value in each source of raw point cloud data, horizontal angle compensation processing is performed on each source of raw point cloud data.

[0213] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: It also includes:

[0214] Obtain the emission timing deviation and rotation speed of the lidar;

[0215] The emission time compensation value is determined based on the emission time deviation value and the rotation speed.

[0216] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: the coordinate system transformation process includes: a transformation from a spherical coordinate system to an Euler coordinate system;

[0217] At least one of the following transformations: from spherical coordinate system to vehicle coordinate system and from spherical coordinate system to local geographic coordinate system; all original point cloud data are in spherical coordinate system.

[0218] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: acquiring raw point cloud data from at least one LiDAR source, including:

[0219] Obtain the raw point cloud data packet from at least one LiDAR source;

[0220] Frame parsing is performed on the raw point cloud data packets of at least one LiDAR to obtain the raw point cloud data of at least one LiDAR.

[0221] It should be noted that the user information (including but not limited to user device information) and data (including but not limited to data used for analysis, data stored, data displayed) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0222] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0223] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0224] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A point cloud data processing method, characterized in that, The method includes: Acquire raw point cloud data from at least one LiDAR source; The raw point cloud data from each source is processed to obtain target point cloud data, which is then packaged into a uniformly formatted target point cloud data package according to preset rules. Based on the identification information of the LiDAR that collected the raw point cloud data, an identity identifier is added to each target point cloud data package to obtain a standard point cloud data package corresponding to each source of raw point cloud data. The preprocessing includes coordinate system transformation. This transformation includes: selecting a reference point on the vehicle as the origin, establishing a three-dimensional coordinate system with the vehicle's front direction as the X-axis, the vertical direction upwards along the X-axis as the Z-axis, and the leftward direction of the vehicle as the Y-axis; converting the three-dimensional coordinate system of the raw point cloud data into a vehicle body coordinate system based on the LiDAR's installation position and orientation; acquiring the vehicle's attitude information; determining the vehicle's tilt angle relative to the local geographic area based on the attitude information; and obtaining the local geographic coordinate system based on the tilt angle and the vehicle body coordinate system. Based on the identity identifier and preset interface mapping relationship carried by each standard point cloud data packet, determine the associated transmission interface of each standard point cloud data packet in the deep processing device; Through the associated transmission interface, each standard point cloud data packet is sent to the depth processing device for depth processing.

2. The method according to claim 1, characterized in that, The preprocessing also includes angle compensation processing.

3. The method according to claim 2, characterized in that, Angle compensation processing is performed on the raw point cloud data from each source, including: Based on the point cloud horizontal angle value, the lidar horizontal angle offset value, and the emission time compensation value in each source of raw point cloud data, horizontal angle compensation processing is performed on each source of raw point cloud data.

4. The method according to claim 3, characterized in that, Also includes: Obtain the emission timing deviation and rotation speed of the lidar; The emission time compensation value is determined based on the emission time deviation value and the rotation speed.

5. The method according to claim 2, characterized in that, The coordinate system transformation process includes at least one of the following: transformation from spherical coordinate system to vehicle coordinate system and transformation from spherical coordinate system to local geographic coordinate system; the original point cloud data of each path are in spherical coordinate system.

6. The method according to claim 1, characterized in that, The acquisition of raw point cloud data from at least one LiDAR source includes: Obtain the raw point cloud data packet from at least one LiDAR source; The raw point cloud data of the at least one LiDAR is obtained by frame parsing of the raw point cloud data packet of the at least one LiDAR.

7. A point cloud data processing device, characterized in that, include: The acquisition module acquires raw point cloud data from at least one LiDAR source. The encapsulation module processes the raw point cloud data from each source to obtain target point cloud data, and encapsulates it into a target point cloud data packet with a uniform format according to preset rules. Based on the identification information of the LiDAR that collected the raw point cloud data, it adds an identity identifier to each target point cloud data packet, obtaining a standard point cloud data packet corresponding to each source of raw point cloud data. The preprocessing includes coordinate system transformation. This transformation includes: selecting a reference point on the vehicle as the origin, establishing a three-dimensional coordinate system with the vehicle's front direction as the X-axis, the vertical direction upwards along the X-axis as the Z-axis, and the leftward direction of the vehicle as the Y-axis; converting the three-dimensional coordinate system of the raw point cloud data into a vehicle coordinate system based on the LiDAR's installation position and orientation; acquiring the vehicle's attitude information; determining the vehicle's tilt angle relative to the local geographic area based on the attitude information; and obtaining the local geographic coordinate system based on the tilt angle and the vehicle coordinate system. The interface determination module determines the associated transmission interface of each standard point cloud data packet in the deep processing device based on the identity identifier carried by each standard point cloud data packet and the preset interface mapping relationship. The sending module sends each standard point cloud data packet to the depth processing device through the associated transmission interface, so that the depth processing device can perform depth processing on each standard point cloud data packet.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the point cloud data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the point cloud data processing method according to any one of claims 1 to 6.

Images