Point cloud data processing method, device, system and edge computing unit

Abstract

The present application discloses a method, device, system and edge computing unit for processing point cloud data. The present application determines whether a mobile target enters the area of ​​interest of a roadside point cloud acquisition device with the help of road condition data collected by auxiliary detection equipment. If so, the processing of the point cloud data returned by the point cloud acquisition device is triggered, or the processing of the point cloud data returned by the cloud acquisition device and the image returned by the image acquisition device is triggered. Through the data processing, the point cloud data or the point cloud data and the image are transmitted to the application end. When the mobile target is not in the area of ​​interest of the point cloud acquisition device, the data collected by the directly corresponding acquisition device is discarded. Thus, the present application filters out the duplicate data collected by the point cloud acquisition device during the period when no mobile target passes by, and can realize accurate and efficient collection of sufficient point cloud data as a training set based on the V2X roadside device, while reducing the requirements of the roadside device for storage capacity and upload bandwidth.

Description

Technical Field

[0001] The present application belongs to the field of V2X (Vehicle to Everything / Vehicle to X, vehicle wireless communication technology) roadside equipment target recognition technology, and in particular, relates to a method, device, system and edge computing unit for point cloud data. Background Art

[0002] With the development of sensor technology, artificial intelligence technology and 5G (5th Generation Mobile Communication Technology), intelligent driving technology has also made great progress, but the perception ability of a single vehicle is limited, which seriously restricts the large-scale application of intelligent driving technology. If the vehicle can receive information about other vehicles and pedestrians in advance, it can greatly reduce the occurrence of accidents. V2X technology can achieve coordination in detection and perception with single vehicle intelligence, thereby greatly improving the perception ability of the vehicle, and therefore is increasingly attracting the attention of researchers in the autonomous driving industry.

[0003] To realize the road target perception of V2X, it is necessary to first obtain data through the sensors (such as laser radar) on the V2X roadside equipment, and then use the embedded computing unit to identify vehicles, pedestrians and other targets, and transmit the identification results to the driving vehicle through the network, so as to expand the intelligent perception ability of the single vehicle. However, the applicant found that the height and angle of the laser radar installed on the street lamp and signal lamp are different from the laser radar on the car. The point cloud data set collected by the existing on-board laser radar cannot be directly used for algorithm training of V2X roadside equipment. In addition, in some specific environments, such as mining areas and ports, point cloud data needs to be collected separately. However, the current research on V2X-based multi-sensor perception-fusion-decision-making autonomous driving technology is still in its infancy, and there is no recognized mature roadside equipment laser radar point cloud data training set available. Therefore, how to accurately and efficiently collect enough point cloud data as a training set by V2X roadside equipment has become a technical problem that needs to be solved in this field. Summary of the invention

[0004] In view of this, the present application provides a point cloud data processing method, device, system and edge computing unit, which are used to accurately and efficiently collect sufficient point cloud data as a training set based on V2X roadside equipment, while reducing the roadside equipment's requirements for storage capacity and upload bandwidth.

[0005] The specific technical solutions are as follows:

[0006] A method for processing point cloud data, comprising:

[0007] Acquiring road condition data collected by a predetermined auxiliary detection device within a detection range, wherein the auxiliary detection device synchronously collects data with a point cloud acquisition device on the road side, or synchronously collects data with both the point cloud acquisition device and the image acquisition device on the road side;

[0008] Processing the road condition data collected by the auxiliary detection device, and determining whether a moving target enters the area of ​​interest of the point cloud acquisition device, wherein the detection range of the auxiliary detection device at least covers the area of ​​interest;

[0009] In the case where it is determined that a mobile target has entered the area of ​​interest, when the auxiliary detection device and the point cloud acquisition device on the road side synchronously perform data acquisition, data processing of the point cloud data returned by the point cloud acquisition device is triggered; when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side synchronously perform data acquisition, data processing of the point cloud data returned by the point cloud acquisition device and processing of the image returned by the image acquisition device are triggered;

[0010] When the mobile target is not in the area of ​​interest, when the auxiliary detection device and the point cloud acquisition device on the road side perform data acquisition synchronously, the point cloud data returned by the point cloud acquisition device will be discarded; when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side perform data acquisition synchronously, the point cloud data returned by the point cloud acquisition device and the image returned by the image acquisition device will be discarded.

[0011] Optionally, the method further includes:

[0012] The image acquisition device performs data acquisition according to a first cycle, wherein the first cycle is a generation cycle of a point cloud frame of point cloud data acquired by the point cloud acquisition device.

[0013] Optionally, the method comprises:

[0014] When triggering data processing of the point cloud data returned by the point cloud acquisition device, or triggering data processing of the point cloud data returned by the point cloud acquisition device and processing of the image returned by the image acquisition device, updating the countdown duration of the timer to a first duration, and continuously updating the countdown duration of the timer to the first duration while the moving target is within the detection range of the auxiliary detection device; wherein the first duration is an estimated time required for the moving target to pass through the area of ​​interest;

[0015] When it is detected that the timer countdown reaches zero, it is determined that the moving target is outside the range of interest, and the point cloud data returned by the point cloud acquisition device is discarded, or the point cloud data returned by the point cloud acquisition device and the image returned by the image acquisition device are discarded.

[0016] Optionally, data processing of the point cloud data returned by the point cloud acquisition device includes:

[0017] Each frame of point cloud data is processed in a predetermined manner, including time stamping, compression and uploading.

[0018] Optionally, when the auxiliary detection device collects data synchronously with the point cloud acquisition device and the image acquisition device on the roadside, the detection range of the image acquisition device is consistent with the range of the area of ​​interest of the point cloud acquisition device.

[0019] Optionally, data processing of the image transmitted back by the image acquisition device includes:

[0020] Each image sent back by the image acquisition device is time-stamped, compressed and uploaded.

[0021] Optionally, the auxiliary detection device is a millimeter wave radar, the point cloud acquisition device is a laser radar, and the image acquisition device is a camera.

[0022] A point cloud data processing device, characterized by comprising:

[0023] An acquisition module, used to acquire road condition data collected by a predetermined auxiliary detection device within a detection range, wherein the auxiliary detection device synchronously collects data with a point cloud acquisition device on the road side, or synchronously collects data with both the point cloud acquisition device and the image acquisition device on the road side;

[0024] A determination module, used to process the road condition data collected by the auxiliary detection device and determine whether a mobile target enters the area of ​​interest of the point cloud acquisition device, wherein the detection range of the auxiliary detection device at least covers the area of ​​interest;

[0025] A first trigger module is used for, when it is determined that a mobile target has entered the area of ​​interest, triggering data processing of point cloud data returned by the point cloud acquisition device when the auxiliary detection device and the point cloud acquisition device on the road side are synchronously performing data acquisition, and triggering data processing of point cloud data returned by the point cloud acquisition device and processing of images returned by the image acquisition device when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side are synchronously performing data acquisition;

[0026] The second trigger module is used to discard the point cloud data returned by the point cloud acquisition device when the auxiliary detection device and the point cloud acquisition device on the road side are synchronously collecting data when the mobile target is not in the area of ​​interest, and to discard the point cloud data returned by the point cloud acquisition device and the image acquisition device when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side are synchronously collecting data.

[0027] An edge computing unit, characterized by comprising:

[0028] Memory for storing computer instruction sets;

[0029] A processor is used to implement the point cloud data processing method as described in any one of the above items by executing the instruction set in the memory.

[0030] A point cloud data processing system, comprising: an auxiliary detection device for assisting in detecting a moving target, a point cloud acquisition device for acquiring point cloud data, and an edge computing unit as described above;

[0031] The detection range of the auxiliary detection device at least covers the area of ​​interest of the point cloud acquisition device, and the auxiliary detection device at least performs data acquisition synchronously with the point cloud acquisition device.

[0032] It can be seen from the above technical solutions that the point cloud data processing method, device, system and edge computing unit disclosed in this application have the following technical effects compared with the existing technology:

[0033] The present application utilizes auxiliary detection equipment to collect road condition data within the detection range, and determines whether a mobile target enters the area of ​​interest of the roadside point cloud collection device based on the road condition data. When a mobile target enters the area of ​​interest, data processing of the point cloud data collected and returned by the point cloud collection device is triggered, or data processing of the point cloud data returned by the cloud collection device and the image returned by the image collection device is triggered. Through the data processing, the point cloud data or the point cloud data and the image are transmitted to the application end. When the mobile target is not in the area of ​​interest of the point cloud collection device, the point cloud data collected by the point cloud collection device is directly discarded, or the point cloud data collected by the point cloud collection device and the image collected by the image collection device are discarded.

[0034] The present application automatically triggers the roadside equipment to process the point cloud data based on mobile target detection, thereby filtering out the duplicate data collected by the point cloud collection equipment during the period when no mobile target passes by. This can achieve accurate and efficient collection of sufficient point cloud data as a training set based on the V2X roadside equipment. Furthermore, since the point cloud data when no mobile target passes by is directly discarded, the requirements of the roadside equipment for storage capacity and upload bandwidth are also reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A flowchart of the method for processing point cloud data provided in this application;

[0037] Figure 2 A schematic diagram of a layout scheme of the roadside data collection equipment provided in this application;

[0038] Figure 3 A schematic diagram of another arrangement scheme of the roadside data collection equipment provided in this application;

[0039] Figure 4 A workflow diagram of an application system of the present application method provided by the present application;

[0040] Figure 5 A structural diagram of the point cloud data processing device provided in this application;

[0041] Figure 6 This is a structural diagram of the edge computing unit provided in this application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] The applicant found that the current research on V2X-based multi-sensor perception-fusion-decision-making autonomous driving technology is still in its infancy, and there is no recognized mature roadside equipment lidar training set available. The amount of data generated by roadside point cloud acquisition equipment such as lidar is very large. If all of them are saved and uploaded indiscriminately in the same way as on-board lidar (for example, uploaded to a cloud server for point cloud annotation and model training applications), it will be a great challenge to local storage capacity and upload bandwidth, and there will be at least the following problems:

[0044] 1) On-board LiDAR data collection cannot be used for algorithm training of V2X roadside equipment;

[0045] 2) All collected LiDAR point cloud data is stored locally, which requires a huge amount of local storage space;

[0046] 3) All collected LiDAR point cloud data are uploaded to the application end such as a cloud server, but the network bandwidth cannot meet the requirements.

[0047] Therefore, it is particularly important to propose a dedicated collection and processing solution for the point cloud data collection of the roadside equipment laser radar. Therefore, the present application provides a point cloud data processing method, device, system and edge computing unit to solve the point cloud data collection and processing problems of the roadside equipment laser radar.

[0048] The point cloud data processing method provided in this application can be applied to an edge computing unit arranged on the roadside, that is, the edge computing unit is used as the execution subject of the method of this application, wherein the edge computing unit can be but is not limited to an industrial computer or an embedded computing device. Figure 1 , the point cloud data processing method of the present application includes at least the following processing steps:

[0049] Step 101: Acquire road condition data collected by a predetermined auxiliary detection device within a detection range, wherein the auxiliary detection device synchronously collects data with a point cloud acquisition device on the road side, or synchronously collects data with both the point cloud acquisition device and the image acquisition device on the road side.

[0050] The point cloud acquisition device is used to collect road point cloud data. Optionally, the point cloud acquisition device can be a laser radar installed on the side of the road.

[0051] The applicant found that, unlike the vehicle-mounted laser radar, the roadside equipment laser radar is in a static state. When there are no moving targets such as vehicles or pedestrians passing by, the data collected is completely repeated and there is no need to save and upload. Therefore, the embodiment of the present application introduces auxiliary detection equipment to realize moving target detection, and only when a moving target is detected, the data processing such as saving and uploading of the point cloud data collected by the roadside equipment laser radar is triggered.

[0052] The auxiliary detection device is specifically used to provide the required road condition data for the mobile target detection of the edge computing unit. The auxiliary detection device can be but is not limited to any one of millimeter wave radar, infrared sensor, ultrasonic sensor, image acquisition device, laser sensor, etc., so as to realize mobile target detection by any means such as millimeter wave, infrared, ultrasonic wave, image, laser, etc. The applicant found that the wavelength of millimeter wave is between centimeter wave and light wave. Compared with optical solutions such as infrared and laser, millimeter wave has strong ability to penetrate rain, snow, fog, smoke, and dust, and has a long transmission distance. It has the characteristics of all-weather and all-day, and the performance is stable, and it is not affected by the shape, color and external light of the target object. Therefore, the embodiment of the present application preferably uses millimeter wave radar as an auxiliary detection device, and realizes mobile target detection by millimeter wave detection means, which correspondingly makes up for the use scenarios that other sensors such as infrared, laser, ultrasonic wave, camera, etc. do not have.

[0053] In the implementation, a possible arrangement scheme of the roadside collection equipment is as follows: Figure 2 As shown, the laser radar is arranged on a street lamp pole or a signal lamp pole beside the road. The mechanical rotating laser radar can cover the surrounding 360° range. The millimeter wave radar is arranged according to the range of interest of the laser radar, so that when the moving target enters the range of interest, it can be detected by the millimeter wave radar, thereby triggering the processing of the data collected by the laser radar.

[0054] Regarding the implementation method of synchronous data collection between the auxiliary detection equipment and the point cloud collection equipment and image collection equipment on the road side, in addition to lidar and millimeter-wave radar, this implementation method also arranges image collection equipment, such as cameras, to facilitate the subsequent execution of 3D point cloud annotation work on the application end (such as cloud platform / cloud server).

[0055] See also Figure 3 Specifically, according to the field of view of the image acquisition device (such as a camera) used, multiple image acquisition devices can be arranged on the light poles or signal poles beside the road to achieve the same coverage as the lidar, so that each frame of the laser point cloud collected has a corresponding image.

[0056] The image acquisition device may be configured to acquire video information in a video stream manner or to continuously acquire images at a set time interval, and there is no limitation on this. Preferably, the image acquisition device acquires video information in a first period (ie, Figure 4 T2) in the figure is used for data collection, and the first cycle is the generation cycle of the point cloud frame of the point cloud data collected by the point cloud collection device (such as laser radar).

[0057] After the arrangement of each acquisition device on the roadside is completed through any of the above two implementations, each device is powered on, and time synchronization and timer reset are performed to trigger the laser radar and millimeter wave radar to start working so that the two acquisition devices, the laser radar and millimeter wave radar, can synchronously collect data, or trigger the laser radar, millimeter wave radar and image acquisition device to start working so that the three acquisition devices, the laser radar, millimeter wave radar and image acquisition device, can synchronously collect data. The time synchronization of each device can be, but is not limited to, GPS (Global Positioning System) time synchronization.

[0058] After starting work, the millimeter-wave radar continuously collects road condition data within the detection range, and transmits the collected road condition data back to the corresponding edge computing unit through local connection. The edge computing unit then obtains the road condition data collected by the millimeter-wave radar within the detection range.

[0059] Step 102: Process the road condition data collected by the auxiliary detection device, and determine whether a mobile target enters the area of ​​interest of the point cloud acquisition device, wherein the detection range of the auxiliary detection device at least covers the area of ​​interest.

[0060] Afterwards, the edge computing unit processes the road condition data sent back by the millimeter-wave radar, detects moving targets (such as vehicles or pedestrians) through data processing, and determines whether any moving targets enter the area of ​​interest of the point cloud acquisition device.

[0061] Step 103: When it is determined that a mobile target has entered the region of interest, execute the following step 1031 or step 1032:

[0062] Step 1031: When the auxiliary detection device and the point cloud acquisition device on the road side synchronously perform data acquisition, data processing of the point cloud data returned by the point cloud acquisition device is triggered.

[0063] With regard to the implementation method of synchronously collecting data between the auxiliary detection equipment and the point cloud collection equipment on the road side, if it is determined that a mobile target has entered the area of ​​interest, data processing of the point cloud data collected and returned by the point cloud collection equipment is triggered, so as to transmit the point cloud data to the application end through the data processing; wherein, during the period when the mobile target is not in the area of ​​interest, the point cloud data collected by the point cloud collection equipment is discarded.

[0064] While the millimeter-wave radar continuously collects road condition data within the detection range and transmits the road condition data back to the edge computing unit through a local connection, the roadside equipment laser radar continuously collects point cloud data within the area of ​​interest and transmits the point cloud data back to the edge computing unit through a local connection. Among them, when the mobile target is detected based on the mobile target and is not in the area of ​​interest of the roadside equipment laser radar, the edge computing unit directly discards the point cloud data returned by the laser radar, and does not save or upload it (for example, upload it to the cloud server for data annotation as a point cloud data training set, etc.) and other data processing, so as to save the local storage space of the edge computing unit and reduce bandwidth requirements.

[0065] On the contrary, once a mobile target is identified as entering the area of ​​interest of the roadside equipment laser radar based on mobile target detection, data processing of the point cloud data collected and returned by the laser radar is triggered.

[0066] The edge computing unit processes point cloud data, including but not limited to predetermined solution, timestamp, compression and upload processing for each frame of point cloud data collected and returned by the roadside equipment laser radar.

[0067] Among them, the above-mentioned solution processing mainly includes: extracting the original point cloud data from the TCP / UDP (Transmission Control Protocol / User Datagram Protocol) package according to the protocol formulated by the lidar manufacturer, and converting the original point cloud data into point cloud data in the Cartesian coordinate system.

[0068] The above-mentioned uploading process may be, but is not limited to, uploading the point cloud data processing results to a cloud platform or a cloud server, etc., to facilitate subsequent applications.

[0069] Step 1032: When the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the roadside are synchronously acquiring data, data processing of the point cloud data returned by the point cloud acquisition device and processing of the image returned by the image acquisition device are triggered.

[0070] Regarding the implementation method of synchronous data collection between the auxiliary detection equipment and the point cloud collection equipment and image collection equipment on the road side, when the edge computing unit detects a moving target based on the road condition data sent back by the millimeter-wave radar, it sends a command to trigger the point cloud data collection and processing of the lidar, such as saving, solving, compressing and uploading point cloud data, etc., and at the same time triggers the processing of the image sent back by the image collection.

[0071] Data processing of image data, including but not limited to timestamping, compressing and uploading each image.

[0072] Similarly, when the mobile target is not in the area of ​​interest of the point cloud acquisition device, the image data collected by the image acquisition device is directly discarded, and the local storage and bandwidth of the edge computing unit are not occupied.

[0073] Step 104: When the moving target is not in the region of interest, execute the following step 1041 or step 1042:

[0074] Step 1041: When the auxiliary detection device and the point cloud acquisition device on the road side are synchronously collecting data, the point cloud data returned by the point cloud acquisition device is discarded.

[0075] In order to realize the synchronous data collection between the auxiliary detection equipment and the roadside point cloud collection equipment, this embodiment combines a timer to process the point cloud data returned by the laser radar only when the mobile target is in the area of ​​interest of the roadside equipment laser radar, and directly discard the point cloud data when it is not in the area of ​​interest. Specifically, the timer is a timer integrated in the edge computing unit.

[0076] For details, see Figure 4When triggering data processing of the point cloud data collected by the laser radar, the edge computing unit simultaneously updates the countdown duration of the timer to the first duration (T1), and continuously updates the countdown duration of the timer to the first duration while the mobile target is in the detection range of the millimeter wave radar. The first duration is the estimated time required for the mobile target to pass through the area of ​​interest, and the update period for continuously updating the countdown duration of the timer to the first duration can be the millimeter wave radar detection period.

[0077] In the process of processing point cloud data, the countdown of the timer is detected in real time. Optionally, after completing the processing of each frame of point cloud data, it is detected whether the timer is reset. If it is reset, it is considered that the moving target is outside the range of interest of the laser radar, and the processing of the point cloud data returned by the laser radar is terminated accordingly, and the point cloud data subsequently returned by the laser radar is discarded.

[0078] That is to say, in the embodiment of the present application, the edge computing unit saves the point cloud data returned by the laser radar only when there is a mobile target in the area of ​​interest of the laser radar of the roadside equipment, and performs processing such as decoding, compression, and uploading on it. When there is no mobile target in the area of ​​interest of the laser radar, the point cloud data is discarded and no data processing is performed on it. Subsequently, the point cloud data uploaded by the edge computing unit can be used on the application side to carry out corresponding applications, such as labeling the point cloud data, annotating it with a mobile target identification box (bounding box) and category (such as "pedestrian", "vehicle", etc.) for mobile target identification, etc., as a point cloud data training set.

[0079] Step 1042: When the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the roadside are synchronously collecting data, the point cloud data returned by the point cloud acquisition device and the image returned by the image acquisition device are discarded.

[0080] Regarding the implementation method of synchronously collecting data between the auxiliary detection equipment and the point cloud collection equipment and image collection equipment on the roadside, this embodiment also combines with a timer to realize that the image data sent back by the camera is processed only when the mobile target is in the area of ​​interest of the roadside equipment laser radar, and the image data is directly discarded when it is not in the area of ​​interest.

[0081] Preferably, the control of whether to process the point cloud data and the image data shares the same timer, namely the timer integrated in the edge computing unit.

[0082] During the process of processing point cloud data and image data, the countdown of the timer is detected in real time. Optionally, after completing the processing of each frame of point cloud data and / or image data, it is detected whether the timer is reset. If it is reset, it is considered that the moving target is outside the range of interest of the lidar, and the processing of the returned point cloud data and image data is terminated accordingly, and the subsequent data returned by the lidar and camera are discarded.

[0083] Subsequently, the application end (e.g., cloud platform / cloud server) can combine the point cloud data and image data uploaded by the edge computing unit to carry out point cloud data annotation and training. Specifically, the received point cloud data and image data are matched through timestamps to achieve data alignment, and the point cloud is assisted in annotation with images, such as annotating the mobile target identification box (bounding box) and category (e.g., "pedestrian", "vehicle", etc.) for mobile target identification. The annotated point cloud data can be used for subsequent network training.

[0084] This embodiment further introduces image acquisition equipment to collect road image data on the basis of laser radar and millimeter wave radar, and uses the image data to assist in the annotation of point cloud data, which can further improve the annotation quality of point cloud data and provide a high-quality sample set for subsequent network training. In addition, since the processing of image data is automatically triggered based on mobile target detection, the requirements of roadside equipment for storage capacity and upload bandwidth in image data processing are reduced.

[0085] To summarize, the method of the embodiment of the present application utilizes auxiliary detection equipment to collect road condition data within the detection range, and determines whether a mobile target enters the area of ​​interest of the roadside point cloud acquisition device based on the road condition data. When a mobile target enters the area of ​​interest, data processing of the point cloud data collected and returned by the point cloud acquisition device is triggered, or data processing of the point cloud data returned by the cloud acquisition device and the image returned by the image acquisition device is triggered. Through the data processing, the point cloud data or the point cloud data and the image are transmitted to the application end. When the mobile target is not in the area of ​​interest of the point cloud acquisition device, the point cloud data collected by the point cloud acquisition device is directly discarded, or the point cloud data collected by the point cloud acquisition device and the image collected by the image acquisition device are discarded.

[0086] The present application automatically triggers the roadside equipment to collect and process point cloud data based on mobile target detection, thereby filtering out duplicate data collected by the point cloud collection equipment during periods when no mobile targets pass by. This allows accurate and efficient collection of sufficient point cloud data as a training set based on V2X roadside equipment, and since point cloud data when no mobile targets pass by are directly discarded, the roadside equipment's requirements for storage capacity and upload bandwidth are also reduced.

[0087] Corresponding to the above method, the present application embodiment also discloses a point cloud data processing device, such as Figure 5 As shown, the device comprises:

[0088] An acquisition module 501 is used to acquire road condition data collected by a predetermined auxiliary detection device within a detection range, wherein the auxiliary detection device synchronously collects data with a point cloud acquisition device on the road side, or synchronously collects data with both the point cloud acquisition device and the image acquisition device on the road side;

[0089] A determination module 502 is used to process the road condition data collected by the auxiliary detection device and determine whether a mobile target enters the area of ​​interest of the point cloud acquisition device, wherein the detection range of the auxiliary detection device at least covers the area of ​​interest;

[0090] The first trigger module 503 is used for, when it is determined that a mobile target has entered the area of ​​interest, triggering data processing of point cloud data returned by the point cloud acquisition device when the auxiliary detection device and the point cloud acquisition device on the road side are synchronously performing data acquisition, and triggering data processing of point cloud data returned by the point cloud acquisition device and processing of images returned by the image acquisition device when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side are synchronously performing data acquisition;

[0091] The second trigger module 504 is used for discarding the point cloud data returned by the point cloud acquisition device when the auxiliary detection device and the point cloud acquisition device on the road side are synchronously collecting data when the mobile target is not in the area of ​​interest, and for discarding the point cloud data returned by the point cloud acquisition device and the image acquisition device when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side are synchronously collecting data.

[0092] In one implementation, the image acquisition device performs data acquisition according to a first cycle, wherein the first cycle is a generation cycle of point cloud frames of point cloud data acquired by the point cloud acquisition device.

[0093] In one embodiment, when the first trigger module 503 triggers data processing of point cloud data returned by the point cloud acquisition device, or triggers data processing of point cloud data returned by the point cloud acquisition device and processing of images returned by the image acquisition device, it updates the countdown duration of the timer to a first duration, and continuously updates the countdown duration of the timer to the first duration while the mobile target is in the detection range of the auxiliary detection device; wherein the first duration is an estimated time required for the mobile target to pass through the area of ​​interest; when it is detected that the timer countdown is zero, it is determined that the mobile target is outside the range of interest, and the point cloud data returned by the point cloud acquisition device is discarded, or the point cloud data returned by the point cloud acquisition device and the image returned by the image acquisition device are discarded.

[0094] In one embodiment, the data processing of the point cloud data returned by the point cloud acquisition device includes:

[0095] Each frame of point cloud data is processed in a predetermined manner, including time stamping, compression and uploading.

[0096] In one embodiment, when the auxiliary detection device synchronously collects data with the point cloud acquisition device and the image acquisition device on the roadside, the detection range of the image acquisition device is consistent with the range of the area of ​​interest of the point cloud acquisition device.

[0097] In one embodiment, the data processing of the image transmitted back by the image acquisition device includes:

[0098] Each image sent back by the image acquisition device is time-stamped, compressed and uploaded.

[0099] In one embodiment, the auxiliary detection device is a millimeter wave radar, the point cloud acquisition device is a laser radar, and the image acquisition device is a camera.

[0100] As for the point cloud data processing device disclosed in the embodiment of the present application, since it corresponds to the point cloud data processing method disclosed in the above method embodiment, the description is relatively simple. For the relevant similarities, please refer to the description of the corresponding method embodiment above, which will not be described in detail here.

[0101] The present application also discloses an edge computing unit, which may be, but is not limited to, an industrial computer or an embedded computing device. Figure 6 As shown, the edge computing unit specifically includes:

[0102] Memory 601, used for storing computer instruction sets;

[0103] A set of computer instructions can be implemented in the form of a computer program.

[0104] The processor 602 is used to implement the point cloud data processing method disclosed in any of the above method embodiments by executing a computer instruction set.

[0105] The processor 602 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices.

[0106] In addition, the edge computing unit may also include components such as a communication interface and a communication bus. The memory, processor, and communication interface communicate with each other through the communication bus.

[0107] The communication interface is used for communication between the edge computing unit and other devices (such as cloud servers). The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0108] In addition, an embodiment of the present application also discloses a point cloud data processing system, including: an auxiliary detection device for assisting in detecting a mobile target, a point cloud acquisition device for collecting point cloud data, and an edge computing unit as disclosed in the above embodiment.

[0109] The detection range of the auxiliary detection device at least covers the area of ​​interest of the point cloud acquisition device, and the auxiliary detection device at least performs data acquisition synchronously with the point cloud acquisition device.

[0110] In one embodiment, the above-mentioned system may also include an image acquisition device for acquiring image data; wherein the auxiliary detection device performs data acquisition synchronously with the point cloud acquisition device and the image acquisition device; and the detection range of the image acquisition device is consistent with the range of the area of ​​interest of the point cloud acquisition device.

[0111] The functions and processing procedures of the auxiliary detection equipment, point cloud acquisition equipment, image acquisition equipment and edge computing unit can be found in the description of the above method embodiments and will not be repeated here.

[0112] To summarize, the point cloud data processing method, device, system and edge computing unit disclosed in the embodiments of the present application propose a solution for collecting and processing the point cloud data of the laser radar based on the automatic triggering of the millimeter-wave radar. Compared with the solution of directly processing the point cloud data of the laser radar (saving, solving, compressing and uploading, etc.), the present application can greatly reduce the useless point cloud data collected by the roadside equipment, thereby reducing the requirements of the roadside equipment for storage capacity and upload bandwidth; in addition, the present application solution also introduces the synchronous collection of image data, which can improve the efficiency and accuracy of subsequent 3D point cloud annotation, while reducing the difficulty of annotation.

[0113] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0114] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0115] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0116] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing point cloud data, It is characterized in that include: Acquiring road condition data collected by a predetermined auxiliary detection device within a detection range, wherein the auxiliary detection device synchronously collects data with a point cloud acquisition device on the road side, or synchronously collects data with both the point cloud acquisition device and the image acquisition device on the road side; Processing the road condition data collected by the auxiliary detection device, and determining whether a moving target enters the area of ​​interest of the point cloud acquisition device, wherein the detection range of the auxiliary detection device at least covers the area of ​​interest; In the case where it is determined that a mobile target has entered the area of ​​interest, when the auxiliary detection device and the point cloud acquisition device on the road side synchronously perform data acquisition, data processing of the point cloud data returned by the point cloud acquisition device is triggered; when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side synchronously perform data acquisition, data processing of the point cloud data returned by the point cloud acquisition device and processing of the image returned by the image acquisition device are triggered; When the mobile target is not in the area of ​​interest, when the auxiliary detection device and the point cloud acquisition device on the road side perform data acquisition synchronously, the point cloud data returned by the point cloud acquisition device will be discarded; when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side perform data acquisition synchronously, the point cloud data returned by the point cloud acquisition device and the image returned by the image acquisition device will be discarded.

2. The method according to claim 1, It is characterized in that The method further comprises: The image acquisition device performs data acquisition according to a first cycle, wherein the first cycle is a generation cycle of a point cloud frame of point cloud data acquired by the point cloud acquisition device.

3. The method according to claim 1, It is characterized in that include: When triggering data processing of the point cloud data returned by the point cloud acquisition device, or triggering data processing of the point cloud data returned by the point cloud acquisition device and processing of the image returned by the image acquisition device, updating the countdown duration of the timer to a first duration, and continuously updating the countdown duration of the timer to the first duration while the moving target is within the detection range of the auxiliary detection device; wherein the first duration is an estimated time required for the moving target to pass through the area of ​​interest; When it is detected that the timer countdown reaches zero, it is determined that the moving target is outside the area of ​​interest, and the point cloud data returned by the point cloud acquisition device is discarded, or the point cloud data returned by the point cloud acquisition device and the image returned by the image acquisition device are discarded.

4. The method according to claim 1, It is characterized in that Data processing of point cloud data sent back by point cloud acquisition equipment, including: Each frame of point cloud data is processed in a predetermined manner, including time stamping, compression and uploading.

5. The method according to claim 1, It is characterized in that When the auxiliary detection device collects data synchronously with the point cloud acquisition device and the image acquisition device on the roadside, the detection range of the image acquisition device is consistent with the range of the area of ​​interest of the point cloud acquisition device.

6. The method according to claim 4, It is characterized in that Data processing of images sent back by the image acquisition device, including: Each image sent back by the image acquisition device is time-stamped, compressed and uploaded.

7. The method according to claim 4, It is characterized in that The auxiliary detection device is a millimeter wave radar, the point cloud acquisition device is a laser radar, and the image acquisition device is a camera.

8. A point cloud data processing device, It is characterized in that include: An acquisition module, used to acquire road condition data collected by a predetermined auxiliary detection device within a detection range, wherein the auxiliary detection device synchronously collects data with a point cloud acquisition device on the road side, or synchronously collects data with both the point cloud acquisition device and the image acquisition device on the road side; A determination module, used to process the road condition data collected by the auxiliary detection device and determine whether a mobile target enters the area of ​​interest of the point cloud acquisition device, wherein the detection range of the auxiliary detection device at least covers the area of ​​interest; A first trigger module is used for, when it is determined that a mobile target has entered the area of ​​interest, triggering data processing of point cloud data returned by the point cloud acquisition device when the auxiliary detection device and the point cloud acquisition device on the road side are synchronously performing data acquisition, and triggering data processing of point cloud data returned by the point cloud acquisition device and processing of images returned by the image acquisition device when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side are synchronously performing data acquisition; The second trigger module is used to discard the point cloud data returned by the point cloud acquisition device when the auxiliary detection device and the point cloud acquisition device on the road side are synchronously collecting data when the mobile target is not in the area of ​​interest, and to discard the point cloud data returned by the point cloud acquisition device and the image acquisition device when the auxiliary detection device and the point cloud acquisition device and the image acquisition device on the road side are synchronously collecting data.

9. An edge computing unit, It is characterized in that include: Memory for storing computer instruction sets; A processor, configured to implement the point cloud data processing method according to any one of claims 1 to 7 by executing an instruction set in the memory.

10. A point cloud data processing system, It is characterized in that include: An auxiliary detection device for assisting in detecting a moving target, a point cloud acquisition device for acquiring point cloud data, and an edge computing unit as claimed in claim 9; The detection range of the auxiliary detection device at least covers the area of ​​interest of the point cloud acquisition device, and the auxiliary detection device at least performs data acquisition synchronously with the point cloud acquisition device.

Images