Point cloud data processing method and device, intelligent chip, storage medium and vehicle
By using intelligent chips to perform gridded processing and statistical data updates on point cloud data, the problem of CPU resource consumption during point cloud data preprocessing is solved, improving the stability and target recognition capabilities of the autonomous driving system, and enhancing the accuracy and safety of autonomous driving decisions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2022-12-29
- Publication Date
- 2026-05-29
Smart Images
Figure CN116052113B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence, specifically to the fields of intelligent transportation, computer vision and autonomous driving, and in particular to a method, apparatus, smart chip, storage medium and vehicle for processing point cloud data. Background Technology
[0002] With the development of computer and electronic technologies, computer vision technology has been widely applied in many fields. For example, in autonomous driving technology, computer vision is often used to analyze the road environment and make driving decisions based on the analysis results.
[0003] Specifically, tasks such as object detection, semantic segmentation, and instance segmentation in computer vision are typically required to identify obstacles in the road environment where the vehicle is located. These tasks rely on data detected by sensors, and to facilitate their implementation, the sensor-detected data usually needs to be preprocessed to obtain the input data for the neural network model performing the task. Summary of the Invention
[0004] This disclosure aims to provide a method, apparatus, smart chip, storage medium, and vehicle for processing point cloud data that improves the efficiency of point cloud data processing and the performance of computer vision tasks.
[0005] According to a first aspect of this disclosure, a method for processing point cloud data executed by an intelligent chip is provided, comprising: performing gridding processing on the point cloud data based on width and height values in the point cloud data to obtain target grid cells corresponding to the point cloud data in a predetermined grid; performing gridding processing on the point cloud data based on width and height values in the point cloud data to obtain target grid cells corresponding to the point cloud data in the predetermined grid; and, in response to updating the point cloud statistics data corresponding to the grid cells in the predetermined grid based on a frame of point cloud data, performing a target task of computer vision based on the point cloud statistics data corresponding to the grid cells in the predetermined grid.
[0006] According to a second aspect of this disclosure, a point cloud data processing apparatus integrated into a smart chip is provided, comprising: a meshing processing module, configured to perform meshing processing on the point cloud data based on width and height values in the point cloud data to obtain target mesh cells corresponding to the point cloud data in a predetermined mesh; a statistical data update module, configured to update the point cloud statistical data corresponding to the target mesh cells based on depth and reflection intensity values of the point cloud data; and a target task execution module, configured to, in response to updating the point cloud statistical data corresponding to the mesh cells in the predetermined mesh based on a frame of point cloud data, execute a computer vision target task based on the point cloud statistical data corresponding to the mesh cells in the predetermined mesh.
[0007] According to a third aspect of this disclosure, a smart chip is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the point cloud data processing method provided in this disclosure.
[0008] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform a method for processing point cloud data provided in this disclosure.
[0009] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program / instructions stored on at least one of a readable storage medium and an electronic device, wherein the computer program / instructions, when executed by a processor, implement the point cloud data processing method provided in this disclosure.
[0010] According to a sixth aspect of this disclosure, an autonomous vehicle is provided, including the smart chip provided in this disclosure.
[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0013] Figure 1 This is a schematic diagram illustrating an application scenario of the point cloud data processing method and apparatus according to embodiments of the present disclosure;
[0014] Figure 2 This is a flowchart illustrating a point cloud data processing method according to an embodiment of the present disclosure;
[0015] Figure 3 This is a schematic diagram illustrating the principle of meshing point cloud data according to an embodiment of the present disclosure;
[0016] Figure 4 This is a schematic diagram illustrating the principle of updating point cloud data according to an embodiment of this disclosure;
[0017] Figure 5 This is a logical architecture diagram of a point cloud data processing method according to an embodiment of the present disclosure;
[0018] Figure 6 This is a chip architecture diagram for implementing a point cloud data processing method according to an embodiment of the present disclosure;
[0019] Figure 7 This is a structural block diagram of a point cloud data processing apparatus according to embodiments of the present disclosure; and
[0020] Figure 8 This is a schematic block diagram of a smart chip used to implement the point cloud data processing method of the embodiments of this disclosure. Detailed Implementation
[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0022] In autonomous vehicles, when using neural network models to perform computer vision tasks, data detected by sensors such as LiDAR can be relied upon. For the implementation of computer vision tasks, the main processor (CPU) typically needs to preprocess the point cloud data collected by sensors such as LiDAR. The CPU then sends the preprocessed data to the data processing unit (DPU) that performs the computer vision task. However, the preprocessing of point cloud data usually consumes significant CPU computing resources, which can affect the performance and operational stability of the entire autonomous driving system.
[0023] To address this problem, this disclosure provides a method, apparatus, smart chip, storage medium, and autonomous vehicle for processing point cloud data. The following will first consider... Figure 1 The application scenarios of the methods and apparatus provided in this disclosure are described.
[0024] Figure 1 This is a schematic diagram illustrating an application scenario of the point cloud data processing method and apparatus according to embodiments of this disclosure.
[0025] like Figure 1 As shown, the application scenario 100 of this embodiment may include an autonomous vehicle 110, which can drive on a road. The autonomous vehicle 110 may integrate sensors for perceiving the environment, such as visual cameras and radar-based ranging sensors. The visual cameras may include monocular cameras, binocular stereo cameras, panoramic vision cameras, and / or infrared cameras. The radar-based ranging sensors may include, for example, lidar, millimeter-wave radar, and ultrasonic radar.
[0026] In one embodiment, the autonomous driving system in the autonomous vehicle 110 can perform a computer vision target task, for example, using point cloud data detected by LiDAR. The target task may include, for example, object detection, semantic segmentation, or instance segmentation. To facilitate the implementation of the target task, the point cloud data may be preprocessed to convert it into point cloud statistics that represent the features of the point cloud data. Subsequently, the point cloud statistics are used as input to a neural network model for performing the target task to obtain the execution result. The neural network model for performing the target task may be based on a convolutional neural network, for example, and this disclosure is not limited thereto.
[0027] After completing the target task, the autonomous driving system can determine the environmental information of the autonomous vehicle 110 (such as obstacle position, obstacle speed, etc.) based on the execution result of the target task, and determine the autonomous driving strategy based on the environmental information.
[0028] In one embodiment, monitoring devices 120 may be installed on both sides of the road. These devices can monitor environmental images of the road and transmit the monitored images to electronic devices 130 via a network. The autonomous vehicle 110 may also upload real-time driving information and sensor-detected data to the electronic devices 130, enabling the electronic devices 130 to remotely monitor the autonomous vehicle 110 and intervene remotely in a timely manner when the autonomous vehicle 110 exhibits abnormal driving behavior, thereby improving the driving safety of the autonomous vehicle 110.
[0029] Electronic device 130 can be, for example, various electronic devices with processing capabilities, including but not limited to laptops, desktop computers, and servers. This electronic device 130 can, for example, fuse received data reflecting the environment of the autonomous vehicle 110 to form a panoramic image, enabling comprehensive remote monitoring of the autonomous vehicle 110. Thus, remote monitoring personnel can determine, based on the panoramic image, whether the autonomous vehicle 110 can operate normally and whether remote control of the autonomous vehicle 110 is necessary.
[0030] In one embodiment, the autonomous vehicle 110 may integrate a smart chip with point cloud processing capabilities to process point cloud data acquired by LiDAR and perform target tasks using computer vision. This smart chip may be, for example, a Field Programmable Gate Array (FPGA), and this disclosure does not limit its application to this type of chip.
[0031] It should be noted that the point cloud data processing method provided in this disclosure can be executed by an autonomous vehicle 110, specifically by a smart chip installed in the autonomous vehicle 110. Correspondingly, the point cloud data processing device provided in this disclosure can be installed in the autonomous vehicle 110, specifically in a smart chip installed in the autonomous vehicle 110.
[0032] It should be understood that Figure 1 The number and type of autonomous vehicles 110, monitoring equipment 120, and electronic devices 130 shown are merely illustrative. Depending on the implementation requirements, any number and type of autonomous vehicles 110, monitoring equipment 120, and electronic devices 130 may be included.
[0033] The following will combine Figures 2-6 The point cloud data processing method provided in this disclosure is described in detail.
[0034] Figure 2 This is a flowchart illustrating a point cloud data processing method according to an embodiment of the present disclosure.
[0035] like Figure 2 As shown, the point cloud data processing method 200 of this embodiment may include operations S210 to S230. This point cloud data processing method is executed by a smart chip. The smart chip may be a programmable logic array chip or other chip with processing capabilities; this disclosure does not limit its application.
[0036] In operation S210, the point cloud data is meshed based on the width and height values to obtain the target mesh cell corresponding to the point cloud data in the predetermined mesh.
[0037] According to embodiments of this disclosure, the size of the predetermined grid can be set according to actual needs. For example, the predetermined grid can be a grid with a size of 192×192, a grid with a size of 384×384, or a grid with a size of 1152×1152, etc.
[0038] According to embodiments of this disclosure, each point cloud data may include (pc_x, pc_y, pc_z, intensity). Wherein (pc_x, pc_y, pc_z) are the three-dimensional coordinates of the point cloud data, and intensity is the reflection intensity value. This embodiment can receive point cloud data detected by a radar sensor in real time and perform meshing processing on each received point cloud data.
[0039] In this embodiment, the number of rows in the grid cell where the point cloud data is mapped to a predetermined grid can be determined based on the difference between the width and height values, and the number of columns in the grid cell can be determined based on the sum of the width and height values. Based on the number of rows and columns, a single grid cell can be uniquely identified as the target grid cell.
[0040] For example, if the width of the point cloud data is set to pc_x and the height of the point cloud data is set to pc_y, this embodiment can use the following formulas (1) and (2) to determine the number of rows pos_y and the number of columns pos_x respectively.
[0041] pos_y=(range-(a*(pc_x-pc_y)))*scale formula (1)
[0042] pos_x=(range-(a*(pc_x+pc_y)))*scale formula (2)
[0043] Wherein, `range` is an upper limit value set based on experience, and this upper limit value can be related to, for example, the detection depth of the LiDAR. The greater the detection depth, the larger the upper limit value. It is understood that the size of the predetermined mesh can be positively correlated with the detection depth, for example. Wherein, `a` is a hyperparameter set based on experience, and the value of `a` can be, for example, 0.707107, etc., which is not limited in this disclosure. `scale` is the scaling ratio, and the value of this scaling ratio can be set according to actual needs; for example, the scaling ratio can be positively correlated with the size of the predetermined mesh, which is not limited in this disclosure.
[0044] In operation S220, the point cloud statistics data corresponding to the target grid cell are updated based on the depth and reflection intensity values of the point cloud data.
[0045] According to embodiments of this disclosure, point cloud statistics may include, for example, at least one of the following depth-related data mapped to the point cloud data of the grid cells: average depth value, maximum depth value, and minimum depth value; and at least one of the following reflection intensity-related data: average intensity value, maximum intensity value, and minimum intensity value. It is understood that the type of point cloud statistics can be set according to actual needs, and this disclosure does not limit it in this regard.
[0046] In one embodiment, the point cloud statistics may also include, for example, the total number of point cloud data mapped to grid cells, in order to understand the distribution density of point cloud data mapped to each grid cell and provide richer information for the execution of the target task.
[0047] In this embodiment, the depth data in the point cloud statistics can be updated based on the depth values of the point cloud data. For example, if the point cloud statistics include a minimum depth value, this embodiment can compare the depth value of the point cloud data with the minimum depth value. If the depth value of the point cloud data is less than the minimum depth value, the minimum depth value is replaced with the depth value of the point cloud data. Alternatively, if the point cloud statistics include an average depth value, this embodiment can first calculate the current average depth value multiplied by the total number of current point cloud data, and then calculate the sum of the multiplied value and the depth values of the point cloud data. Finally, the ratio of the calculated sum to (the total number of current point cloud data + 1) is used as the updated average depth value. Simultaneously, the total number of current point cloud data can be incremented by 1 to obtain the updated total number of point cloud data. Similarly, the data regarding reflection intensity values in the point cloud statistics can be updated in the same way.
[0048] In operation S230, in response to updating the point cloud statistics corresponding to the grid cells in the predetermined grid based on a frame of point cloud data, the target task of computer vision is performed based on the point cloud statistics corresponding to the grid cells in the predetermined grid.
[0049] According to embodiments of this disclosure, the target task may include, for example, an object detection task, a semantic segmentation task, or an instance segmentation task. This embodiment converts point cloud data into feature data representing point cloud features through operations S210-S220. This feature data is the point cloud statistical data. This embodiment can represent the point cloud statistical data of each grid cell in a predetermined grid in vector form. Subsequently, the point cloud statistical data of all grid cells in the predetermined grid are organized into a matrix form or a vector sequence form, which is used as input data for a neural network model (e.g., a CNN model) performing the target task. The neural network model processes the input data to obtain detection results, semantic segmentation results, or instance segmentation results, thereby completing the execution of the target task.
[0050] According to an embodiment of this disclosure, a frame of point cloud data may include, for example, 28,800 point cloud data points. In this embodiment, after all 28,800 point cloud data points have been meshed and the point cloud statistics data of the corresponding mesh cells have been updated based on the 28,800 point cloud data points, the target task can be executed.
[0051] The technical solution of this disclosure allows a single intelligent chip to perform point cloud data preprocessing and target task execution. Compared to related technologies that require a main processor CPU for point cloud data preprocessing, this reduces communication overhead and avoids consuming CPU computing resources. This improves the execution efficiency of the target task, contributing to the stability and high performance of the autonomous driving system. Furthermore, by meshing the point cloud data based on width and height values, this disclosure enhances the ability to identify targets at different depths during target task execution. This allows the autonomous driving system to better determine the distance between the target and the vehicle, thereby improving the accuracy of autonomous driving decisions and enhancing the driving safety of autonomous vehicles.
[0052] Figure 3 This is a schematic diagram illustrating the principle of meshing point cloud data according to an embodiment of the present disclosure.
[0053] According to embodiments of this disclosure, before performing meshing processing on point cloud data, it can be determined, for example, whether the depth value of the point cloud data is within a predetermined depth range. If so, the point cloud data is then meshed. The predetermined depth range can be determined, for example, based on the detection range of the LiDAR. In this way, abnormal point cloud data collected by the LiDAR can be filtered out, reducing the processing of unnecessary point cloud data and improving the processing efficiency of point cloud data.
[0054] In one embodiment, when performing meshing processing on point cloud data, for example, the number of rows and columns of the target mesh cell corresponding to the point cloud data can be determined first based on the height and width values of the point cloud data using formulas (1) to (2) described above. Then, it is determined whether the number of rows and columns are both greater than 0. If so, the target mesh cell corresponding to the point cloud data in the predetermined mesh is determined based on the number of rows and columns. In this way, meaningless point cloud data can be eliminated, the processing of unnecessary point cloud data can be reduced, and the processing efficiency of point cloud data can be improved. This is because if the number of rows or columns is less than or equal to 0, it means that the point cloud data does not fall into the predetermined mesh after mapping, which to some extent reflects that the height and width values of the point cloud data are unreasonable.
[0055] For example, such as Figure 3 As shown, this embodiment 300 can perform gridding processing of point cloud data through operations S311 to S315.
[0056] The S311 operation acquires point cloud data (pc_x, pc_y, pc_z, intensity). Specifically, it can receive LiDAR data packets sent by the LiDAR via a Gigabit Medium Independent Interface (GMII Interface) in real time. By parsing the LiDAR data packets, the point cloud data (pc_x, pc_y, pc_z, intensity) can be obtained. The principle of parsing LiDAR data packets is similar to related technologies and will not be elaborated here.
[0057] In operation S312, it is determined whether the depth value pc_z of the point cloud data is within a predetermined range. This predetermined range may include, for example, a lower depth limit min_height and an upper depth limit max_height. If the depth value of the point cloud data is greater than or equal to the lower depth limit min_height and less than or equal to the upper depth limit max_height, then the depth value pc_z of the point cloud data is determined to be within the predetermined range. If it is within the predetermined range, operation S313 is executed. If it is not within the predetermined range, the acquired point cloud data is discarded, and operation S311 is returned to acquire new point cloud data.
[0058] In operation S313, the row number of the target mesh cell in the predetermined grid is determined based on the difference between the width value pc_x and the height value pc_y of the point cloud data. Simultaneously, the column number of the target mesh cell in the predetermined grid is determined based on the sum of the width value pc_x and the height value pc_y of the point cloud data. For example, formulas (1) and (2) described above can be used to determine the row and column numbers respectively.
[0059] In operation S314, determine whether the number of rows and columns are greater than 0. If both are greater than 0, then execute operation S315. Otherwise, discard the acquired point cloud data and return to operation S311 to acquire new point cloud data.
[0060] In operation S315, the target grid cell corresponding to the point cloud data in the predetermined grid is determined based on the number of rows and columns. For example, if the predetermined grid is stored in the form of a grid cell sequence, the arrangement position of the target grid cell in the predetermined grid can be determined as point2grid_[i] = pos_y * width + pos_x. Here, width is the number of grid cells included in a single row of the predetermined grid.
[0061] Figure 4 This is a schematic diagram illustrating the principle of updating point cloud data according to an embodiment of this disclosure.
[0062] According to embodiments of this disclosure, the intelligent chip can, for example, maintain a pointer table corresponding to a predetermined grid. This pointer table includes a pointer value corresponding to each grid cell within the predetermined grid. This pointer value can, for example, be used to determine whether each corresponding grid cell has corresponding point cloud statistics. In other words, the pointer value can reflect, to some extent, whether point cloud data mapped to the corresponding grid cell already exists. Thus, when updating point cloud data, the pointer value can be used to determine whether to modify the point cloud statistics corresponding to the target grid cell or to generate the point cloud statistics corresponding to the target grid cell. When it is necessary to generate the target grid cell, it is not necessary to first read the existing point cloud statistics, thereby reducing unnecessary data readings and improving the efficiency of point cloud data updates.
[0063] like Figure 4 As shown, in this embodiment 400, the smart chip can maintain a pointer table 410, where each pointer value occupies 1 bit of space. If each storage address `addr` in the smart chip's storage space has 64 bits of space, then each storage address `addr` can store 64 pointer values. For example, if the size of the predetermined grid 420 is set to 192×192, then three storage addresses in the storage space can store the pointer values corresponding to a row of grid cells in the predetermined grid. Accordingly, a total of 576 storage addresses can store the pointer values corresponding to all grid cells included in the 192×192 predetermined grid.
[0064] Based on the pointer table 410, when updating point cloud statistics, this embodiment can first determine whether the target grid cell has corresponding point cloud statistics according to the pointer table 410.
[0065] For example, the target pointer value 432 corresponding to the target grid cell 431 can be determined first based on the mapping relationship between the pointer value and the grid cell. That is, based on the position of the target grid cell 431 in the predetermined grid 420, the pointer table 410 is consulted to obtain the target pointer value 432. For example, the storage space of the 0th bit in the storage address addr 0 stores the pointer value corresponding to the grid cell in the 1st row and 1st column of the predetermined grid 420. The storage space of the 1st bit in the storage address addr 0 stores the pointer value corresponding to the grid cell in the 1st row and 2nd column of the predetermined grid 420, and so on. The storage space of the 63rd bit in the storage address addr 575 stores the pointer value corresponding to the grid cell in the 192nd row and 192nd column of the predetermined grid 420. Subsequently, based on the value of the determined target pointer value, it is determined whether the target grid cell has corresponding point cloud statistics. For example, if the value of the pointer value is 0, it can be determined that the target grid cell does not have corresponding point cloud statistics. If the value of this pointer is not 0, then it can be determined that the target grid cell has corresponding point cloud statistics.
[0066] If it is determined that the target mesh cell has corresponding point cloud statistics, the point cloud statistics 433 corresponding to the target mesh cell 431 can be updated based on the depth and reflection intensity values of the point cloud data. For example, new point cloud statistics can be determined based on existing point cloud statistics and the depth and reflection intensity values of the point cloud data, thereby completing the update of the point cloud statistics.
[0067] If it is determined that the target mesh cell does not have corresponding point cloud statistics, the corresponding point cloud statistics 433 can be determined based on the depth and reflection intensity values of the point cloud data. That is, point cloud statistics 433 is generated based on the depth and reflection intensity values. It can be understood that after generating the point cloud statistics corresponding to the target mesh cell, the pointing value in the pointing table 410 corresponding to the target mesh cell, i.e., the target pointing value 432, can be changed so that the changed target pointing value indicates that the target mesh cell 431 has corresponding point cloud statistics. For example, the target pointing value 432 can be changed from 0 to a non-zero value.
[0068] In one embodiment, the point cloud statistical data corresponding to each grid cell in the predetermined grid can be stored, for example, in a predetermined storage space of a memory connected to the smart chip. The predetermined memory can be, for example, a Double Data Rate (DDR) synchronous dynamic random access memory, while the memory built into the smart chip can be, for example, random access memory (RAM), system memory (ROM), etc., and this disclosure does not limit this. This embodiment, by storing the point cloud statistical data in the predetermined storage space, can reduce the occupation of the smart chip's built-in memory, thereby improving the processing efficiency of the smart chip. The predetermined storage space can, for example, have a subspace uniquely corresponding to each grid cell in the predetermined grid, for storing the point cloud statistical data corresponding to each grid cell.
[0069] Thus, in this embodiment 400, when the target grid cell has corresponding point cloud statistics, the point cloud statistics can first be read from the subspace corresponding to the target grid cell in the predetermined storage space. Subsequently, the read point cloud statistics are updated according to the depth value and reflection intensity value of the point cloud data.
[0070] In one embodiment, after updating the corresponding point cloud statistics (e.g., after generating the point cloud statistics), the updated point cloud statistics can be stored in the subspace corresponding to the target mesh cell. Specifically, for example, the original data stored in the corresponding subspace can be deleted, and the updated point cloud statistics can be written to the corresponding subspace.
[0071] Figure 5 This is a logical architecture diagram of a point cloud data processing method according to an embodiment of the present disclosure.
[0072] like Figure 5 As shown, for example, the logical architecture shown in Embodiment 500 can be used to implement the point cloud data processing method.
[0073] In this embodiment 500, the logic architecture can be implemented by the functional modules in the smart chip and the external memory DDR 520.
[0074] The point cloud statistics data corresponding to the grid cells in the predetermined grid are stored in DDR 520. The smart chip may be equipped with a DDR read & write module 519, which is used to read the existing point cloud statistics data from DDR 520 and write the updated point cloud statistics data into DDR 520.
[0075] For example, depending on actual needs, point cloud statistics can include the average intensity value, sum of intensity values, maximum depth value, and total number of point cloud data corresponding to each grid cell. Correspondingly, the functional modules in the smart chip can include an intensity value summing module 511, an intensity value averaging module 513, a counting module 512, and a maximum depth value determination module 514. The intensity value summing module 511 sums the reflection intensity values of the point cloud data mapped to each grid cell, thereby updating the total intensity value. The counting module 512 counts the point cloud data mapped to each grid cell, thereby updating the total number of point cloud data corresponding to each grid cell. The intensity value averaging module 513 calculates the average intensity value based on the sum of intensity values updated by the intensity value summing module 511 and the total number of point cloud data corresponding to each grid cell obtained by the counting module 512, thereby updating the average intensity value.
[0076] In one embodiment, the smart chip may also maintain a pointer table as described above, and the pointer table module 518 is used to store and maintain this pointer table. Upon receiving new point cloud data (pc_x, pc_y, pc_z, intensity), the smart chip may, for example, first determine the position of the target mesh cell corresponding to the point cloud data in a predetermined grid based on the width value pc_x and the height value pc_y. Subsequently, the pointer table module 518 queries the pointer table based on this position to obtain the pointer value corresponding to the target mesh cell.
[0077] If the pointer value is non-zero, the DDR read & write module 519 can read the point cloud statistics data corresponding to the target grid cell from the DDR 520 according to the position of the target grid cell in the predetermined grid. Subsequently, the intensity summation module 511, the counting module 512, and the maximum depth determination module 514 update the sum of intensity values, the total number of point cloud data corresponding to the target grid cell, and the maximum depth value in the read point cloud statistics data. Specifically, the intensity summation module 511 adds the read intensity sum to the reflection intensity value (intensity) in the point cloud data to obtain the updated intensity sum. The counting module 512 increments the total number of point cloud data corresponding to the target grid cell by 1 to obtain the updated total. The maximum depth determination module 514 compares the read maximum depth value with the depth value (pc_z) in the point cloud data. If the depth value (pc_z) is greater than the read maximum depth value, the read maximum depth value is replaced by the depth value (pc_z). Otherwise, the read maximum depth value remains unchanged.
[0078] In one embodiment, after updating the total intensity value and the total number of point cloud data corresponding to the target mesh cell, the intensity value averaging module 513 can, for example, calculate the average intensity value based on the updated total intensity value and the updated total number, and update the read average intensity value using the calculated average intensity value.
[0079] If the value pointed to by the query is 0, the intensity value summing module 511 can directly use the reflection intensity value in the point cloud data as the sum of intensity values, the maximum depth value can use the depth value in the point cloud data as the maximum intensity value, and the counting module 512 can set the total value to 1.
[0080] In one embodiment, for example, after the total number updated by the counting module 512 reaches the total number of point clouds included in a frame of point cloud, the intensity value averaging module 513 can then calculate the average intensity value based on the updated sum of intensity values and the updated total number. That is, in response to updating the point cloud statistics data corresponding to the grid cells in the predetermined grid based on a frame of point cloud data, the average intensity value can be updated based on the sum of intensity values in the point cloud statistics data and the total number of point cloud data corresponding to the grid cells. In this way, the number of times the average intensity value is updated can be reduced, thereby reducing the computational load of the smart chip and improving the processing efficiency of point cloud data. This is because, typically, when performing the target task, the point cloud statistics data of a frame of point cloud data is required as input, and the average intensity value calculated in the intermediate stage is not valuable.
[0081] In one embodiment, the DDR read / write module 519 in the smart chip can, for example, iterate through the subspaces corresponding to each grid cell in the DDR to read existing point cloud statistics data. Then, the intensity value averaging module 513 calculates the average intensity value based on the sum of the intensity values read from the existing point cloud statistics data and the total number of point cloud data. The DDR read / write module 519 then stores the calculated average intensity value in the subspace corresponding to the grid cell. Alternatively, after processing a frame of point cloud data, the smart chip can first control the pointer table module 518 to determine non-zero pointer values, and designate the grid cells mapped to these non-zero pointer values as the grid cells to be updated. Then, the DDR read / write module 519 iterates through the subspaces corresponding to the grid cells to be updated in the DDR 520 to read existing point cloud statistics data, and the intensity value averaging module 513 calculates the average intensity value based on the sum of the intensity values read from the existing point cloud statistics data and the total number of point cloud data. This reduces the need to traverse the empty subspace of DDR 520 during the calculation of the average strength value, thus improving the processing efficiency of point cloud data.
[0082] In one embodiment, after each update of the point cloud statistics based on the point cloud data, the data compression and encapsulation module 517 can compress and encapsulate the updated data to form a fixed format. Then, the DDR read / write module 519 writes the compressed and encapsulated fixed-format data into the DDR 520. It is understood that, except for the last point cloud data in a frame, after updating the point cloud statistics based on other point cloud data, the average intensity value in the compressed and encapsulated point cloud statistics can be a default initial value; this disclosure does not limit this.
[0083] In one embodiment, depending on actual needs, the intelligent chip may also include a depth value summation module 515 for summing the depth values of the point cloud data corresponding to each grid cell. The intelligent chip may also include a depth value averaging module 516 for calculating the average depth value based on the total number updated by the counting module 512 and the sum of depth values updated by the depth value summation module 515. It is understood that the calculation principle and timing of the depth value average are similar to those of the intensity value average, and will not be elaborated further here.
[0084] Figure 6 This is a chip architecture diagram for implementing a point cloud data processing method according to an embodiment of the present disclosure.
[0085] like Figure 6 As shown, in this embodiment 600, the intelligent chip 620 that performs the point cloud data processing method may include, for example, a GMII interface 621, a packet parsing module 622, a preprocessing module 623, and a CNN module 624. The preprocessing module 623 may include, for example, the components described above. Figure 5 The system comprises various logic modules. The GMII interface 621 receives data packets sent by the LiDAR 610 over the network. The data packet parsing module 622 parses the data packets received by the GMII interface 621 to obtain point cloud data 601. Subsequently, the preprocessing module 623 preprocesses the point cloud data 601 to obtain point cloud statistical data 602. Depending on the requirements, the point cloud statistical data 602 can be, for example, 6-dimensional data, including n values representing non-empty point cloud statistical data. o The data includes empty data, average intensity value, sum of intensity values, count of the total number of point cloud data corresponding to the grid cell, average depth value, and maximum depth value. For example, the space occupied by this 6-dimensional data can be 4 bytes, 6 bytes, 4 bytes, 4 bytes, 6 bytes, and 4 bytes respectively, and this disclosure does not limit this. After obtaining the 6-dimensional data through preprocessing, the 6-dimensional data can be written into DDR 630.
[0086] After writing all the data of a point cloud frame to DDR 630, the CNN module 624 can, for example, read the point cloud statistics of a point cloud frame from DDR 630, and perform computer vision target tasks using a CNN model based on the read point cloud statistics, such as instance segmentation tasks, which are not limited in this disclosure.
[0087] Based on the point cloud data processing method executed by the intelligent chip provided in this disclosure, this disclosure also provides a point cloud data processing device disposed in the intelligent chip. The following will be combined with... Figure 7 The device is described in detail.
[0088] Figure 7 This is a structural block diagram of a point cloud data processing apparatus according to an embodiment of the present disclosure.
[0089] like Figure 7 As shown, the point cloud data processing device 700 of this embodiment may include a gridding processing module 710, a statistical data update module 720, and a target task execution module 730. This point cloud data processing device 700 can be integrated into a smart chip.
[0090] The meshing module 710 is used to perform meshing processing on the point cloud data based on the width and height values in the point cloud data to obtain the target mesh cells corresponding to the point cloud data in a predetermined mesh. In one embodiment, the meshing module 710 can be used to perform the operation S210 described above, which will not be repeated here.
[0091] The statistical data update module 720 is used to update the point cloud statistical data corresponding to the target mesh cell based on the depth value and reflection intensity value of the point cloud data. In one embodiment, the statistical data update module 720 can be used to perform the operation S220 described above, which will not be repeated here.
[0092] The target task execution module 730 is used to update the point cloud statistics corresponding to the grid cells in a predetermined grid based on a frame of point cloud data, and to perform a computer vision target task based on the point cloud statistics corresponding to the grid cells in the predetermined grid. In one embodiment, the target task execution module 730 can be used to perform the operation S230 described above, which will not be repeated here.
[0093] According to embodiments of this disclosure, the aforementioned statistical data update module 720 may include a statistical data determination submodule and a data update submodule. The statistical data determination submodule is used to determine whether a target grid cell has corresponding point cloud statistical data based on a pointer table corresponding to a predetermined grid. The pointer table includes pointer values corresponding to grid cells in the predetermined grid, and the pointer values indicate whether the corresponding grid cell has corresponding point cloud statistical data. The data update submodule is used to update the point cloud statistical data corresponding to the target grid cell based on the depth value and reflection intensity value of the point cloud data in response to the target grid cell having corresponding point cloud statistical data.
[0094] According to embodiments of this disclosure, the aforementioned statistical data update module 720 may further include a data determination submodule, configured to determine the corresponding point cloud statistical data for the target grid cell based on the depth value and reflection intensity value of the point cloud data in response to the absence of corresponding point cloud statistical data for the target grid cell. The aforementioned point cloud data processing apparatus 700 may further include a pointer value modification module, configured to modify the pointer value corresponding to the target grid cell in the pointer table in response to the determination of the corresponding point cloud statistical data for the target grid cell based on the depth value and reflection intensity value of the point cloud data, so that the modified pointer value indicates that the target grid cell has corresponding point cloud statistical data.
[0095] According to embodiments of this disclosure, the point cloud data processing apparatus 700 may further include a data storage module, configured to store the updated point cloud statistical data in a subspace corresponding to the target grid cell within a predetermined storage space in response to completing the update of the point cloud statistical data corresponding to the target grid cell. The data update submodule may include a data reading unit and a data update unit. The data reading unit is used to read the point cloud statistical data from the subspace corresponding to the target grid cell. The data update unit is used to update the read point cloud statistical data based on the depth value and reflection intensity value of the point cloud data.
[0096] According to embodiments of this disclosure, point cloud statistics include the average intensity value, sum of intensity values, maximum depth value, and total number of point cloud data corresponding to the grid cell. Specifically, the aforementioned statistics update module 720 can be used to update the sum of intensity values, maximum depth value, and total number of point cloud data corresponding to the grid cell in the point cloud statistics for the target grid cell based on the depth value and reflection intensity value of the point cloud data. The aforementioned point cloud data processing apparatus 700 may further include an average value update module, used to update the average intensity value in the point cloud statistics corresponding to the grid cell based on the sum of intensity values and the total number of point cloud data corresponding to the grid cell in response to updating the point cloud statistics corresponding to the grid cell in a predetermined grid based on a frame of point cloud data.
[0097] According to embodiments of this disclosure, the above-mentioned average value update module may include a cell to be updated determination submodule, a data reading submodule, an average value determination submodule, and a data storage submodule. The cell to be updated determination submodule is used to determine the cell to be updated in the predetermined grid according to a pointer table corresponding to the predetermined grid. The pointer table includes pointer values corresponding to the cell in the predetermined grid, indicating whether the corresponding cell has corresponding point cloud statistical data; the cell to be updated is a cell that has corresponding point cloud statistical data. The data reading submodule is used to read the point cloud statistical data corresponding to the cell to be updated from the subspace corresponding to the cell to be updated in the predetermined storage space. The average value determination submodule is used to determine the average intensity value corresponding to the cell to be updated based on the sum of the intensity values in the read point cloud statistical data and the total number of point cloud data. The data storage submodule is used to store the average intensity value corresponding to the cell to be updated in the subspace corresponding to the cell to be updated.
[0098] According to an embodiment of this disclosure, the above-described meshing module 710 can be used to perform meshing processing on the point cloud data based on the width and height values in the point cloud data in response to the point cloud data's depth value being within a predetermined depth range.
[0099] According to embodiments of this disclosure, the above-described meshing processing module 710 may include a row number determination submodule, a column number determination submodule, and a target cell determination submodule. The row number determination submodule is used to determine the row number of the target mesh cell corresponding to the point cloud data within a predetermined mesh based on the difference between the width and height values. The column number determination submodule is used to determine the column number of the target mesh cell corresponding to the point cloud data within the predetermined mesh based on the sum of the width and height values. The target cell determination submodule is used to determine the target mesh cell corresponding to the point cloud data within the predetermined mesh based on the row and column numbers when both are greater than 0.
[0100] According to embodiments of this disclosure, the smart chip includes a programmable logic array chip.
[0101] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in this disclosed technical solution comply with relevant laws and regulations, necessary confidentiality measures have been taken, and it does not violate public order and good morals. In this disclosed technical solution, user authorization or consent has been obtained before acquiring or collecting user personal information.
[0102] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0103] Figure 8A schematic block diagram of a smart chip 800 that can be used to implement a point cloud data processing method according to embodiments of the present disclosure is shown. The smart chip can be disposed in an electronic device intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0104] like Figure 8 As shown, the intelligent chip 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 can also store various programs and data required for the operation of the device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0105] Multiple components in the smart chip 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard or mouse; an output unit 807, such as various types of displays or speakers; a storage unit 808, such as a disk or optical disk; and a communication unit 809, such as a network interface card (NIC), a modem, or a wireless transceiver. The communication unit 809 allows the device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0106] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as point cloud data processing methods. For example, in some embodiments, the point cloud data processing methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed into the smart chip 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the point cloud data processing methods described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform point cloud data processing methods by any other suitable means (e.g., by means of firmware).
[0107] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0108] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0109] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0111] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0112] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.
[0113] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0114] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A method for processing point cloud data executed by an intelligent chip, said intelligent chip being integrated in an autonomous vehicle, the method comprising: Based on the width and height values in the point cloud data, the point cloud data is gridded to obtain the target grid cell corresponding to the point cloud data in the predetermined grid. The smart chip maintains a pointer table corresponding to the predetermined grid, and the pointer table includes a pointer value corresponding to the grid cell in the predetermined grid. The pointer value indicates whether the corresponding grid cell has the corresponding point cloud statistics data. Based on the depth and reflection intensity values of the point cloud data, the point cloud statistics corresponding to the target mesh cell are updated; and In response to updating the point cloud statistics corresponding to the grid cells in the predetermined grid based on a frame of point cloud data, the target task of computer vision is executed based on the point cloud statistics corresponding to the grid cells in the predetermined grid. The step of updating the point cloud statistical data corresponding to the target grid cell includes: Based on the position of the target grid cell in the predetermined grid, the pointing table is queried to obtain the target pointing value; Based on the target pointing value, determine whether the target grid cell has corresponding point cloud statistics. In response to the fact that the target mesh cell has corresponding point cloud statistics, the point cloud statistics corresponding to the target mesh cell are updated according to the depth value and reflection intensity value of the point cloud data; In response to the absence of corresponding point cloud statistics for the target mesh cell, the point cloud statistics corresponding to the target mesh cell are determined based on the depth value and reflection intensity value of the point cloud data.
2. The method according to claim 1, further comprising: In response to determining the point cloud statistics corresponding to the target grid cell based on the depth value and reflection intensity value of the point cloud data, the pointing value corresponding to the target grid cell in the pointing table is changed so that the changed pointing value indicates that the target grid cell has the corresponding point cloud statistics.
3. The method according to claim 1, further comprising: In response to completing the update of the point cloud statistics data corresponding to the target grid cell, the updated point cloud statistics data is stored in the subspace corresponding to the target grid cell in the predetermined storage space; The step of updating the point cloud statistics corresponding to the target mesh cell in response to the point cloud cell having corresponding point cloud statistics data, based on the depth value and reflection intensity value of the point cloud data, includes: Read point cloud statistics data from the subspace corresponding to the target grid cell; as well as The point cloud statistics are updated based on the depth and reflection intensity values of the point cloud data.
4. The method according to any one of claims 1 to 3, wherein, Point cloud statistics include the average intensity value, the sum of intensity values, the maximum depth value, and the total number of point cloud data corresponding to the grid cell; The step of updating the point cloud statistics data corresponding to the target grid cell based on the depth value and reflection intensity value of the point cloud data includes: Based on the depth value and the reflection intensity value of the point cloud data, the sum of intensity values, the maximum depth value, and the total number of point cloud data corresponding to the grid cell in the point cloud statistics data of the target grid cell are updated. The method further includes: in response to updating the point cloud statistics data corresponding to the grid cells in the predetermined grid based on a frame of point cloud data, updating the average intensity value in the point cloud statistics data corresponding to the grid cells based on the sum of the intensity values in the point cloud statistics data corresponding to the grid cells and the total number of point cloud data corresponding to the grid cells.
5. The method according to claim 4, wherein, The average intensity value in the point cloud statistics data corresponding to the grid cell is updated based on the sum of the intensity values in the point cloud statistics data corresponding to the grid cell and the total number of point cloud data corresponding to the grid cell, including: Based on the pointer table corresponding to the predetermined grid, the grid cells to be updated in the predetermined grid are determined; wherein, the pointer table includes pointer values corresponding to the grid cells in the predetermined grid, and the pointer values indicate whether the corresponding grid cells have corresponding point cloud statistics; the grid cells to be updated are grid cells that have corresponding point cloud statistics. Read the point cloud statistics data corresponding to the grid cell to be updated from the subspace corresponding to the grid cell to be updated in the predetermined storage space; Based on the sum of intensity values in the read point cloud statistics and the total number of point cloud data, determine the average intensity value corresponding to the grid cell to be updated; and The average intensity value corresponding to the grid cell to be updated is stored in the subspace corresponding to the grid cell to be updated.
6. The method according to claim 1, wherein, The step of performing gridding processing on the point cloud data based on the width and height values includes: In response to the point cloud data having a depth value within a predetermined depth range, the point cloud data is meshed based on its width and height values.
7. The method according to claim 6, wherein, The point cloud data is meshed based on its width and height values, including: Based on the difference between the width value and the height value, determine the row number of the target grid cell corresponding to the point cloud data in the predetermined grid; Based on the sum of the width and height values, determine the column number of the target grid cell corresponding to the point cloud data within the predetermined grid; and In response to the number of rows and the number of columns being greater than 0, the target grid cell corresponding to the point cloud data in the predetermined grid is determined based on the number of rows and the number of columns.
8. The method according to any one of claims 1-7, wherein, The smart chip includes a programmable logic array chip.
9. A device for processing point cloud data integrated into a smart chip, the smart chip being integrated in an autonomous vehicle, the device comprising: The gridding processing module is used to perform gridding processing on the point cloud data according to the width and height values in the point cloud data to obtain the target grid cell corresponding to the point cloud data in the predetermined grid. The smart chip maintains a pointer table corresponding to the predetermined grid, and the pointer table includes a pointer value corresponding to the grid cell in the predetermined grid. The pointer value indicates whether the corresponding grid cell has the corresponding point cloud statistics data. The statistical data update module is used to update the point cloud statistical data corresponding to the target mesh cell based on the depth and reflection intensity values of the point cloud data; and The target task execution module is used to update the point cloud statistics data corresponding to the grid cells in the predetermined grid based on a frame of point cloud data, and to execute the target task of computer vision based on the point cloud statistics data corresponding to the grid cells in the predetermined grid. The statistical data update module includes: The query submodule is used to query the pointing table based on the position of the target grid cell in the predetermined grid to obtain the target pointing value; The statistical data determination submodule is used to determine whether the target grid cell has corresponding point cloud statistical data based on the target pointing value; The data update submodule is used to update the point cloud statistics corresponding to the target grid cell based on the depth value and reflection intensity value of the point cloud data in response to the fact that the target grid cell has corresponding point cloud statistics. The data determination submodule is used to determine the point cloud statistics corresponding to the target grid cell based on the depth value and reflection intensity value of the point cloud data in response to the absence of corresponding point cloud statistics for the target grid cell.
10. The apparatus according to claim 9, further comprising: The pointing value modification module is used to modify the pointing value corresponding to the target grid cell in the pointing table in response to determining the point cloud statistics data corresponding to the target grid cell based on the depth value and reflection intensity value of the point cloud data, so that the modified pointing value indicates that the target grid cell has the corresponding point cloud statistics data.
11. The apparatus according to claim 9, further comprising: The data storage module is used to store the updated point cloud statistical data in a subspace corresponding to the target grid cell in a predetermined storage space in response to completing the update of the point cloud statistical data corresponding to the target grid cell. The data update submodule includes: A data reading unit is used to read point cloud statistical data from the subspace corresponding to the target grid cell; and The data update unit is used to update the read point cloud statistical data based on the depth value and reflection intensity value of the point cloud data.
12. The apparatus according to any one of claims 9 to 11, wherein, Point cloud statistics include the average intensity value, sum of intensity values, maximum depth value, and total number of point cloud data corresponding to the grid cell; the statistics update module is used for: Based on the depth value and the reflection intensity value of the point cloud data, the sum of intensity values, the maximum depth value, and the total number of point cloud data corresponding to the grid cell in the point cloud statistics data of the target grid cell are updated. The device further includes an average value update module, which is used to update the point cloud statistics data corresponding to the grid cells in the predetermined grid based on a frame of point cloud data, and update the average value of the intensity values in the point cloud statistics data corresponding to the grid cells based on the sum of the intensity values in the point cloud statistics data corresponding to the grid cells and the total number of point cloud data corresponding to the grid cells.
13. The apparatus according to claim 12, wherein, The average value update module includes: The module for determining the unit to be updated is used to determine the grid unit to be updated in the predetermined grid according to the pointer table corresponding to the predetermined grid; wherein, the pointer table includes pointer values corresponding to the grid units in the predetermined grid, and the pointer values indicate whether the corresponding grid unit has corresponding point cloud statistics data; the grid unit to be updated is a grid unit that has corresponding point cloud statistics data. The data reading submodule is used to read the point cloud statistical data corresponding to the grid cell to be updated from the subspace corresponding to the grid cell to be updated in the predetermined storage space; The average value determination submodule is used to determine the average intensity value corresponding to the grid cell to be updated based on the sum of intensity values in the read point cloud statistical data and the total number of point cloud data; and The data storage submodule is used to store the average intensity value corresponding to the grid cell to be updated into the subspace corresponding to the grid cell to be updated.
14. The apparatus according to claim 9, wherein, The meshing processing module is used for: In response to the point cloud data having a depth value within a predetermined depth range, the point cloud data is meshed based on its width and height values.
15. The apparatus according to claim 14, wherein, The meshing processing module includes: The row number determination submodule is used to determine the row number of the target grid cell corresponding to the point cloud data in the predetermined grid based on the difference between the width value and the height value. The column number determination submodule is used to determine the column number of the target mesh cell corresponding to the point cloud data in the predetermined mesh based on the sum of the width value and the height value; and The target cell determination submodule is used to determine the target grid cell in the predetermined grid corresponding to the point cloud data based on the number of rows and the number of columns in response to the number of rows and the number of columns being greater than 0.
16. The apparatus according to any one of claims 9 to 15, wherein, The smart chip includes a programmable logic array chip.
17. A smart chip, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.
18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 8.
19. A computer program product comprising a computer program / instructions stored on at least one of a readable storage medium and an electronic device, wherein the computer program / instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.
20. An autonomous vehicle, comprising: The smart chip according to claim 17.