Method and apparatus for processing point cloud data

Abstract

This application provides a method and apparatus for processing point cloud data, which can be applied to intelligent vehicles, connected vehicles, and autonomous vehicles. The method includes: acquiring a three-dimensional (3D) point cloud map, the 3D point cloud map comprising n 3D spatial points, each of the n 3D spatial points including position information and normal vector information, where n is a positive integer; storing the principal scale of each 3D spatial point in a first image, the principal scale being obtained by rasterizing the position information, the principal scale corresponding to the pixel coordinate value in the first image; and storing the target normal vector information of each 3D spatial point in a second image, the target normal vector information being obtained by rasterizing the normal vector information. The solution of this application enables the loading, transmission, storage, and data processing of 3D point cloud maps in a lightweight manner.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a method and apparatus for processing point cloud data. Background Technology

[0002] With advancements in LiDAR (light detection and ranging) data acquisition and display technology, LiDAR sensors can be applied to an increasing number of fields, such as intelligent robot navigation, autonomous driving, and motion-sensing games.

[0003] However, each frame of LiDAR point cloud data contains tens of thousands of points, making the storage and transmission of such large amounts of data extremely expensive and time-consuming. For example, communication of LiDAR 3D point cloud maps over a network requires significant time and network bandwidth resources, limiting the cloud-based use of 3D point cloud map data. Furthermore, archiving and storing 3D point cloud maps also consumes substantial storage space, similarly limiting the commercial application of laser data. Summary of the Invention

[0004] This application provides a method and apparatus for processing point cloud data, enabling 3D point cloud maps to be loaded, transmitted, stored, and processed in a lightweight manner.

[0005] In a first aspect, a method for processing point cloud data is provided, comprising: acquiring a three-dimensional 3D point cloud map, the 3D point cloud map comprising n 3D spatial points, each of the n 3D spatial points comprising position information and normal vector information, where n is a positive integer; storing the principal scale of each 3D spatial point in a first image, the principal scale being obtained by rasterizing the position information, the principal scale corresponding to the pixel coordinate value in the first image; and storing the target normal vector information of each 3D spatial point in a second image, the target normal vector information being obtained by rasterizing the normal vector information.

[0006] In this embodiment, after acquiring the 3D point cloud map, the position information of each of the n 3D spatial points in the 3D point cloud map is rasterized to obtain the principal scale, and the principal scale of each 3D spatial point is stored in the first image; the normal vector information of each 3D spatial point is rasterized to obtain the target normal vector, and the target normal vector information of each 3D spatial point is stored in the second image. This allows the 3D point cloud map to be loaded, transmitted, stored, and processed in a lightweight manner.

[0007] It should be understood that the principal scale corresponds to the pixel coordinate value in the first image, or it can be described as the principal scale being used to represent the pixel coordinate value in the first image; or it can be described as the principal scale being the pixel coordinate value in the first image.

[0008] It should be understood that after obtaining the first image and the second image, a positioning layer can be generated based on the first image and the second image.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction respectively.

[0010] It should be understood that since the normal vector has a magnitude of 1, storing the normal vectors of two of the three directions will allow us to obtain the normal vector of the other direction.

[0011] Therefore, in this embodiment, the target normal vector information can be obtained by rasterizing only the components of the normal vector information in the first direction and the components in the second direction. The first and second directions can be two of the three directions. This further reduces the storage space of the normal vector information, allowing 3D point cloud maps to be loaded, transmitted, stored, and processed in a lightweight manner.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, storing the target normal vector information of each 3D spatial point in the second image includes: storing the subscale and target normal vector information of each 3D spatial point in the second image, wherein the subscale is obtained by rasterizing the position information.

[0013] It should be understood that location information typically suffers from a loss of accuracy when rasterized to the main scale, thus introducing errors. For example, during the rasterization process, this error can reach up to 0.25m at a resolution of 0.5m. Such a large error will inevitably reduce the positioning performance of the positioning module.

[0014] Therefore, in this embodiment, the main scale can be more finely depicted using a secondary scale. Specifically, the secondary scale is obtained by rasterizing the position information and stored in the second image. This ensures that the error introduced during the rasterization process is within an acceptable range, thereby improving the positioning performance of the positioning module.

[0015] It should be understood that the subscale obtained by rasterizing the location information can also be described as the subscale calculated based on the location information.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the first image is a multi-layer single-channel image, each layer of the multi-layer image includes 8 bits, each of the 8 bits corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

[0017] It should be understood that the layers, rows, columns, and bits included in the main scale correspond to the pixel coordinates of the first image.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the second image is a single-channel image, and the sub-scale and target normal vector information of each 3D spatial point occupies two pixel bits of the second image.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the sub-tick of each 3D spatial point occupies 6 bits of the two pixel bits, and the target normal vector information of each 3D spatial point occupies 10 bits of the two pixel bits.

[0020] It should be understood that, in the embodiments of this application, when storing the sub-scale in the second image, the sub-scale and target normal vector information of each 3D spatial point only need to occupy two pixels in the second image. Among them, the sub-scale of each 3D spatial point only occupies 6 bits in the two pixels, thereby reducing the error of rasterization of position information and improving the positioning performance of the positioning module while occupying less storage space.

[0021] It should be understood that when each sub-scale of a 3D spatial point occupies 6 bits, the sub-scale corresponding to the position information in each direction occupies 2 bits.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the resolution of the main scale is an integer multiple of the resolution of the sub-scale.

[0023] For example, when 2 bits are allocated to the sub-scale corresponding to the position information in each direction, the resolution of the main scale is 4 times that of the sub-scale.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: calculating an elevation offset that makes the elevation offset of the 3D spatial point with the smallest height among the n 3D spatial points zero.

[0025] In other words, the elevation offset is the height value of the 3D spatial point with the smallest height among n 3D spatial points. The height of the 3D spatial point with the smallest height among n 3D spatial points can be offset to 0 according to the elevation offset, and at the same time, the heights of the other points among the n 3D spatial points will also be offset according to the elevation offset.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: storing attribute information in an attribute file, the attribute information including: the number of layers of the first image, the elevation offset, and the value of n.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: generating a positioning layer based on the first image, the second image, and the attribute file.

[0028] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: constructing the association between the position information and normal vector information of each 3D spatial point.

[0029] Since each 3D spatial point in a point cloud map contains position information and normal vector information, and these two information are stored in different images during the rasterization process, it is necessary to establish association rules for them to be used when restoring the point cloud map.

[0030] In conjunction with the first aspect, in some implementations of the first aspect, the construction of the association between the position information and normal vector information of each 3D spatial point includes: obtaining a key based on the principal scale of each 3D spatial point and the size of the first image; obtaining a key value based on the target normal vector information of each 3D spatial point; and generating a key-value pair based on the key and key value of each 3D spatial point.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, after constructing the association between the position information and normal vector information of each 3D spatial point, the method further includes: sorting the key sequentially from smallest to largest in terms of layer, row, column, and bit dimensions.

[0032] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: restoring the 3D point cloud map based on the positioning layer and the association relationship.

[0033] In conjunction with the first aspect, in some implementations of the first aspect, the process of restoring the 3D point cloud map based on the positioning layer and the association includes: obtaining the attribute information from the attribute file; restoring n 3D spatial points sequentially from the smallest to the largest dimensions of layer, row, column, and bit according to the first image; restoring the position information and normal vector information corresponding to the n 3D spatial points one by one according to the association and the target normal vector information and sub-scale stored in the second image; and restoring the 3D point cloud map based on the position information and normal vector information of the n 3D spatial points.

[0034] In conjunction with the first aspect, in some implementations of the first aspect, the acquisition of the 3D point cloud map includes: acquiring an initial 3D point cloud map, the initial 3D point cloud map including m 3D spatial points, each of the m 3D spatial points including position information and normal vector information, where m is a positive integer; calculating the normal vectors of the m 3D spatial points; extracting the 3D spatial points corresponding to the normal vectors whose covariance is less than or equal to a first threshold; and generating the 3D point cloud map based on the extracted 3D spatial points.

[0035] In this embodiment, obtaining a 3D point cloud map specifically includes: obtaining an initial 3D point cloud map, calculating the normal vectors of m 3D spatial points in the initial 3D point cloud map, extracting the 3D spatial points corresponding to the normal vectors whose covariance is less than or equal to a first threshold, and generating a 3D point cloud map based on the extracted 3D spatial points. It should be understood that normal vectors with a covariance less than or equal to the first threshold can represent more reliable surface features. This solution significantly reduces the size of the data that should be stored in the point cloud map while retaining important 3D spatial points.

[0036] It should be understood that, in the embodiments of this application, the operation of extracting normal vectors with covariance less than or equal to a first threshold can be denoted as extracting surface features or extracting feature points.

[0037] In conjunction with the first aspect, in some implementations of the first aspect, the calculation of the normal vectors of the m 3D spatial points includes: calculating the normal vectors of the m 3D spatial points respectively by principal component analysis.

[0038] In conjunction with the first aspect, in some implementations of the first aspect, generating the 3D point cloud map based on the extracted 3D spatial points includes: generating an intermediate 3D point cloud map based on the extracted 3D spatial points; segmenting the intermediate 3D point cloud map into tile maps; merging the tile maps in the overlapping areas; and downsampling the merged tile maps to generate the 3D point cloud map.

[0039] In this embodiment, an intermediate 3D point cloud map can be generated based on the extracted 3D spatial points; the intermediate 3D point cloud map can be segmented into tile maps; the tile maps in the overlapping areas can be merged; and the merged tile maps can be downsampled to generate the 3D point cloud map. This can further reduce the number of 3D spatial points that should be stored in the point cloud map.

[0040] Secondly, an apparatus for processing point cloud data is provided, the apparatus comprising: an acquisition unit for acquiring a three-dimensional 3D point cloud map, the 3D point cloud map comprising n 3D spatial points, each of the n 3D spatial points comprising position information and normal vector information, where n is a positive integer; a processing unit for storing the principal scale of each 3D spatial point in a first image, the principal scale being obtained by rasterizing the position information, the principal scale corresponding to pixel coordinate values ​​in the first image; the processing unit further comprising storing the target normal vector information of each 3D spatial point in a second image, the target normal vector information being obtained by rasterizing the normal vector information.

[0041] In conjunction with the second aspect, in some implementations of the second aspect, the target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction respectively.

[0042] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is also used to: store the sub-scale and target normal vector information of each 3D spatial point in the second image, wherein the sub-scale is obtained by rasterizing the position information.

[0043] In conjunction with the second aspect, in some implementations of the second aspect, the first image is a multi-layer single-channel image, each layer of the multi-layer image includes 8 bits, each of the 8 bits corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

[0044] In conjunction with the second aspect, in some implementations of the second aspect, the second image is a single-channel image, and the sub-scale and target normal vector information of each 3D spatial point occupies two pixel bits of the second image.

[0045] In conjunction with the second aspect, in some implementations of the second aspect, the sub-tick of each 3D spatial point occupies 6 bits of the two pixel bits, and the target normal vector information of each 3D spatial point occupies 10 bits of the two pixel bits.

[0046] In conjunction with the second aspect, in some implementations of the second aspect, the resolution of the main scale is an integer multiple of the resolution of the sub-scale.

[0047] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is also used to: calculate an elevation offset that makes the elevation offset of the 3D spatial point with the smallest height among the n 3D spatial points zero.

[0048] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: store attribute information in an attribute file, the attribute information including: the number of layers of the first image, the elevation offset, and the value of n.

[0049] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: generate a positioning layer based on the first image, the second image, and the attribute file.

[0050] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is also used to: construct the association between the position information and normal vector information of each 3D spatial point.

[0051] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: obtain a key based on the principal scale of each 3D spatial point and the size of the first image; obtain a key value based on the target normal vector information of each 3D spatial point; and generate a key-value pair based on the key and key value of each 3D spatial point.

[0052] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is also used to: sort the key sequentially from smallest to largest in terms of layer, row, column, and bit dimensions.

[0053] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is also used to: recover the 3D point cloud map based on the positioning layer and the association relationship.

[0054] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: obtain the attribute information from the attribute file; recover the n 3D spatial points sequentially from the smallest to the largest dimensions of layer, row, column, and bit according to the first image; recover the position information and normal vector information corresponding to the n 3D spatial points one by one according to the association relationship and the target normal vector information and sub-scale stored in the second image; and recover the 3D point cloud map according to the position information and normal vector information of the n 3D spatial points.

[0055] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to: acquire an initial 3D point cloud map, the initial 3D point cloud map comprising m 3D spatial points, each of the m 3D spatial points comprising position information and normal vector information, where m is a positive integer; the processing unit is further configured to: calculate the normal vectors of the m 3D spatial points; extract the corresponding 3D spatial points whose covariance is less than or equal to a first threshold; and generate the 3D point cloud map based on the extracted 3D spatial points.

[0056] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is also used to: calculate the normal vectors of the m 3D space points respectively by principal component analysis.

[0057] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: generate an intermediate 3D point cloud map based on the extracted 3D spatial points; segment the intermediate 3D point cloud map into tile maps; merge the tile maps in the overlapping regions; and downsample the merged tile maps to generate the 3D point cloud map.

[0058] Thirdly, a method for recovering a 3D point cloud map is provided, characterized in that the 3D point cloud map includes n 3D spatial points, each of the n 3D spatial points includes position information and normal vector information, where n is a positive integer; the method includes:

[0059] A positioning layer is obtained, which is generated based on a first image, a second image, and an attribute file. The first image stores the principal scale of each 3D spatial point. The principal scale is obtained by rasterizing the position information, and the principal scale corresponds to the pixel coordinate value in the first image. The second image stores the target normal vector information of each 3D spatial point. The target normal vector information is obtained by rasterizing the normal vector information. The attribute file stores attribute information, including: the layer number of the first image, the elevation offset, and the value of n.

[0060] Obtain the correlation between the position information and normal vector information of each 3D spatial point;

[0061] Restore the 3D point cloud map based on the positioning layer and its relationships.

[0062] In this embodiment, the location layer and the correlation between the position information and normal vector information of each 3D spatial point are obtained, and the 3D point cloud map is recovered based on the location layer and the correlation. This allows for a simpler recovery of the position information and normal vector information of n 3D spatial points in the 3D point cloud map.

[0063] It should be understood that the generation of the positioning layer and the construction of its relationships can be completed by the cloud server, while the process of restoring the 3D point cloud map can be implemented by the vehicle. Specifically, the vehicle can first obtain the positioning layer and its relationships from the cloud, and then restore the 3D point cloud map based on the positioning layer and its relationships.

[0064] It should be understood that, if the vehicle has the relevant computing capabilities, the above operations can also be completed on the vehicle side; or, the cloud can also restore the 3D point cloud map and then send it to the vehicle side. This application does not limit this, and the specific method should be determined based on the actual situation.

[0065] In conjunction with the third aspect, in some implementations of the third aspect, the first image is a multi-layer single-channel image, each layer in the multi-layer includes 8 bits, each of the 8 bits corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

[0066] In conjunction with the third aspect, in some implementations of the third aspect, restoring a 3D point cloud map based on the positioning layer and the association relationship includes: obtaining attribute information from the attribute file; restoring n 3D spatial points sequentially from the smallest to the largest dimensions of layer, row, column, and bit according to the first image; restoring the position information and normal vector information corresponding to the n 3D spatial points one by one according to the association relationship and the target normal vector information stored in the second image; and restoring the 3D point cloud map according to the position information and normal vector information of the n 3D spatial points.

[0067] In conjunction with the third aspect, in some implementations of the third aspect, the second image is also used to store the sub-scale of each 3D spatial point, which is obtained by rasterizing the position information; the step of recovering the position information and normal vector information corresponding to n 3D spatial points one by one according to the association relationship and the target normal vector information stored in the second image includes: recovering the position information and normal vector information corresponding to n 3D spatial points one by one according to the association relationship and the target normal vector information and sub-scale stored in the second image.

[0068] In this embodiment, the second image is further used to store the sub-scale of each 3D spatial point, so that during the restoration process, the position information and normal vector information corresponding to n 3D spatial points can be restored one by one according to the correlation relationship and the target normal vector information and sub-scale stored in the second image. This allows for a simpler and more accurate restoration of the 3D point cloud map.

[0069] In conjunction with the third aspect, in some implementations of the third aspect, obtaining the association between the position information and normal vector information of each 3D spatial point includes: obtaining the key-value pair corresponding to each 3D spatial point, wherein the key-value pair corresponding to each 3D spatial point is generated by the key and key-value pair corresponding to each 3D spatial point, the key of each 3D spatial point is obtained based on the principal scale of each 3D spatial point and the size of the first image, and the key-value pair of each 3D spatial point is obtained based on the target normal vector information of each 3D spatial point.

[0070] In conjunction with the third aspect, in some implementations of the third aspect, the keys of n 3D spatial points are sorted sequentially from smallest to largest in terms of layer, row, column, and bit dimensions.

[0071] In conjunction with the third aspect, in some implementations of the third aspect, the resolution of the main scale is an integer multiple of the resolution of the subscale.

[0072] In conjunction with the third aspect, in some implementations of the third aspect, the elevation offset can make the elevation offset of the 3D spatial point with the smallest height among n 3D spatial points zero.

[0073] In conjunction with the third aspect, in some implementations of the third aspect, the target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction respectively.

[0074] In conjunction with the third aspect, in some implementations of the third aspect, the second image is a single-channel image, and the sub-scale and target normal vector information of each 3D spatial point occupy two pixel bits of the second image.

[0075] In conjunction with the third aspect, in some implementations of the third aspect, the sub-tick of each 3D spatial point occupies 6 bits out of two pixels, and the target normal vector information of each 3D spatial point occupies 10 bits out of two pixels.

[0076] Fourthly, an apparatus for recovering a 3D point cloud map is provided, characterized in that the 3D point cloud map includes n 3D spatial points, each of the n 3D spatial points includes position information and normal vector information, where n is a positive integer; the apparatus includes:

[0077] The acquisition unit is used to acquire the positioning layer, which is generated based on the first image, the second image, and the attribute file. The first image is used to store the principal scale of each 3D spatial point. The principal scale is obtained by rasterizing the position information, and the principal scale corresponds to the pixel coordinate value in the first image. The second image is used to store the target normal vector information of each 3D spatial point. The target normal vector information is obtained by rasterizing the normal vector information. The attribute file is used to store attribute information, including: the layer number of the first image, the elevation offset, and the value of n.

[0078] The acquisition unit is also used to acquire the relationship between the position information and normal vector information of each 3D spatial point;

[0079] The processing unit is used to recover the 3D point cloud map based on the positioning layer and the relationship.

[0080] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the first image is a multi-layer single-channel image, each layer in the multi-layer includes 8 bits, each of the 8 bits corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

[0081] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the processing unit is also used to: obtain attribute information from the attribute file; recover n 3D spatial points sequentially from the dimensions of layer, row, column, and bit bit according to the first image in ascending order; recover the position information and normal vector information corresponding to the n 3D spatial points one by one according to the association relationship and the target normal vector information stored in the second image; and recover the 3D point cloud map according to the position information and normal vector information of the n 3D spatial points.

[0082] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the second image is also used to store the sub-scale of each 3D spatial point, which is obtained by rasterizing the position information; the processing unit is also used to: recover the position information and normal vector information corresponding to n 3D spatial points one by one according to the correlation and the target normal vector information and sub-scale stored in the second image.

[0083] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the acquisition unit is further used to: acquire key-value pairs corresponding to each 3D spatial point, wherein each key-value pair corresponding to each 3D spatial point is generated by the key and key-value pair corresponding to each 3D spatial point, the key of each 3D spatial point is obtained based on the principal scale of each 3D spatial point and the size of the first image, and the key-value pair of each 3D spatial point is obtained based on the target normal vector information of each 3D spatial point.

[0084] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the keys of the n 3D spatial points are sorted sequentially from smallest to largest in terms of layer, row, column, and bit dimensions.

[0085] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the resolution of the main scale is an integer multiple of the resolution of the subscale.

[0086] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the elevation offset can make the elevation offset of the 3D spatial point with the smallest height among n 3D spatial points zero.

[0087] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction respectively.

[0088] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the second image is a single-channel image, and the sub-scale and target normal vector information of each 3D spatial point occupy two pixel bits of the second image.

[0089] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the sub-tick of each 3D spatial point occupies 6 bits out of two pixels, and the target normal vector information of each 3D spatial point occupies 10 bits out of two pixels.

[0090] Fifthly, a computing device is provided, comprising: at least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform a method for processing point cloud data as described in the first aspect or any possible implementation thereof; or,

[0091] To perform a method for restoring a 3D point cloud map, as in the third aspect or any possible implementation of the third aspect.

[0092] Optionally, depending on the actual situation, the aforementioned computing devices can be deployed in the cloud or in the vehicle, or some modules of the aforementioned computing devices can be deployed in the cloud and some modules in the vehicle. The specific deployment method needs to be determined based on the actual situation, and this application does not limit it in this regard.

[0093] Sixthly, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to perform the method for processing point cloud data as described in the first aspect or any possible implementation thereof; or,

[0094] The method for restoring a 3D point cloud map is performed as described in the third aspect or any possible implementation thereof.

[0095] A seventh aspect provides a computer-readable storage medium storing program code for execution by a device, the program code including instructions for performing a method for processing point cloud data in the first aspect or any possible implementation thereof; or,

[0096] Includes instructions for performing a method to restore a 3D point cloud map in the third aspect or any possible implementation of the third aspect.

[0097] Eighthly, a chip is provided, the chip including a processor and a data interface, the processor reading instructions stored in a memory through the data interface to execute the method for processing point cloud data in the first aspect or any possible implementation of the first aspect; or,

[0098] A method for restoring a 3D point cloud map in performing the third aspect or any possible implementation of the third aspect.

[0099] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is configured to execute the instructions stored in the memory. When the instructions are executed, the processor is configured to perform the method for processing point cloud data in the first aspect or any possible implementation of the first aspect; or,

[0100] A method for restoring a 3D point cloud map in performing the third aspect or any possible implementation of the third aspect.

[0101] A ninth aspect provides a vehicle including various modules for performing a method for recovering a 3D point cloud map as described in the third aspect or any possible implementation thereof. Attached Figure Description

[0102] Figure 1 This is an example diagram illustrating a possible application scenario provided by an embodiment of this application;

[0103] Figure 2 This is a system architecture example diagram provided in an embodiment of this application;

[0104] Figure 3 This is an example diagram of a method for processing point cloud data provided in an embodiment of this application;

[0105] Figure 4 This is an exemplary flowchart of a point cloud data processing embodiment provided in this application;

[0106] Figure 5 This is an exemplary flowchart of another point cloud data processing method provided in this application embodiment;

[0107] Figure 6 This is an example diagram of a target area provided in an embodiment of this application;

[0108] Figure 7 This is an example diagram of a method for generating a point cloud tile map containing normal vector information provided in an embodiment of this application;

[0109] Figure 8 This is an example diagram of location information rasterization provided in an embodiment of this application;

[0110] Figure 9 This is an example diagram of a main and secondary scale provided in an embodiment of this application;

[0111] Figure 10 This application provides a positioning error distribution map based on a point cloud compressed layer.

[0112] Figure 11 This application provides a localization error distribution map based on a point cloud map after initial downsampling.

[0113] Figure 12 This is an example diagram of a device for processing point cloud data provided in an embodiment of this application;

[0114] Figure 13 This is an example diagram illustrating a method for recovering a 3D point cloud map provided in an embodiment of this application;

[0115] Figure 14 This is an example diagram of a device for recovering a 3D point cloud map provided in an embodiment of this application;

[0116] Figure 15 This is an exemplary block diagram of the hardware structure of a device provided in an embodiment of this application;

[0117] Figure 16 This is a functional block diagram of a vehicle to which this application embodiment applies;

[0118] Figure 17 This is an example diagram of an autonomous driving system to which this application embodiment applies;

[0119] Figure 18 This is an example diagram of a cloud-based command-driven autonomous vehicle application according to an embodiment of this application. Detailed Implementation

[0120] To facilitate understanding, the following section first introduces several concepts and terms involved in this application.

[0121] 3D point cloud, also known as laser point cloud (PCD) or point cloud, is a massive collection of points representing the spatial distribution and surface characteristics of an object, obtained by using laser light to acquire the 3D spatial coordinates of each sampled point on the object's surface within the same spatial reference frame. Compared to images, 3D point clouds, while lacking detailed texture information, contain rich 3D spatial information.

[0122] Point cloud data refers to a collection of vectors in a 3D coordinate system. These vectors are usually represented in the form of x, y, z 3D coordinates, and generally represent the geometry of the outer surface of an object. In addition, point cloud data can also include RGB information, that is, the color information of each coordinate point, or other information.

[0123] LiDAR (Light Detection and Ranging) is a radar system that uses laser beams to detect the position, velocity, and other characteristics of targets. Its working principle involves emitting a detection signal (laser beam) towards the target, then comparing the received signal reflected back from the target (target echo) with the emitted signal. After appropriate processing, information about the target can be obtained, such as its distance, azimuth, altitude, velocity, attitude, and even shape. This allows for the detection, tracking, and identification of targets such as aircraft and missiles.

[0124] Six degrees of freedom (DOF) refers to the ability to move freely in six directions. Specifically, it refers to free translational motion in three directions (forward / backward, upward / downward, left / right) and free rotational motion about three axes (pitch, roll, and vertical).

[0125] Rasterization: A raster is a pixel, and rasterization is the process of converting vector graphics into bitmaps (raster images). The most basic rasterization algorithm renders a 3D scene represented by polygons onto a two-dimensional (2D) surface.

[0126] Point cloud normal vector: Point cloud data is sampled from the surface of an object, and the normal vector of the object's surface is the point cloud normal vector. The point cloud normal vector is an important feature of 3D point clouds, providing rich 3D spatial information and can be widely used for target detection in 3D point clouds.

[0127] Principal component analysis (PCA) is a statistical analysis method for simplifying datasets. It uses orthogonal transformations to linearly transform the observations of a series of potentially correlated variables, projecting them into a series of linearly uncorrelated variables called principal components. Specifically, a principal component can be viewed as a linear equation containing a series of linear coefficients indicating the direction of the projection.

[0128] Downsampling, also known as image reduction or image downsampling, involves reducing the number of sampling points. For an N*M image, if the downsampling coefficient is 'a', it means that every 'a' points in each row and column of the original image are taken to form a new image. For example, raster sampling samples points in a raster space into a single point to achieve downsampling. Here, N, M, and a are all integers greater than or equal to 1.

[0129] Tiles: refers to square raster images of a map within a certain area, cut into rows and columns according to a certain size and format, and according to a zoom level or scale. The square raster images after being sliced ​​are figuratively called tiles.

[0130] A tile map is a map made up of individual small square pictures pieced together.

[0131] Portable Network Graphics (PNG) is a bitmap format that uses lossless compression algorithms. It was designed to replace Graphics Interchange Format (GIF) and Tag Image File Format (TIFF), while adding features not found in GIF. PNG uses a lossless data compression algorithm derived from LZ77 and is commonly used in Java programs, web pages, or S60 (Series 60) programs due to its high compression ratio and small file size.

[0132] Simultaneous localization and mapping (SLAM) is a concept that aims to enable a robot to start from an unknown location in an unknown environment, locate its own position and orientation by repeatedly observing map features (such as corners, pillars, etc.) during movement, and then incrementally build a map based on its own position, thereby achieving the goal of simultaneous localization and mapping.

[0133] The iterative closest point (ICP) algorithm is a classic data registration algorithm. Its key feature is that it finds corresponding point pairs between the source and target point clouds, constructs a rotation and translation matrix based on these pairs, and uses this matrix to transform the source point cloud into the target point cloud's coordinate system. It then estimates the error function between the transformed source and target point clouds. If the error function value exceeds a threshold, the above calculation is iterated until a given error requirement is met.

[0134] To facilitate understanding, a brief introduction to the background technology involved in this application will be given first below.

[0135] As LiDAR data acquisition and display technologies become increasingly advanced and their prices decrease, the application of LiDAR sensors in more and more fields is becoming a trend. However, each frame of LiDAR point cloud data contains tens of thousands of points, and the storage and transmission of such large amounts of data are very expensive and time-consuming. For example, communication of point cloud maps over the network requires significant time and network bandwidth resources, limiting the cloud-based use of LiDAR point cloud map data. Furthermore, archiving and storing LiDAR point cloud maps also consumes substantial storage space, similarly limiting the commercial application scope of laser data.

[0136] LiDAR point cloud localization maps are core foundational data in the field of autonomous driving. Even a relatively small high-precision LiDAR localization map can contain millions of points, making the storage space required for LiDAR localization maps covering an entire city or even the whole country unimaginable. LiDAR localization technology based on large-scale pre-fabricated point cloud maps has been widely applied in the field of autonomous driving due to its high reliability and availability. However, due to the limitations of computing and storage resources on autonomous vehicles, LiDAR localization maps based on raw point cloud data pose significant challenges to real-time on-vehicle LiDAR data processing and offline map archiving.

[0137] Based on this, this application provides a method for processing point cloud data. The method primarily involves storing the principal scale of each 3D spatial point in a 3D point cloud map in a first image and storing the target normal vector information of each 3D spatial point in a second image, thereby obtaining a positioning layer based on the first and second images. This allows 3D point cloud maps to be loaded, transmitted, stored, and processed in a lightweight manner.

[0138] It should be understood that the method for processing point cloud data in this application is mainly implemented through a cloud server. However, if the vehicle has sufficient computing power, the above method can also be completed by the vehicle itself, without limitation. But for ease of description, the process of generating the positioning layer will be assumed to be performed by the cloud server in the following text.

[0139] This application also provides a method for recovering 3D point cloud maps, which mainly involves obtaining a positioning layer and recovering the 3D point cloud map based on the positioning layer, thereby making it relatively easy to recover the 3D point cloud map.

[0140] It should be understood that the method for restoring the 3D point cloud map in this application is mainly implemented through the vehicle-mounted computing device. However, in actual operation, a cloud server can also be used to restore the 3D point cloud map, and simultaneously obtain the collected 3D point cloud data from the vehicle and match it to generate control commands, which are then sent to the vehicle. This application does not limit this. However, for ease of description, the process of generating and restoring the 3D point cloud map is assumed to be performed by the vehicle-mounted computing device in the following text.

[0141] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0142] Figure 1 This is an example diagram illustrating a possible application scenario provided by an embodiment of this application. This scenario, for example, can be applied to vehicle positioning in autonomous driving scenarios. Figure 1As shown, a sensor 11 and a computing device 12 can be installed in the vehicle 10. The sensor 11 is used to detect and scan 3D point cloud data in the target scene. As an example, the sensor 11 may include a LiDAR, a contact scanner, a depth camera, etc., which are not limited in this application. The computing device 12 is connected to the sensor 11 and is used to acquire the 3D point cloud data scanned by the sensor 11. The vehicle 10 also needs to acquire a laser point cloud positioning map from other devices, such as a cloud server, and achieve positioning by matching the 3D point cloud data collected by the vehicle 10 with the acquired laser point cloud positioning map, thereby further realizing 6DOF pose estimation.

[0143] It should be understood that Figure 1 The scenario described is merely an example; the method of this application can also be applied to other types of scenarios. For example, it can also be applied to scenarios such as intelligent robot navigation and motion-sensing games.

[0144] Figure 2 This is a system architecture example diagram provided in an embodiment of this application. For example... Figure 2 As shown, the system architecture 200 includes: a laser map (i.e., a 3D point cloud map) generation module 210, a positioning layer generation module 220, and a 3D LiDAR positioning module 230. These three modules will be described in detail below.

[0145] The laser map generation module 210 includes an offline mapping submodule and an online mapping submodule. The laser map generation module 210 is used to complete online or offline mapping work using the SLAM algorithm, generating a high-quality laser point cloud map of the target scene, and providing it to the positioning layer generation module 220.

[0146] The positioning layer generation module 220 is used to obtain 3D spatial point information from the original laser point cloud map output by the laser map generation module 210 according to the requirements of the 3D LiDAR positioning module 230, and then generate a positioning layer with a high compression ratio by combining the point cloud data processing method involved in this application and distribute it for deployment on the autonomous vehicle. The requirements of the 3D LiDAR positioning module 230 on the autonomous vehicle can be met with a small amount of storage space, communication bandwidth and computational complexity.

[0147] The 3D LiDAR positioning module 230 is used to recover the positioning layer to a 3D point cloud map using the decompression algorithm (i.e., the method for recovering the 3D point cloud map) provided in this application. Specifically, the 3D point cloud data collected by the autonomous vehicle is matched with the recovered 3D point cloud map (e.g., particle filtering, iterative closest point (ICP) algorithm, etc.) to achieve 6DOF pose estimation.

[0148] Figure 3This is an example diagram of a method for processing point cloud data provided in an embodiment of this application. Figure 3 The method shown can be applied to system architecture 200. For example... Figure 3 As shown, method 300 includes steps S310 to S330, which are described in detail below.

[0149] S310, acquire a 3D point cloud map.

[0150] The 3D point cloud map consists of n 3D spatial points. Each of the n 3D spatial points includes position information and normal vector information, where n is a positive integer.

[0151] Optionally, the aforementioned 3D point cloud map can be an initial 3D point cloud map acquired by LiDAR.

[0152] Optionally, the aforementioned 3D point cloud map can also be a 3D point cloud map obtained by processing an initial 3D point cloud map acquired by LiDAR. For ease of description, the aforementioned 3D point cloud map will be considered as a 3D point cloud map obtained by processing an initial 3D point cloud map acquired by LiDAR in the following text, but it should be understood that this does not constitute a limitation of this application.

[0153] Optionally, a 3D point cloud map can be obtained through the following processing: obtaining an initial 3D point cloud map, which includes m 3D spatial points, each of which includes position information and normal vector information, where m is a positive integer; calculating the normal vectors of the m 3D spatial points; extracting the 3D spatial points corresponding to the normal vectors whose covariance is less than or equal to a first threshold; and generating the 3D point cloud map based on the extracted 3D spatial points.

[0154] It should be understood that normal vectors with a covariance less than or equal to a first threshold can represent more reliable surface features. The solution of this application significantly reduces the size of the data that should be stored in the point cloud map while retaining important 3D spatial points. It should be understood that in the embodiments of this application, the operation of extracting normal vectors with a covariance less than or equal to the first threshold can be denoted as extracting surface features or extracting feature points.

[0155] Optionally, the normal vectors of the m 3D space points can be calculated separately using principal component analysis. It should be understood that other existing methods can also be used, and this application does not limit this to any particular method.

[0156] Optionally, the initial 3D point cloud map can be downsampled before feature point extraction. For example, the initial 3D point cloud map can be downsampled by 0.1m × 0.1m × 0.1m.

[0157] Optionally, generating the 3D point cloud map based on the extracted 3D spatial points includes: generating an intermediate 3D point cloud map based on the extracted 3D spatial points; segmenting the intermediate 3D point cloud map into tile maps; merging the tile maps in the overlapping areas; and downsampling the merged tile maps to generate the 3D point cloud map.

[0158] For example, the merged tile map is downsampled by 0.5m×0.5m×0.5m to obtain a 3D point cloud map.

[0159] In other words, the 3D point cloud map in this application can be obtained after downsampling, feature point extraction, tileization, and other processing. It should be understood that, in the following text, the 3D point cloud map after tileization can also be referred to as a point cloud tile map or simply as a point cloud map, and this application does not make a distinction between them.

[0160] It should be understood that the specific implementation methods described below are all based on the final point cloud tile map. Furthermore, the process of processing the initial 3D point cloud map to obtain the point cloud tile map can be found in the descriptions of methods 400 and 500 below.

[0161] S320 stores the master scale of each 3D spatial point in the first image.

[0162] The main scale is obtained by rasterizing the position information, and the main scale corresponds to the pixel coordinate value in the first image.

[0163] In other words, this application requires first rasterizing the position information to obtain the main scale, and then storing the main scale in the first image. The main scale corresponds to the pixel coordinate value in the first image; it can also be described as the main scale representing the pixel coordinate value in the first image; or as the main scale being the pixel coordinate value in the first image.

[0164] Optionally, the first image is a multi-layer single-channel image, each layer in the multi-layer image includes 8 bits, each of the 8 bits corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

[0165] It should be understood that the layers, rows, columns, and bits included in the main scale correspond to the pixel coordinates of the first image.

[0166] It should be understood that the calculation method for the main scale is explained below. Figure 8 Related descriptions for some parts.

[0167] S330 stores the target normal vector information of each 3D spatial point in the second image.

[0168] Among them, the target normal vector information is obtained by rasterizing the normal vector information.

[0169] In other words, this application requires first rasterizing the normal vector information to obtain the target normal vector information, and then storing the target normal vector information in the second image.

[0170] It should be understood that after obtaining the first image and the second image, a positioning layer can be generated based on the first image and the second image.

[0171] In this embodiment, after acquiring the 3D point cloud map, the position information of each of the n 3D spatial points in the 3D point cloud map is rasterized to obtain the principal scale, and the principal scale of each 3D spatial point is stored in the first image; the normal vector information of each 3D spatial point is rasterized to obtain the target normal vector, and the target normal vector information of each 3D spatial point is stored in the second image; then, a positioning layer is generated based on the first and second images. This allows the 3D point cloud map to be loaded, transmitted, stored, and processed in a lightweight manner.

[0172] It should be understood that since the normal vector has a magnitude of 1, storing the normal vectors of two of the three directions will allow us to obtain the normal vector of the other direction.

[0173] Therefore, optionally, the target normal vector information can be obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction respectively. The first and second directions can be two of the three directions. This further reduces the storage space of the normal vector information, allowing 3D point cloud maps to be loaded, transmitted, stored, and processed in a lightweight manner.

[0174] Optionally, storing the target normal vector information of each 3D spatial point in the second image includes storing the sub-scale and target normal vector information of each 3D spatial point in the second image. The sub-scale is obtained by rasterizing the position information.

[0175] It should be understood that location information typically suffers from precision loss during rasterization of the main scale, thus introducing errors. For example, this error can reach up to 0.25m at a resolution of 0.5m during rasterization, and such a large error will inevitably reduce the positioning performance of the positioning module. Therefore, in this embodiment, the main scale can be more finely depicted using a secondary scale. Specifically, the secondary scale is obtained by rasterizing the location information and stored in the second image. This ensures that the error introduced by the location information during rasterization is within an acceptable range, thereby improving the positioning performance of the positioning module.

[0176] It should be understood that the subscale obtained by rasterizing the location information can also be described as the subscale calculated based on the location information.

[0177] Optionally, the second image is a single-channel image, with the sub-scale and target normal vector information of each 3D spatial point occupying two pixels in the second image.

[0178] Optionally, the sub-tick of each 3D spatial point occupies 6 bits out of two pixels, and the target normal vector information of each 3D spatial point occupies 10 bits out of two pixels.

[0179] It should be understood that, in the embodiments of this application, when storing the sub-scale in the second image, the sub-scale and target normal vector information of each 3D spatial point only need to occupy two pixels in the second image. Among them, the sub-scale of each 3D spatial point only occupies 6 bits in the two pixels, thereby reducing the error of rasterization of position information and improving the positioning performance of the positioning module while occupying less storage space.

[0180] It should be understood that when each sub-scale of a 3D spatial point occupies 6 bits, the sub-scale corresponding to the position information in each direction occupies 2 bits.

[0181] Optionally, the resolution of the main scale is an integer multiple of the resolution of the sub-scale. For example, if the resolution of the main scale is 0.5m, and 2 bits are allocated to the sub-scale corresponding to the position information in each direction, the resolution of the sub-scale is 0.125m. In this case, the resolution of the main scale is four times the resolution of the sub-scale. See [link to relevant documentation] for details. Figure 9 Partial description.

[0182] Optionally, method 300 may further include: calculating an elevation offset. It should be understood that the elevation offset is such that the elevation offset of the 3D spatial point with the smallest height among the n 3D spatial points is 0.

[0183] In other words, the elevation offset is the height value of the 3D spatial point with the smallest height among n 3D spatial points. The height of the 3D spatial point with the smallest height among n 3D spatial points can be offset to 0 according to the elevation offset, and at the same time, the heights of the other points among the n 3D spatial points will also be offset according to the elevation offset.

[0184] Optionally, method 300 may further include storing attribute information in an attribute file, the attribute information including: the number of layers of the first image, the elevation offset, and the value of n.

[0185] Optionally, method 300 may further include: generating a positioning layer based on the first image, the second image, and the attribute file.

[0186] It should be understood that since each 3D spatial point in a point cloud map contains position information and normal vector information, and these two information are stored in different images during the rasterization process, it is necessary to establish association rules for them in order to be used when restoring the point cloud map.

[0187] Therefore, method 300 may further include: constructing the relationship between the position information and normal vector information of each 3D spatial point.

[0188] Optionally, constructing the association between the position information and normal vector information of each 3D spatial point includes: obtaining a key based on the principal scale of each 3D spatial point and the size of the first image; obtaining a key value based on the target normal vector information of each 3D spatial point; and generating key-value pairs based on the key and key value of each 3D spatial point.

[0189] Optionally, after constructing the association between the position information and normal vector information of each 3D spatial point, method 300 may further include: sorting the keys sequentially from smallest to largest in terms of layer, row, column, and bit dimensions.

[0190] Optionally, method 300 may also include: recovering a 3D point cloud map based on the positioning layer and the association relationship.

[0191] It should be understood that the process of restoring the 3D point cloud map can be completed by the 3D LiDAR positioning module 230 in the aforementioned system architecture 200.

[0192] Optionally, restoring the 3D point cloud map based on the positioning layer and the association relationship includes: obtaining attribute information from the attribute file; restoring n 3D spatial points sequentially from the smallest to the largest dimensions of layer, row, column, and bit according to the first image; restoring the position information and normal vector information corresponding to the n 3D spatial points one by one according to the target normal vector information and sub-scale stored in the second image based on the association relationship; and restoring the 3D point cloud map based on the position information and normal vector information of the n 3D spatial points.

[0193] It should be understood that the process of restoring the 3D point cloud map can also be found in the specific implementation description below.

[0194] The following will combine Figures 4 to 11 The specific implementation method of processing point cloud data in this application is described in detail.

[0195] Figure 4 This is an exemplary flowchart of a point cloud data processing embodiment provided in this application. Figure 5 This is an exemplary flowchart of another point cloud data processing method provided in an embodiment of this application. Figure 4As shown, the method 400 includes steps S410 to S460. It should be understood that the embodiments of this application do not limit the order of the above steps; any solution that can implement this application through any order of the above steps falls within the protection scope of this application. These steps are described in detail below.

[0196] S410 generates a point cloud tile map containing normal vector information.

[0197] First, it is necessary to obtain the original laser point cloud map (i.e., the initial 3D point cloud map) of the target area. For example, the target area can be as follows: Figure 6 As shown ( Figure 6 The target area in the image is a laser point cloud map of a certain park and its underground parking lot. The actual size of the target area is 500m × 800m.

[0198] In this embodiment, the original laser point cloud map of the target area can first undergo preliminary downsampling processing of 0.1m × 0.1m × 0.1m. And... Figure 6 The original laser point cloud map shown is approximately 140MB in size after initial downsampling.

[0199] After the initial downsampling, it can be based on Figure 7 The method shown processes the point cloud data in the map to obtain a point cloud tile map containing normal vector information.

[0200] The following is in conjunction with the appendix Figure 7 A method for generating point cloud tile maps containing normal vector information is described.

[0201] Figure 7 This is an example diagram illustrating a method for generating a point cloud tile map containing normal vector information, provided in an embodiment of this application. Figure 7 As shown, the method 700 includes steps S710 to S750, which are described in detail below.

[0202] S710, Estimate the normal vector.

[0203] Optionally, in this embodiment, the normal vectors of 3D spatial points in the point cloud map after preliminary downsampling can be estimated using principal component analysis. It should be understood that the normal vectors can also be estimated using other methods in the prior art, and this application does not limit this to such methods.

[0204] S720, extract surface features (i.e., extract feature points).

[0205] After estimating the normal vectors of 3D spatial points in the point cloud map, we can retain only the normal vectors with small covariance and a magnitude close to 1.

[0206] In this embodiment, the operation of retaining normal vectors with small covariance and a magnitude close to 1 can be denoted as extracting surface features or extracting feature points, because normal vectors that meet this condition can represent more reliable surface features.

[0207] In this embodiment, feature extraction further reduces the size of the point cloud map after initial downsampling from 140MB to 26MB.

[0208] In this embodiment, for a 500m×800m park, the sparsity of the point cloud map is ensured by surface feature extraction (i.e., the amount of point cloud data in the point cloud map is greatly reduced), which significantly reduces the storage space of the point cloud map, while retaining important feature points.

[0209] S730, split map tiles.

[0210] For example, in this embodiment, the positioning layer can be organized in the form of a tile map with a size of 80m×80m, dividing the original point cloud map containing surface features into a tile map of a specified size.

[0211] S740, overlapping space merging.

[0212] Because there is spatial overlap during the process of segmenting point cloud maps into tile maps, it is necessary to merge multiple tile maps at the same location.

[0213] S750, downsampling.

[0214] Downsampling is achieved by using a three-dimensional voxel grid, ensuring that only one point closest to the centroid is retained within each voxel as the sampling point. For example, the three-dimensional voxel grid size is 0.5m × 0.5m × 0.5m.

[0215] Through the above steps S710 to S750, a 3D point cloud tile map containing normal vector feature information is obtained.

[0216] After generating a point cloud tile map containing normal vector information, steps S420 to S440 can be combined to generate a positioning layer with a high compression ratio. Steps S420 to S440 are described below. It should be understood that, for ease of description, the point cloud tile map containing normal vector information will be directly referred to as a point cloud map in the following text.

[0217] S420, feature point location information compression.

[0218] In this embodiment, the compression process of feature point location information is roughly as follows: the location information of each feature point is mapped to an integer vector by rasterization, so that the location information of each feature point corresponds to a bit of the grayscale information of a pixel at a certain position in a single-channel image (that is, the location information is rasterized into the main scale); then, the existing PNG image compression algorithm is used to achieve efficient compression of the location information and store it in the file [tag_layer.png] (that is, the main scale is stored in the first image).

[0219] Figure 8 This is an example diagram of location information rasterization provided in an embodiment of this application. Figure 8 In the diagram, (a) represents n unordered point clouds (i.e., feature points) in the point cloud map, where p i Let p represent the i-th feature point in an n-point cloud. It should be understood that, in the following text, p will be used as the reference point. i The rasterization of feature point location information is introduced using an example. It should also be understood that in this embodiment, the rasterization grid resolution is 0.5m, but it should be understood that this resolution is only an example and does not constitute a limitation of this application.

[0220] Figure 8 (b) in the image represents a multi-layer 8-bit single-channel PNG image, meaning that each layer contains 8 bits. Figure 8 In (c), the 8 bits contained in the gray layer in (b) are shown. It can be seen that each bit corresponds to a resolution z_major_resolution in the Z direction to represent elevation information. Furthermore, in this embodiment, the main scale precision z_major_resolution in the Z direction can be 0.5m.

[0221] Then, feature point p i The corresponding image layer information (i.e., the target image layer) is:

[0222] layer = Round(p iz / (8×z_major_resolution))

[0223] Where Round means rounding, p iz Representing feature point p i Height in the Z direction.

[0224] Once the image layer number is obtained, the feature point p is determined. i The target image layer to which it belongs. Let's say... Figure 8 As shown in (b), this image layer is marked in gray. This is because each image channel contains 8 bits, as... Figure 8 As shown in (c), the next step is to determine the feature point p.i The bit index major_z in the target image layer is calculated as follows:

[0225] major_z=Round((p iz -layer×8×z_major_resolution) / z_major_resolution))

[0226] Similarly, calculate the feature point p. i In the target image layer, the pixel coordinates, i.e., rows and columns:

[0227] major_r = Round(p iy / y_major_resolution)

[0228] major_c = Round(p ix / x_major_resolution)

[0229] It should be understood that in this example, x_major_resolution and y_major_resolution are also 0.5m.

[0230] The feature point p can be determined through the above calculations. i The bit that belongs to the major_r-th row, major_c-th column, and major_z-th bit of the image in the layer, such as... Figure 8 As shown in (d) in the figure.

[0231] It should be understood that the above layer, major_r, major_c, and major_z are the main scales obtained by rasterizing the position information.

[0232] Using the method described above, all feature points are traversed and rasterized into multi-layer 8-bit single-channel PNG images. Then, existing PNG image compression algorithms are used to efficiently compress the location information and store it in the file [tag_layer.png].

[0233] Optionally, in this embodiment, to ensure that the z value is calculated starting from a value close to zero, then p iz The offset from start_z is required; that is, the z-values ​​(height values) of all feature points need to be offset by start_z. Here, start_z is determined by the minimum Z-value min in the point cloud map. z Find:

[0234] start_z = Floor(min z / z_min_resolution)×z_min_resolution

[0235] Floor represents rounding down.

[0236] It should be understood that there is usually a loss of precision when rasterizing to the above main scale, for example, when feature points p... i The process of rasterizing the location information from floating-point values ​​to pixel indices in the image coordinate system introduces errors. For a resolution of 0.5m, this error can reach a maximum of 0.25m. A large error will inevitably reduce the positioning performance of the positioning module. Usually, the error introduced during rasterization can be reduced by increasing the resolution (e.g., 0.2m resolution), but increasing the resolution will cause the image size to increase exponentially (the image size will be 6.25 times that of the previous size).

[0237] Therefore, to reduce the errors introduced by rasterization, this embodiment proposes a method to more finely depict the main scale using a sub-scale approach. This sub-scale approach typically only requires an additional 1 bit to double the raster resolution. The following section combines... Figure 9 The approach to subscale representation is explained in detail.

[0238] Figure 9 This is an example diagram of a main and secondary scale provided in an embodiment of this application. For example... Figure 9 As shown, for a 0.5m × 0.5m × 0.5m voxel grid, let the black solid dots be feature points and the gray solid dots be the centers of the voxel grid. Direct rasterization of feature points will use the gray solid dots in the figure to represent the feature point positions. In this case, if 1 bit is allocated to the sub-scale (meaning 1 bit is allocated in each of the XYZ directions), the main scale in each direction can be further divided in two, reducing the size of the voxel grid to 0.25m × 0.25m × 0.25m. Figure 9 As shown, the nearest black hollow dot is used to represent the feature point during rasterization. Clearly, compared to gray solid dots, black hollow dots can represent the feature point position more accurately, and the maximum error is reduced from 0.25m to 0.125m.

[0239] In this embodiment, it is recommended to allocate 2 bits to the sub-scale (i.e., 2 bits in each of the XYZ directions). This further divides the main scale in each direction into 4 parts, ensuring that the error introduced by the rasterization process does not exceed 0.06m. The sub-scale is then calculated using the following formula, where the % symbol represents the modulo operation:

[0240] minor_c = Round(p ix / x_min_resolution)%4

[0241] minor_r = Round(p iy / y_min_resolution)%4

[0242] minor_z = Round(p iz / z_min_resolution)%4

[0243] Furthermore, the sub-scales (minor_c, minor_r, and minor_z) of each feature point require a total of 6 bits.

[0244] It should be understood that in this embodiment, the sub-tick information in the feature point location information will not be saved to the aforementioned file [tag_layer.png], but will be saved together with the normal vector information in the file [tag_n.png] (i.e., in the second image). Specifically, please refer to the description of step S440 below.

[0245] S430, feature point normal vector information compression.

[0246] In this embodiment, the feature point normal vector information is compressed mainly based on the following two considerations:

[0247] On the one hand: since the normal vector has a magnitude of 1, only two of the normal vector components need to be stored to recover the third component.

[0248] This embodiment will take storing x′ and y′ components as an example. During the recovery process, the components can be recovered using the condition that the modulus is 1. The value of . Additionally, it's important to note that the sign information of z′ cannot be obtained when recovering z′. Therefore, it's necessary to ensure that the sign of z′ is always positive to solve this problem. Thus, the following operation needs to be performed on the normal vector:

[0249] x′=-x′, y′=-y′, z′=-z′; z′<0

[0250] On the other hand, since the normal vector information also needs to be saved in the image and compressed efficiently using existing image compression algorithms, the floating-point normal vector information must also be rasterized into integer values.

[0251] For the x′ or y′ components of the normal vector, which vary within the range of [-1, 1], the impact of normal vector accuracy on laser positioning performance was evaluated. Optionally, in this embodiment, 5 bits can be used to store one normal vector component, in which case the maximum error introduced by rasterization is 0.03. Therefore, 10 bits are needed to represent the normal vector information of a feature point.

[0252] For information on storing the subscale values ​​in the normal vector and position information, please refer to step S440.

[0253] S440, save as a positioning layer.

[0254] Based on the location information compression algorithm and normal vector information compression algorithm described above, we traverse n feature points in the point cloud data and save them in the following three files respectively.

[0255] First: the file [tag_layer.png].

[0256] Position information (excluding sub-scales) is compressed to obtain a multi-layer single-channel PNG image. Figure 8 As shown in the image, the naming format is

tag_layer.png

tag_layer.png

[0257] layer_bit|=(1<<layer)

[0258] In this context, the symbol “|” represents the bitwise AND operation, and “<<” represents the bitwise shift operation.

[0259] Second: the file [tag_n.png].

[0260] The sub-scale information in the location information will be stored together with the normal vector information. Since the normal vector information of a feature point occupies 10 bits (its components x′ and y′ each occupy 5 bits), and the sub-scale information occupies 6 bits, a feature point requires a total of 16 bits:

[0261] (x′<<3|y′>>2,y′<<6|minor_z<<4|minor_r<<2|minor_c)

[0262] This embodiment uses a 16-bit tiled image onto an 8-bit single-channel image, requiring only two pixel locations (each pixel contains 8 bits) to store the normal vector information and sub-tick of a feature point. If the point cloud map contains n feature points, then 2n pixels are needed for storage. Furthermore, the size of the image [tag_n.png] used to store the normal vector information and sub-tick is:

[0263]

[0264] col = Floor(2n / row)

[0265] Where row represents the height of the image and col represents the width of the image.

[0266] Third: The file

tag.txt

[0267] After traversing all n feature points of the point cloud map, the layer number of the PNG image (i.e., the layer number of the first image), the z-value offset (i.e., the elevation offset start_z), and the number of feature points n can be obtained and saved to the attribute information file [tag.txt].

[0268] S450, establish the association between position information and normal vector information.

[0269] Since the feature points in a point cloud map contain both location information and normal vector information, and these two information are stored in different PNG images during compression, it is necessary to establish association rules between them for use when restoring the point cloud map.

[0270] Specifically, the key is generated based on the main tick of the feature points and the size of [tag_layer.png]:

[0271] key=layer*SIZE*SIZE*8+major_r*SIZE*8+major_c*8+major_z

[0272] SIZE represents the size of [tag_layer.png].

[0273] The value corresponding to this key is the normal vector of that feature point, and the two form a key-value pair.<key,normal> After the feature points have been traversed, the keys are sorted in ascending order, specifically from the layers, major_r, major_c, and major_z dimensions in ascending order. This clarifies the association rules between the normal vectors and the position information.

[0274] S460, locate the layer to restore the point cloud map.

[0275] When restoring a point cloud map from a localized layer, the first step is to retrieve the attribute information of the point cloud file to be restored from the tag.txt file. This includes the PNG image layer number for location information, the z-value offset, and the number of feature points (n). Then, the feature points are restored sequentially from the smallest to the largest dimensions: layer, major_r, major_c, and major_z.

[0276] The existence of a layer is determined by its layer. For a given layer image, rows, columns, and bits are further traversed. If a feature point exists at that location, the corresponding bit is set to 1. At this point, the layer, major_c, major_r, and major_z information can be obtained. Simultaneously, the normal vector components x′ and y′ of the feature point and the sub-tick information minor_x, minor_y, and minor_z are obtained from [tag_n.png]. By inversely transforming back according to the compression algorithm described above, the position information and normal vector information of the feature point can be obtained. Finally, based on the recovered position information and normal vector information, a point cloud map with normal vector feature information is reconstructed for laser positioning.

[0277] In this embodiment, the above method enables point cloud maps to be loaded, transmitted, and processed in a lightweight manner, and the error introduced by rasterization during map compression is within an acceptable range.

[0278] Furthermore, the point cloud map can be easily and accurately recovered using the aforementioned method for recovering point cloud maps.

[0279] The compression efficiency of the proposed solution is verified below using point cloud data (140MB in size) of a certain park and its underground parking lot (500m×800m) after initial downsampling. Furthermore, measured data is used to evaluate the impact of point cloud maps based on the initially downsampled point cloud and those compressed using the aforementioned solution on laser positioning performance.

[0280] Specifically, according to verification, in the above embodiment, the data size of the point cloud file containing normal vector feature information obtained by extracting surface features is reduced to 26MB. Then, the compression algorithm proposed in this application is applied to the downsampled point cloud file to obtain the final compressed positioning layer, and the map data size is further reduced to 2.2MB (of which, [tag.txt] is 415 bytes; [tag_n.png] is 1.8MB; and [tag_layer.png] is 391KB).

[0281] Experimental results show that the compression efficiency of the proposed scheme is approximately 64:1 compared to the point cloud map (voxel grid size 0.1m×0.1m×0.1m) after initial downsampling; and approximately 12:1 compared to the point cloud map after surface feature extraction, thus ensuring that the point cloud map can be loaded, transmitted and processed in a lightweight manner.

[0282] Furthermore, this application also evaluates the difference in localization performance between a point cloud compressed layer (i.e., a localization layer obtained by compression using the method described in this application) and a point cloud map after initial downsampling. For example... Figure 10The image shows the positioning error distribution based on the point cloud compressed layer (i.e., the positioning layer obtained by compression according to the scheme of this application). Figure 11 This is a localization error distribution map based on the point cloud map after initial downsampling. Figure 10 and Figure 11 The horizontal axis represents the error (unit: m), and the vertical axis represents the number of errors. More detailed quantitative indicators can be found in Table 1.

[0283] Table 1: Statistical Table of Positioning Accuracy Indicators for Different Positioning Map Categories

[0284]

[0285] Experimental results show that the positioning performance of the point cloud compressed positioning layer is basically consistent with that of the point cloud map after initial downsampling, thus proving that the error introduced by rasterization in the map compression process of this application is within an acceptable range.

[0286] Figure 12 This is an example diagram of a device for processing point cloud data provided in an embodiment of this application. Figure 12 As shown, the device 1200 includes an acquisition unit 1210 and a processing unit 1220.

[0287] The acquisition unit 1210 is used to acquire a three-dimensional 3D point cloud map, which includes n 3D spatial points. Each of the n 3D spatial points includes position information and normal vector information, where n is a positive integer.

[0288] The processing unit 1220 is used to store the main scale of each 3D spatial point in the first image. The main scale is obtained by rasterizing the position information and corresponds to the pixel coordinate value in the first image.

[0289] The processing unit 1220 is also used to store the target normal vector information of each 3D spatial point in the second image, which is obtained by rasterizing the normal vector information.

[0290] Optionally, the target normal vector information can be obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction respectively.

[0291] Optionally, the processing unit 1220 can also be used to store the sub-scale and target normal vector information of each 3D spatial point in a second image, wherein the sub-scale is obtained by rasterizing the position information.

[0292] Optionally, the first image can be a multi-layer single-channel image, each layer of which includes 8 bits, each of which corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

[0293] Optionally, the second image can be a single-channel image, where the sub-scale and target normal vector information of each 3D spatial point occupies two pixels in the second image.

[0294] Optionally, the sub-tick of each 3D spatial point can occupy 6 bits of the two pixels, and the target normal vector information of each 3D spatial point can occupy 10 bits of the two pixels.

[0295] Optionally, the resolution of the main scale can be an integer multiple of the resolution of the subscale.

[0296] Optionally, the processing unit 1220 can also be used to: calculate an elevation offset that makes the elevation offset of the 3D spatial point with the smallest height among the n 3D spatial points zero.

[0297] Optionally, the processing unit 1220 can also be used to store attribute information in an attribute file, the attribute information including the number of layers of the first image, the elevation offset, and the value of n.

[0298] Optionally, the processing unit 1220 can also be used to generate a positioning layer based on the first image, the second image, and the attribute file.

[0299] Optionally, the processing unit 1220 can also be used to: construct the association between the position information and normal vector information of each 3D spatial point.

[0300] Optionally, the processing unit 1220 may also be used to: obtain a key based on the principal scale of each 3D spatial point and the size of the first image; obtain a key value based on the target normal vector information of each 3D spatial point; and generate a key-value pair based on the key and key value of each 3D spatial point.

[0301] Optionally, the processing unit 1220 can also be used to sort the key sequentially from smallest to largest in terms of layer, row, column, and bit dimensions.

[0302] Optionally, the processing unit 1220 can also be used to: recover the 3D point cloud map based on the positioning layer and the association relationship.

[0303] Optionally, the processing unit 1220 can also be used to: obtain the attribute information from the attribute file; recover the n 3D spatial points sequentially from the smallest to the largest dimensions of layer, row, column, and bit according to the first image; recover the position information and normal vector information corresponding to the n 3D spatial points one by one according to the association relationship and the target normal vector information and sub-scale stored in the second image; and recover the 3D point cloud map according to the position information and normal vector information of the n 3D spatial points.

[0304] Optionally, the acquisition unit 1210 can also be used to: acquire an initial 3D point cloud map, the initial 3D point cloud map including m 3D spatial points, each of the m 3D spatial points including position information and normal vector information, where m is a positive integer; the processing unit 1220 can also be used to: calculate the normal vectors of the m 3D spatial points; extract the corresponding 3D spatial points whose covariance is less than or equal to a first threshold; and generate the 3D point cloud map based on the extracted 3D spatial points.

[0305] Optionally, the processing unit 1220 can also be used to calculate the normal vectors of the m 3D spatial points respectively using principal component analysis.

[0306] Optionally, the processing unit 1220 can also be used to: generate an intermediate 3D point cloud map based on the extracted 3D spatial points; segment the intermediate 3D point cloud map into tile maps; merge the tile maps in the overlapping areas; and downsample the merged tile maps to generate the 3D point cloud map.

[0307] Figure 13 This is an example diagram illustrating a method for recovering a 3D point cloud map provided in an embodiment of this application. The 3D point cloud map includes n 3D spatial points, each of which includes position information and normal vector information, where n is a positive integer. For example... Figure 13 As shown, the method 1300 includes steps S1310 to S1330, which will be described in detail below.

[0308] S1310, Get the positioning layer.

[0309] The positioning layer is generated based on the first image, the second image, and the attribute file.

[0310] The first image is used to store the principal scale of each 3D spatial point. The principal scale is obtained by rasterizing the position information and corresponds to the pixel coordinate value in the first image.

[0311] The second image is used to store the target normal vector information for each 3D spatial point. The target normal vector information is obtained by rasterizing the normal vector information.

[0312] The attribute file is used to store attribute information, which includes: the number of layers in the first image, the elevation offset, and the value of n.

[0313] S1320: Obtain the relationship between the position information and normal vector information of each 3D spatial point.

[0314] S1330, restores the 3D point cloud map based on the positioning layer and the relationship.

[0315] In this embodiment, the location layer and the correlation between the position information and normal vector information of each 3D spatial point are obtained, and the 3D point cloud map is recovered based on the location layer and the correlation. This allows for a simpler recovery of the position information and normal vector information of n 3D spatial points in the 3D point cloud map.

[0316] It should be understood that, in the embodiments of this application, the generation of the positioning layer and the construction of the association relationship can be completed by the cloud server, while the process of restoring the 3D point cloud map can be implemented by the vehicle. Specifically, the vehicle can first obtain the positioning layer and the association relationship from the cloud, and then restore the 3D point cloud map based on the positioning layer and the association relationship.

[0317] It should be understood that, if the vehicle has the relevant computing capabilities, the above operations can also be completed on the vehicle side; or, the cloud can also restore the 3D point cloud map and then send it to the vehicle side. This application does not limit this, and the specific method should be determined based on the actual situation.

[0318] Optionally, the first image is a multi-layer single-channel image, where each layer in the multi-layer image includes 8 bits, and each of the 8 bits corresponds to a resolution in the elevation direction. The main scale includes layers, rows, columns, and bits.

[0319] Optionally, restoring the 3D point cloud map based on the positioning layer and the association relationship includes: obtaining attribute information from the attribute file; restoring n 3D spatial points sequentially from the smallest to the largest dimensions of layer, row, column, and bit according to the first image; restoring the position information and normal vector information corresponding to the n 3D spatial points one by one according to the association relationship and the target normal vector information stored in the second image; and restoring the 3D point cloud map according to the position information and normal vector information of the n 3D spatial points.

[0320] Optionally, the second image is also used to store the sub-scale of each 3D spatial point, which is obtained by rasterizing the position information; the step of recovering the position information and normal vector information corresponding to n 3D spatial points one by one according to the association relationship and the target normal vector information stored in the second image includes: recovering the position information and normal vector information corresponding to n 3D spatial points one by one according to the association relationship and the target normal vector information and sub-scale stored in the second image.

[0321] In this embodiment, the second image is further used to store the sub-scale of each 3D spatial point, so that during the restoration process, the position information and normal vector information corresponding to n 3D spatial points can be restored one by one according to the correlation relationship and the target normal vector information and sub-scale stored in the second image. This makes it easier and more accurate to restore the 3D point cloud map, thereby improving the positioning performance of the positioning module.

[0322] Optionally, obtaining the association between the position information and normal vector information of each 3D spatial point includes: obtaining the key-value pair corresponding to each 3D spatial point, wherein the key-value pair corresponding to each 3D spatial point is generated by the key and key-value pair corresponding to each 3D spatial point, the key of each 3D spatial point is obtained according to the principal scale of each 3D spatial point and the size of the first image, and the key-value pair of each 3D spatial point is obtained according to the target normal vector information of each 3D spatial point.

[0323] Optionally, the keys of the n 3D spatial points are sorted in ascending order of the dimensions of layer, row, column, and bit.

[0324] Optionally, the resolution of the main scale is an integer multiple of the resolution of the subscale.

[0325] Optionally, the elevation offset can be configured such that the elevation offset of the 3D spatial point with the smallest height among the n 3D spatial points is 0.

[0326] In other words, the elevation offset is the height value of the 3D spatial point with the smallest height among n 3D spatial points. The height of the 3D spatial point with the smallest height among n 3D spatial points can be offset to 0 according to the elevation offset, and at the same time, the heights of the other points among the n 3D spatial points will also be offset according to the elevation offset.

[0327] Optionally, the target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction, respectively.

[0328] Optionally, the second image is a single-channel image, with the sub-scale and target normal vector information of each 3D spatial point occupying two pixels in the second image.

[0329] Optionally, the sub-tick of each 3D spatial point occupies 6 bits out of two pixels, and the target normal vector information of each 3D spatial point occupies 10 bits out of two pixels.

[0330] It should be understood that for a detailed description of Method 1300 and other details, please refer to the relevant introduction in Method 300, which will not be repeated here.

[0331] Figure 14This is an example diagram of a device for recovering a 3D point cloud map provided in an embodiment of this application. The 3D point cloud map includes n 3D spatial points, each of which includes position information and normal vector information, where n is a positive integer. Figure 14 As shown, the device 1400 includes an acquisition unit 1410 and a processing unit 1420.

[0332] Among them, the acquisition unit 1410 is used to acquire the positioning layer.

[0333] The positioning layer is generated based on the first image, the second image, and the attribute file. The first image stores the principal scale of each 3D spatial point. The principal scale is obtained by rasterizing the position information, and it corresponds to the pixel coordinates in the first image. The second image stores the target normal vector information of each 3D spatial point. The target normal vector information is obtained by rasterizing the normal vector information. The attribute file stores attribute information, including: the layer number of the first image, the elevation offset, and the value of n.

[0334] The acquisition unit 1410 is also used to acquire the relationship between the position information and normal vector information of each 3D spatial point.

[0335] Processing unit 1420 is used to recover a 3D point cloud map based on the positioning layer and the relationship.

[0336] Optionally, the first image can be a multi-layer single-channel image, where each layer in the multi-layer image includes 8 bits, and each of the 8 bits corresponds to a resolution in the elevation direction. The main scale includes layers, rows, columns, and bits.

[0337] Optionally, the processing unit 1420 can also be used to: obtain attribute information from an attribute file; recover n 3D spatial points from the first image in ascending order of layer, row, column, and bit dimensions; recover the position information and normal vector information corresponding to the n 3D spatial points one by one according to the association relationship and the target normal vector information stored in the second image; and recover a 3D point cloud map according to the position information and normal vector information of the n 3D spatial points.

[0338] Optionally, the second image can also be used to store the sub-scale of each 3D spatial point, which is obtained by rasterizing the position information. The processing unit 1420 can also be used to: recover the position information and normal vector information corresponding to n 3D spatial points one by one according to the correlation relationship and the target normal vector information and sub-scale stored in the second image.

[0339] Optionally, the acquisition unit 1410 can also be used to: acquire key-value pairs corresponding to each 3D spatial point, wherein the key-value pairs corresponding to each 3D spatial point are generated by the key and key-value pairs corresponding to each 3D spatial point, the key of each 3D spatial point is obtained based on the principal scale of each 3D spatial point and the size of the first image, and the key-value pairs of each 3D spatial point are obtained based on the target normal vector information of each 3D spatial point.

[0340] Optionally, the keys of the n 3D spatial points can be sorted sequentially from smallest to largest in terms of layer, row, column, and bit dimensions.

[0341] Optionally, the resolution of the main scale can be an integer multiple of the resolution of the subscale.

[0342] Optionally, the elevation offset can be configured such that the elevation offset of the 3D spatial point with the smallest height among the n 3D spatial points is 0.

[0343] Optionally, the target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction, respectively.

[0344] Optionally, the second image can be a single-channel image, where the sub-scale and target normal vector information of each 3D spatial point can occupy two pixels in the second image.

[0345] Optionally, the sub-tick of each 3D spatial point can occupy 6 bits out of two pixels, and the target normal vector information of each 3D spatial point can occupy 10 bits out of two pixels.

[0346] Figure 15 This is an exemplary block diagram of the hardware structure of a device provided in an embodiment of this application. The device 1500 (specifically, it can be a computer device) includes a memory 1510, a processor 1520, a communication interface 1530, and a bus 1540. The memory 1510, the processor 1520, and the communication interface 1530 are interconnected via the bus 1540.

[0347] The memory 1510 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1510 may store a program. When the program stored in the memory 1510 is executed by the processor 1520, the processor 1520 is used to execute various steps of the method for processing point cloud data according to the embodiments of this application; or, to execute various steps of the method for restoring a 3D point cloud map according to the embodiments of this application.

[0348] The processor 1520 may be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute related programs to implement the method of processing point cloud data in the method embodiment of this application; or to implement the method of restoring 3D point cloud map in the method embodiment of this application.

[0349] The processor 1520 can also be an integrated circuit chip with signal processing capabilities. In implementation, the method for processing point cloud data or reconstructing a 3D point cloud map described in this application can be accomplished through integrated logic circuits in the processor 1520 or through software instructions.

[0350] The processor 1520 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 1510. The processor 1520 reads the information in memory 1510 and, in conjunction with its hardware, completes the functions required by the modules included in the apparatus of the embodiments of this application, or executes the method for processing point cloud data in the method embodiments of this application; or executes the method for restoring 3D point cloud maps in the method embodiments of this application.

[0351] The communication interface 1530 uses a transceiver device, such as, but not limited to, a transceiver, to enable communication between the device 1500 and other devices or communication networks.

[0352] Bus 1540 may include a pathway for transmitting information between various components of device 1500 (e.g., memory 1510, processor 1520, communication interface 1530).

[0353] This application also provides a vehicle that includes modules for performing any of the methods described above for restoring 3D point cloud maps.

[0354] Optionally, if the vehicle has sufficient computing power, it may also include modules for performing any of the methods described above for processing point cloud data; this application does not limit this.

[0355] Optionally, the vehicles involved in this application may be traditional internal combustion engine vehicles, hybrid vehicles, pure electric vehicles, centralized drive vehicles, and distributed drive vehicles, etc., and this application does not limit them.

[0356] For example, Figure 16 This is a functional block diagram of a vehicle applicable to an embodiment of this application. The vehicle 100 can be a manually driven vehicle, or the vehicle 100 can be configured for fully or partially automated driving modes.

[0357] In one example, vehicle 100 can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human intervention, determine the possible behaviors of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of the other vehicle performing the possible behavior, and control vehicle 100 based on the determined information. When vehicle 100 is in autonomous driving mode, vehicle 100 can be set to operate without human interaction.

[0358] The vehicle 100 may include various subsystems, such as a driving system 110, a sensing system 120, a control system 130, one or more peripheral devices 140, a power supply 160, a computer system 150, and a user interface 170.

[0359] Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0360] Exemplarily, the mobility system 110 may include components for providing powered motion to the vehicle 100. In one embodiment, the mobility system 110 may include an engine 111, a transmission 112, an energy source 113, and wheels 114 / tires. The engine 111 may be an internal combustion engine, an electric motor, an air compressor engine, or other combinations of engines; for example, a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compressor engine. The engine 111 can convert the energy source 113 into mechanical energy.

[0361] For example, energy source 113 may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 113 may also provide energy to other systems of vehicle 100.

[0362] For example, the transmission 112 may include a gearbox, a differential, and a drive shaft; wherein the transmission 112 can transmit mechanical power from the engine 111 to the wheels 114.

[0363] In one embodiment, the transmission 112 may also include other components, such as a clutch. The drive shaft may include one or more shafts that can be coupled to one or more wheels 114.

[0364] For example, the sensing system 120 may include several sensors for sensing information about the environment surrounding the vehicle 100.

[0365] For example, the sensing system 120 may include a positioning system 121 (e.g., a global positioning system, BeiDou system, or other positioning system), an inertial measurement unit (IMU) 122, a radar 123, a laser rangefinder 124, a camera 125, and a vehicle speed sensor 126. The sensing system 120 may also include sensors from the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the autonomous vehicle 100.

[0366] The positioning system 121 can be used to estimate the geographical location of the vehicle 100. The IMU 122 can be used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 122 can be a combination of an accelerometer and a gyroscope.

[0367] For example, radar 123 can use radio information to sense objects in the surrounding environment of vehicle 100. In some embodiments, in addition to sensing objects, radar 123 can also be used to sense the speed and / or direction of travel of objects.

[0368] For example, the laser rangefinder 124 can use a laser to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 124 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0369] For example, camera 125 can be used to capture multiple images of the surrounding environment of vehicle 100. For example, camera 125 can be a still camera or a video camera.

[0370] For example, the vehicle speed sensor 126 can be used to measure the speed of the vehicle 100. For instance, the vehicle speed can be measured in real time. The measured vehicle speed can be transmitted to the control system 130 to control the vehicle.

[0371] like Figure 16 As shown, the control system 130 controls the operation of the vehicle 100 and its components. The control system 130 may include various components, such as a steering system 131, an accelerator 132, a braking unit 133, a computer vision system 134, a route control system 135, and an obstacle avoidance system 136.

[0372] For example, the steering system 131 can be operated to adjust the forward direction of the vehicle 100. For example, in one embodiment, it can be a steering wheel system. The throttle 132 can be used to control the operating speed of the engine 111 and thus the speed of the vehicle 100.

[0373] For example, braking unit 133 can be used to control the deceleration of vehicle 100; braking unit 133 can use friction to slow down wheel 114. In other embodiments, braking unit 133 can convert the kinetic energy of wheel 114 into electric current. Braking unit 133 can also take other forms to slow down the rotational speed of wheel 114 to control the speed of vehicle 100.

[0374] like Figure 16 As shown, the computer vision system 134 is operable to process and analyze images captured by the camera 125 to identify objects and / or features in the environment surrounding the vehicle 100. These objects and / or features may include traffic information, road boundaries, and obstacles. The computer vision system 134 may use object recognition algorithms, structure-from-motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 134 may be used to map the environment, track objects, estimate object velocities, and so on.

[0375] For example, the route control system 135 can be used to determine the driving route of the vehicle 100. In some embodiments, the route control system 135 can combine data from sensors, GPS, and one or more predetermined maps to determine the driving route of the vehicle 100.

[0376] like Figure 16 As shown, obstacle avoidance system 136 can be used to identify, assess and avoid or otherwise traverse potential obstacles in the environment of vehicle 100.

[0377] In one instance, the control system 130 may include additional or alternative components besides those shown and described. Alternatively, some of the components shown above may be reduced.

[0378] like Figure 16 As shown, vehicle 100 can interact with external sensors, other vehicles, other computer systems or users through peripheral device 140; wherein peripheral device 140 may include wireless communication system 141, on-board computer 142, microphone 143 and / or speaker 144.

[0379] In some embodiments, peripheral device 140 may provide a means for vehicle 100 to interact with user interface 170. For example, on-board computer 142 may provide information to users of vehicle 100. User interface 116 may also operate on-board computer 142 to receive user input; on-board computer 142 may be operated via touchscreen. In other cases, peripheral device 140 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 143 may receive audio (e.g., voice commands or other audio input) from users of vehicle 100. Similarly, speaker 144 may output audio to users of vehicle 100.

[0380] like Figure 16 As shown, the wireless communication system 141 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 141 can use 3G cellular communication; such as code division multiple access (CDMA), EVDO, Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or 4G cellular communication, such as long term evolution (LTE); or 5G cellular communication. The wireless communication system 141 can also communicate using Wi-Fi and wireless local area networks (WLANs).

[0381] In some embodiments, the wireless communication system 141 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee protocol; other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system 141 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between the vehicle and / or roadside stations.

[0382] like Figure 16 As shown, power source 160 can provide power to various components of vehicle 100. In one embodiment, power source 160 can be a rechargeable lithium-ion battery or a lead-acid battery. One or more such battery packs can be configured to provide power to various components of vehicle 100. In some embodiments, power source 160 and energy source 113 can be implemented together, as is the case in some fully electric vehicles.

[0383] For example, some or all of the functions of vehicle 100 may be controlled by computer system 150, wherein computer system 150 may include at least one processor 151 that executes instructions 153 stored in a non-transitory computer-readable medium, such as memory 152. Computer system 150 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed manner.

[0384] For example, processor 151 can be any conventional processor, such as a commercially available central processing unit (CPU).

[0385] Alternatively, the processor can be a dedicated device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 16 The illustrations functionally depict a processor, memory, and other components of a computer within the same block; however, those skilled in the art will understand that the processor, computer, or memory may actually include multiple processors, computers, or memories that may or may not be stored in the same physical enclosure. For example, memory may be a hard disk drive or other storage media located in an enclosure different from that of the computer. Therefore, references to a processor or computer will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration assemblies, may each have their own processor that performs calculations only relevant to the component's specific function.

[0386] In the various aspects described herein, the processor may be located remotely from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor located within the vehicle, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0387] In some embodiments, memory 152 may contain instructions 153 (e.g., program logic) that can be used by processor 151 to perform various functions of vehicle 100, including those described above. Memory 152 may also include additional instructions, such as instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the mobility system 110, sensing system 120, control system 130, and peripheral devices 140.

[0388] For example, in addition to instruction 153, memory 152 may also store data, such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information can be used by vehicle 100 and computer system 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0389] like Figure 16 As shown, the user interface 170 can be used to provide information to or receive information from a user of the vehicle 100. Optionally, the user interface 170 may include one or more input / output devices within a set of peripheral devices 140, such as a wireless communication system 141, an on-board computer 142, a microphone 143, and a speaker 144.

[0390] In embodiments of this application, computer system 150 can control the functions of vehicle 100 based on input received from various subsystems (e.g., mobility system 110, sensing system 120, and control system 130) and from user interface 170. For example, computer system 150 can utilize input from control system 130 to control braking unit 133 to avoid obstacles detected by sensing system 120 and obstacle avoidance system 136. In some embodiments, computer system 150 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0391] Alternatively, one or more of these components may be installed separately from or associated with vehicle 100. For example, memory 152 may exist partially or completely separately from vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0392] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 16 This should not be construed as a limitation on the embodiments of this application.

[0393] Optionally, vehicle 100 may be an autonomous vehicle traveling on a road, capable of identifying objects in its surrounding environment to determine adjustments to its current speed. These objects may be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object may be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the autonomous vehicle can be determined.

[0394] Optionally, the vehicle 100 or a computing device associated with the vehicle 100 (such as...) Figure 16 The computer system 150, computer vision system 134, and memory 152 can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.).

[0395] Optionally, since each identified object depends on the behavior of others, the behavior of a single identified object can also be predicted by considering all identified objects together. Vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine, based on the predicted behavior of the objects, that the vehicle will need to adjust to a steady state (e.g., accelerate, decelerate, or stop). In this process, other factors can also be considered in determining the speed of vehicle 100, such as the lateral position of vehicle 100 on the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0396] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0397] The aforementioned vehicle 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0398] In one possible implementation, the above Figure 16 The vehicle 100 shown may be an autonomous vehicle. The autonomous driving system is described in detail below.

[0399] Figure 17 This is an example diagram of an autonomous driving system applicable to an embodiment of this application. For example... Figure 17The illustrated autonomous driving system includes a computer system 201, which includes a processor 203 coupled to a system bus 205. The processor 203 can be one or more processors, each of which may include one or more processor cores. A display adapter 207 drives a display 209, which is coupled to the system bus 205. The system bus 205 is coupled to an input / output (I / O) bus 213 via a bus bridge 211, and an I / O interface 215 is coupled to the I / O bus. The I / O interface 215 communicates with various I / O devices, such as input devices 217 (e.g., keyboard, mouse, touchscreen), and media trays 221 (e.g., CD-ROM, multimedia interfaces). A transceiver 223 can send and / or receive radio communication information, and a camera 255 can capture static and dynamic digital video images. The interface connected to the I / O interface 215 may be a USB port 225.

[0400] The processor 203 can be any conventional processor, such as a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination thereof.

[0401] Alternatively, processor 203 may be a dedicated device such as an application-specific integrated circuit (ASIC); processor 203 may be a neural network processor or a combination of a neural network processor and the aforementioned conventional processor.

[0402] Optionally, in some embodiments, the computer system 201 may be located remotely from the autonomous vehicle and may communicate wirelessly with the autonomous vehicle. In other aspects, some processes described in this application are executed on a processor located within the autonomous vehicle, while others are executed by a remote processor, including taking actions necessary to perform a single manipulation.

[0403] Computer system 201 can communicate with software deployment server 249 via network interface 229. Network interface 229 can be a hardware network interface, such as a network interface card (NIC). Network 227 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Optionally, network 227 can also be a wireless network, such as a WiFi network or a cellular network.

[0404] like Figure 17As shown, the hard disk drive interface is coupled to the system bus 205, the hardware driver interface 231 can be connected to the hard disk drive 233, and the system memory 235 is coupled to the system bus 205. Data running in the system memory 235 may include the operating system 237 and applications 243. The operating system 237 may include an interpreter (shell) 239 and a kernel 241. The shell 239 is an interface between the user and the operating system kernel. The shell can be the outermost layer of the operating system; the shell can manage the interaction between the user and the operating system, such as waiting for user input, interpreting user input for the operating system, and processing various operating system outputs. The kernel 241 may consist of the parts of the operating system used to manage memory, files, peripherals, and system resources. Interacting directly with the hardware, the operating system kernel typically runs processes and provides inter-process communication, CPU time slice management, interrupts, memory management, I / O management, etc. Application 243 includes programs related to controlling autonomous driving, such as programs managing the interaction between the autonomous vehicle and obstacles on the road, programs controlling the autonomous vehicle's route or speed, and programs controlling the interaction between the autonomous vehicle and other autonomous vehicles on the road. Application 243 also exists on the system of software deployment server 249. In one embodiment, when autonomous driving-related programs 247 need to be executed, computer system 201 can download the application from software deployment server 249.

[0405] For example, application 243 could also be a program that allows autonomous vehicles to interact with lane lines on the road, meaning a program that can track lane lines in real time.

[0406] For example, application 243 could also be a program that controls autonomous vehicles to perform automatic parking.

[0407] For example, sensor 253 may be associated with computer system 201 and may be used to detect the environment around computer 201.

[0408] For example, sensor 253 can detect lanes on the road, such as lane lines, and can track changes in lane lines within a certain range in front of the vehicle in real time as the vehicle moves (e.g., while driving). As another example, sensor 253 can detect animals, cars, obstacles, and pedestrian crossings. Furthermore, the sensor can also detect the environment surrounding these objects, such as the environment around the animal (e.g., other animals nearby), weather conditions, and ambient light levels.

[0409] Alternatively, if the computer 201 is located on an autonomous vehicle, the sensors may be cameras, infrared sensors, chemical detectors, microphones, etc.

[0410] For example, in a lane tracking scenario, sensor 253 can be used to detect lane lines in front of the vehicle, enabling the vehicle to perceive lane changes during travel and plan and adjust the vehicle's driving in real time accordingly.

[0411] For example, in an automatic parking scenario, sensor 253 can be used to detect the size or position of parking spaces and surrounding obstacles around the vehicle, thereby enabling the vehicle to perceive the distance between the parking spaces and surrounding obstacles, perform collision detection during parking, and prevent the vehicle from colliding with obstacles.

[0412] In one example Figure 16 The computer system 150 shown can also receive information from other computer systems or transfer information to other computer systems. Alternatively, sensor data collected from the sensing system 120 of the vehicle 100 can be transferred to another computer for processing, as described below. Figure 18 Let's take an example to illustrate.

[0413] Figure 18 This is an example diagram illustrating the application of cloud-based command-driven autonomous vehicles according to an embodiment of this application. For example... Figure 18 As shown, data from computer system 312 can be transmitted via a network to cloud-side server 32 for further processing; computer system 312 can also obtain data processed or stored by cloud-side server 32. The network and intermediate nodes can include various configurations and protocols, including the Internet, World Wide Web, intranet, virtual private network, wide area network, local area network, private network using proprietary communication protocols of one or more companies, Ethernet, WiFi, and HTTP, as well as various combinations thereof; such communication can be conducted by any device capable of transmitting data to and from other computers, such as modems and wireless interfaces.

[0414] In one example, server 32 may include a server with multiple computers, such as a load balancing server cluster, which exchanges information with different nodes in the network for the purpose of receiving, processing, and transmitting data from computer system 312. The server may be configured similarly to computer system 312, having a processor 33, memory 34, instruction set 35, and data processing unit 36.

[0415] Optionally, in this embodiment, the cloud-side server 32 may include modules for generating the association between the positioning layer and the position information and normal vector information of each 3D spatial point. The vehicle's computer system 312 can obtain relevant data from the cloud-side server 32 to reconstruct the 3D point cloud map.

[0416] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0417] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0418] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0419] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0420] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0421] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0422] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing point cloud data, characterized in that, The method is applied in the field of autonomous driving, including: A 3D point cloud map is obtained, wherein the 3D point cloud map includes n 3D spatial points, and each of the n 3D spatial points includes position information and normal vector information, where n is a positive integer; The principal scale of each 3D spatial point is stored in the first image. The principal scale is obtained by rasterizing the position information and corresponds to the pixel coordinate value in the first image. The subscale and target normal vector information of each 3D spatial point are stored in the second image. The subscale is obtained by rasterizing the position information, and the target normal vector information is obtained by rasterizing the normal vector information.

2. The method according to claim 1, characterized in that, The target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction, respectively.

3. The method according to claim 1 or 2, characterized in that, The first image is a multi-layer single-channel image, each layer of the multi-layer image includes 8 bits, each of the 8 bits corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

4. The method according to claim 1, characterized in that, The second image is a single-channel image, and the sub-scale and target normal vector information of each 3D spatial point occupy two pixel bits in the second image.

5. The method according to claim 4, characterized in that, The sub-scale of each 3D spatial point occupies 6 bits of the two pixel positions, and the target normal vector information of each 3D spatial point occupies 10 bits of the two pixel positions.

6. The method according to claim 1, characterized in that, The resolution of the main scale is an integer multiple of the resolution of the sub-scale.

7. The method according to claim 1 or 2, characterized in that, The method further includes: Calculate the elevation offset, which is the height value of the 3D spatial point with the smallest height among the n 3D spatial points.

8. The method according to claim 7, characterized in that, The method further includes: The attribute information is stored in an attribute file, and the attribute information includes: the number of layers of the first image, the elevation offset, and the value of n.

9. The method according to claim 8, characterized in that, The method further includes: A positioning layer is generated based on the first image, the second image, and the attribute file.

10. The method according to claim 9, characterized in that, The method further includes: Construct the association between the position information and normal vector information of each 3D spatial point.

11. The method according to claim 10, characterized in that, The process of constructing the association between the position information and normal vector information of each 3D spatial point includes: The key is obtained based on the principal scale of each 3D spatial point and the size of the first image; The key value is obtained based on the target normal vector information of each 3D spatial point; Generate key-value pairs based on the key and key-value of each 3D spatial point.

12. The method according to claim 11, characterized in that, After constructing the association between the position information and normal vector information of each 3D spatial point, the method further includes: The keys are sorted sequentially from smallest to largest in terms of layer, row, column, and bit.

13. The method according to claim 10, characterized in that, The method further includes: The 3D point cloud map is recovered based on the positioning layer and the association relationship.

14. The method according to claim 13, characterized in that, The step of restoring the 3D point cloud map based on the positioning layer and the association relationship includes: Obtain the attribute information from the attribute file; Based on the first image, the n 3D spatial points are recovered sequentially from the smallest to the largest dimensions of layer, row, column, and bit. Based on the aforementioned correlation and the target normal vector information and sub-scale information stored in the second image, the position information and normal vector information corresponding to the n 3D spatial points are recovered one by one; The 3D point cloud map is reconstructed based on the position information and normal vector information of the n 3D spatial points.

15. The method according to claim 1 or 2, characterized in that, The acquisition of the 3D point cloud map includes: Obtain an initial 3D point cloud map, which includes m 3D spatial points. Each of the m 3D spatial points includes position information and normal vector information, where m is a positive integer. Calculate the normal vectors of the m 3D space points; Extract the 3D space points corresponding to the normal vectors whose covariance is less than or equal to the first threshold; The 3D point cloud map is generated based on the extracted 3D spatial points.

16. The method according to claim 15, characterized in that, The calculation of the normal vectors of the m 3D space points includes: The normal vectors of the m 3D space points are calculated using principal component analysis.

17. The method according to claim 15, characterized in that, The step of generating the 3D point cloud map based on the extracted 3D spatial points includes: Generate an intermediate 3D point cloud map based on the extracted 3D spatial points; The intermediate 3D point cloud map is segmented into a tile map; Merge the tile maps in the overlapping areas; The merged tile map is downsampled to generate the 3D point cloud map.

18. An apparatus for processing point cloud data, characterized in that, The device is used in the field of autonomous driving and includes: The acquisition unit is used to acquire a three-dimensional 3D point cloud map, wherein the 3D point cloud map includes n 3D spatial points, and each of the n 3D spatial points includes position information and normal vector information, where n is a positive integer; The processing unit is configured to store the principal scale of each 3D spatial point in a first image, wherein the principal scale is obtained by rasterizing the position information and the principal scale corresponds to the pixel coordinate value in the first image; the processing unit is further configured to store the sub-scale and target normal vector information of each 3D spatial point in a second image, wherein the sub-scale is obtained by rasterizing the position information and the target normal vector information is obtained by rasterizing the normal vector information.

19. The apparatus according to claim 18, characterized in that, The target normal vector information is obtained by rasterizing the components of the normal vector information in the first direction and the components in the second direction, respectively.

20. The apparatus according to claim 18 or 19, characterized in that, The first image is a multi-layer single-channel image, each layer of the multi-layer image includes 8 bits, each of the 8 bits corresponds to a resolution in the elevation direction, and the main scale includes layers, rows, columns and bits.

21. The apparatus according to claim 18, characterized in that, The second image is a single-channel image, and the sub-scale and target normal vector information of each 3D spatial point occupy two pixel bits in the second image.

22. The apparatus according to claim 21, characterized in that, The sub-scale of each 3D spatial point occupies 6 bits of the two pixel positions, and the target normal vector information of each 3D spatial point occupies 10 bits of the two pixel positions.

23. The apparatus according to claim 18, characterized in that, The resolution of the main scale is an integer multiple of the resolution of the sub-scale.

24. The apparatus according to claim 18 or 19, characterized in that, The processing unit is also used for: Calculate the elevation offset, which is the height value of the 3D spatial point with the smallest height among the n 3D spatial points.

25. The apparatus according to claim 24, characterized in that, The processing unit is also used for: The attribute information is stored in an attribute file, and the attribute information includes: the number of layers of the first image, the elevation offset, and the value of n.

26. The apparatus according to claim 25, characterized in that, The processing unit is also used for: A positioning layer is generated based on the first image, the second image, and the attribute file.

27. The apparatus according to claim 26, characterized in that, The processing unit is also used for: Construct the association between the position information and normal vector information of each 3D spatial point.

28. The apparatus according to claim 27, characterized in that, The processing unit is also used for: The key is obtained based on the principal scale of each 3D spatial point and the size of the first image; The key value is obtained based on the target normal vector information of each 3D spatial point; Generate key-value pairs based on the key and key-value of each 3D spatial point.

29. The apparatus according to claim 28, characterized in that, The processing unit is also used for: The keys are sorted sequentially from smallest to largest in terms of layer, row, column, and bit.

30. The apparatus according to claim 27, characterized in that, The processing unit is also used for: The 3D point cloud map is recovered based on the positioning layer and the association relationship.

31. The apparatus according to claim 30, characterized in that, The processing unit is also used for: Obtain the attribute information from the attribute file; Based on the first image, the n 3D spatial points are recovered sequentially from the smallest to the largest dimensions of layer, row, column, and bit. Based on the aforementioned correlation and the target normal vector information and sub-scale information stored in the second image, the position information and normal vector information corresponding to the n 3D spatial points are recovered one by one; The 3D point cloud map is reconstructed based on the position information and normal vector information of the n 3D spatial points.

32. The apparatus according to claim 18 or 19, characterized in that, The acquisition unit is also used for: Obtain an initial 3D point cloud map, which includes m 3D spatial points. Each of the m 3D spatial points includes position information and normal vector information, where m is a positive integer. The processing unit is also used for: Calculate the normal vectors of the m 3D space points; Extract the 3D space points corresponding to the normal vectors whose covariance is less than or equal to the first threshold; The 3D point cloud map is generated based on the extracted 3D spatial points.

33. The apparatus according to claim 32, characterized in that, The processing unit is also used for: The normal vectors of the m 3D space points are calculated using principal component analysis.

34. The apparatus according to claim 32, characterized in that, The processing unit is also used for: Generate an intermediate 3D point cloud map based on the extracted 3D spatial points; The intermediate 3D point cloud map is segmented into a tile map; Merge the tile maps in the overlapping areas; The merged tile map is downsampled to generate the 3D point cloud map.

35. A computing device, characterized in that, include: At least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform the method as claimed in any one of claims 1 to 17.

36. A computer-readable storage medium, characterized in that, The computer-readable medium stores program code for execution by the device, the program code including instructions for performing the method as described in any one of claims 1 to 17.

Images