Laser radar point cloud compression method, system, device and medium
By using a sampling weight matrix-based sampling method in lidar point cloud compression, the problem of failing to fully utilize the density characteristics of point cloud in the prior art is solved, adaptive point cloud compression is achieved, and data expression efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202411648469.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-02-28
AI Technical Summary
The existing lidar point cloud compression method fails to fully utilize the "near-dense and distant" point clouds, resulting in oversampling in dense areas and wasting computing resources, while sparse areas may be insufficiently sampled, affecting the accurate perception of distant objects.
The method based on the sampling weight matrix is used to sample point cloud data. Adaptive point cloud compression is achieved by constructing training sets, processing point cloud features, determining reconstruction losses and optimizing the sampling weight matrix.
Through adaptive sampling rate adjustment, the original point cloud information can be better expressed while reducing the amount of data involved in encoding, improving the compression efficiency and accuracy of point cloud data.
Smart Images
Figure CN119540372A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of laser radar, and in particular to a laser radar point cloud compression method, system, equipment and medium. Background Art
[0002] LiDAR determines the distance of an object by emitting laser pulses and calculating the reflection time. This imaging principle determines that the point cloud density presents the characteristics of "closer, denser, farther, sparser". When an object is close to the radar, the number of points detected per unit area increases, and the point cloud is dense; when the object is far away, the number of points per unit area decreases, and the point cloud is sparse.
[0003] The current compression method uses a uniform sampling rate and does not fully consider the characteristics of point clouds, which are "dense near and sparse far". This may result in the inability to effectively utilize the characteristics of point clouds in different regions for optimized compression when processing point cloud data. Dense areas may be oversampled, wasting computing resources, while sparse areas may be undersampled, affecting the accurate perception of distant objects. Summary of the invention
[0004] In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present invention proposes a laser radar point cloud compression method, comprising the following steps: Get the point cloud data collected by the laser radar; Sampling the point cloud data based on a sampling weight matrix; The sampled point cloud data is structured and compressed.
[0005] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Construct the first training set; Processing the data in the first training set to obtain point cloud features in a preset format; The sampling weight matrix is determined based on the reconstruction loss of the point cloud features in the preset format.
[0006] In some embodiments, determining the sampling weight matrix based on the reconstruction loss of the features of the preset format further comprises: Multiplying the initial sampling weight matrix with the data in the first training set to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; Calculating the similarity between the sampling feature and the point cloud feature and adding the similarity to the balance weight parameter to obtain the objective function; The initial sampling weight matrix is optimized based on the objective function until the objective function reaches a preset requirement, and the optimization is stopped to obtain the sampling weight matrix.
[0007] In some embodiments, calculating the similarity between the sampled features and the point cloud features and adding the similarity to the balance weight parameter to obtain the objective function further includes: Calculating the similarity between the sampling feature and the point cloud feature using the L1 or L2 norm; The sampling weight matrix is characterized by the L0 norm and multiplied by the preset weight as the balancing weight parameter.
[0008] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Construct the second training set; Processing the data in the second training set to obtain point cloud features in a preset format; Multiplying the data in the second training set by the initial sampling weight matrix to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; The sampling weight matrix is determined based on the point cloud features and the sampling features.
[0009] In some embodiments, determining the sampling weight matrix based on the point cloud features and the sampling features further includes: Inputting the point cloud features and the sampling features into the target detection network respectively and obtaining the target function using the detection results; The search is performed based on the objective function and preset search conditions until the objective function meets the preset requirements, and the search is stopped to obtain the sampling weight matrix.
[0010] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Selecting a region of interest from the point cloud data; The region of interest is sampled.
[0011] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a laser radar point cloud compression system, comprising: An acquisition module, configured to acquire point cloud data collected by the laser radar; A sampling module, configured to sample the point cloud data based on a sampling weight matrix; The compression module is configured to perform structured conversion and compression on the sampled point cloud data.
[0012] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer device, including: at least one processor; and A memory storing a computer program executable on the processor, wherein the processor executes the steps of any one of the above-described laser radar point cloud compression methods when executing the program.
[0013] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it performs the steps of any one of the lidar point cloud compression methods described above.
[0014] The present invention has one of the following beneficial technical effects: the scheme proposed in the present invention samples point cloud data through a sampling weight matrix, and can adaptively compress point cloud data so that the sampling rates of point clouds at different positions are different. It can reduce the amount of data involved in encoding while still being able to better express the original point cloud information. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.
[0016] Figure 1 A schematic diagram of a flow chart of a laser radar point cloud compression method provided by an embodiment of the present invention; Figure 2 A flowchart of a laser radar point cloud compression method provided by an embodiment of the present invention; Figure 3 A schematic diagram of converting point cloud features into voxel features provided by an embodiment of the present invention; Figure 4 A schematic diagram of converting point cloud features into pillar features provided by an embodiment of the present invention; Figure 5 A schematic diagram of obtaining a sampling weight matrix based on reconstruction loss provided by an embodiment of the present invention; Figure 6 A schematic diagram of obtaining a sampling weight matrix based on target detection loss provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of a laser radar point cloud compression system provided by an embodiment of the present invention; Figure 8 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention; Fig. 9 A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION
[0017] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0018] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.
[0019] According to one aspect of the present invention, an embodiment of the present invention provides a laser radar point cloud compression method, such as Figure 1 As shown, it may include the steps of: S1, obtain the point cloud data collected by the laser radar; S2, sampling the point cloud data based on a sampling weight matrix; S3, performing structured conversion and compression on the sampled point cloud data.
[0020] The solution proposed in the present invention samples point cloud data through a sampling weight matrix, and can adaptively compress point cloud data so that the sampling rates of point clouds at different locations are different. The original point cloud information can still be better expressed while reducing the amount of data involved in encoding.
[0021] In some embodiments, Figure 2 As shown in the figure, the original LiDAR point cloud includes not only a sequence of coordinate values of 3D points (usually x, y, z in a Cartesian coordinate system), but also other attributes, such as reflection intensity. Due to the large amount of point cloud data, point cloud data is usually compressed for long-term storage or transmission. Before the compression process, the original point cloud data (PC) is sampled using a sampling weight matrix. In the existing point cloud compression process, a uniform sampling rate is used. When faced with the characteristics of point cloud data being "dense near and sparse far", the point cloud data cannot be effectively compressed. However, through the sampling weight matrix, different sampling rates can be set for point clouds at different locations, thereby achieving a low sampling rate for point clouds near and a high sampling rate for far away points, and more effectively compressing point cloud data.
[0022] In addition, since the LiDAR point cloud is an unstructured point cloud captured directly from the LiDAR sensor, the sampled point cloud data is also an unstructured point cloud. It is necessary to convert the unstructured sampled point cloud data into a structured intermediate representation (IR) and apply compression coding algorithms to the IR, such as kd tree, octree, grid, and range image. When reconstruction is required, IR decoding is performed, and finally IR is converted into the original point cloud to obtain the reconstructed laser point cloud data. In this way, in view of the characteristics of the LiDAR point cloud of "dense near and sparse far", sampling is performed using a sampling weight matrix, and point clouds with a weight of 0 do not participate in subsequent IR encoding, so that the sampling rates of point clouds at different locations are different, and the original point cloud information can be better expressed while reducing the amount of data involved in encoding.
[0023] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Construct the first training set; Processing the data in the first training set to obtain point cloud features in a preset format; The sampling weight matrix is determined based on the reconstruction loss of the point cloud features in the preset format.
[0024] Specifically, the point cloud data can be converted into point cloud features in a preset format, and the reconstruction loss of the point cloud features before and after sampling is used as the target to learn the sampling weight matrix. For example, the point cloud data can be converted into voxel features or pillar features.
[0025] like Figure 3 The voxel feature shown in the figure is a feature representation obtained by aggregating the information of all points in the voxel. It divides the point cloud space into small voxel units, and each voxel can be regarded as a small three-dimensional cube. In this way, by converting the original point cloud data into voxel features, irregular point cloud data can be converted into a regular voxel grid, which is convenient for subsequent processing and calculation. Moreover, for point cloud data with a certain spatial structure, voxel features can better retain spatial information.
[0026] like Figure 4The pillar feature shown in the figure divides the point cloud into columnar areas on a two-dimensional plane, and each columnar area is called a pillar. The point cloud in each pillar is extracted to obtain the pillar feature. Compared with the voxel feature, the pillar feature is more efficient in processing point cloud data on a two-dimensional plane. It can better preserve the spatial structure information of the point cloud in the horizontal direction. Moreover, for point cloud data in scenes such as autonomous driving, the pillar feature can better adapt to the distribution characteristics of the road surface and surrounding objects.
[0027] In some embodiments, determining the sampling weight matrix based on the reconstruction loss of the features of the preset format further comprises: Multiplying the initial sampling weight matrix with the data in the first training set to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; Calculating the similarity between the sampling feature and the point cloud feature and adding the similarity to the balance weight parameter to obtain the objective function; The initial sampling weight matrix is optimized based on the objective function until the objective function reaches a preset requirement, and the optimization is stopped to obtain the sampling weight matrix.
[0028] In some embodiments, calculating the similarity between the sampled features and the point cloud features and adding the similarity to the balance weight parameter to obtain the objective function further includes: Calculating the similarity between the sampling feature and the point cloud feature using the L1 or L2 norm; The sampling weight matrix is characterized by the L0 norm and multiplied by the preset weight as the balancing weight parameter.
[0029] Specifically, Figure 5 As shown, the sampling weight matrix can be multiplied by the point cloud feature to obtain the sampling feature, and the similarity between the sampling feature and the point cloud feature is used as the reconstruction loss. At the same time, in order to balance the compression ratio and the reconstruction loss, a balance weight parameter can be added as the final objective function, that is: RL(Feature(PC),Feature(PC*W))+
[0030] Among them, PC is the original point cloud, Feature is the voxel or pillar feature obtained by processing the original point cloud data, W (Learnable Sampling Weight) is the sampling weight matrix, and RL represents the reconstruction loss, which is used to calculate the similarity between Feature (PC) and Feature (PC*W). It can be calculated using the L1 or L2 norm. The smaller the reconstruction loss, the better the sampling feature can express the point cloud features. is the balance weight parameter, where is the weight, To characterize the sampling weight matrix using the L0 norm, the meaning is the number of non-zeros in W.
[0031] When solving W, the objective function needs to be as small as possible, and the compression ratio of sampling needs to be taken into account at the same time, that is, the number of non-zero values in W needs to be as small as possible, and the voxel or pillar obtained from the original point cloud can be well expressed. Therefore, algorithms such as sparse coding can be used to solve W. After obtaining W, W is multiplied by the original point cloud. Point clouds with a weight of 0 do not participate in the subsequent IR encoding. Only the point clouds corresponding to the non-zero weights in W need to be saved, so the number of point clouds that need to be encoded can be significantly reduced.
[0032] In some embodiments, different first training sets may be constructed for different road sections or intersections, and personalized optimization may be performed to obtain a sampling weight matrix that is more suitable for the intersection or road section, thereby improving the performance of point cloud compression.
[0033] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Construct the second training set; Processing the data in the second training set to obtain point cloud features in a preset format; Multiplying the data in the second training set by the initial sampling weight matrix to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; The sampling weight matrix is determined based on the point cloud features and the sampling features.
[0034] Specifically, Figure 6 As shown in the figure, the point cloud features corresponding to the unsampled point cloud data and the sampled features after sampling are used for target detection respectively, and combined with a search algorithm, such as a genetic evolutionary algorithm, the performance difference of the target detection results between the two, such as the mAP (average accuracy) difference, is used as the optimization target to search for W, so as to find the optimal W, that is: DL(Feature(PC), Feature(PC* W )) Among them, DL represents the change in target detection performance, and the performance difference of the same target detection network before and after sampling is statistically analyzed, such as the difference in mAP (average accuracy). Then, based on DL, a search algorithm is used to find the optimal W.
[0035] In some embodiments, determining the sampling weight matrix based on the point cloud features and the sampling features further includes: Inputting the point cloud features and the sampling features into the target detection network respectively and obtaining the target function using the detection results; The search is performed based on the objective function and preset search conditions until the objective function meets the preset requirements, and the search is stopped to obtain the sampling weight matrix.
[0036] Specifically, a target detection network can be trained based on the original point cloud data, and IoU loss, classification loss, etc. can be used as loss functions. After obtaining this target detection network, target detection is performed on point cloud features and sampling features respectively, and the target detection results of the two are counted to calculate DL. In some embodiments, search conditions can be set, such as the number of non-zeros in W is not greater than a preset number, and then a search algorithm, such as a genetic evolution algorithm, is used to find the optimal W. After obtaining W, W is multiplied by the original point cloud. Point clouds with weights of 0 do not participate in subsequent IR encoding. Only point clouds corresponding to non-zero weights in W need to be saved, so the number of point clouds that need to be encoded can be significantly reduced.
[0037] In some embodiments, different second training sets can be constructed for different road sections or intersections to obtain corresponding W, and personalized optimization can be performed to obtain a sampling weight matrix that is more suitable for the intersection or section, thereby improving the performance of point cloud compression.
[0038] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Selecting a region of interest from the point cloud data; The region of interest is sampled.
[0039] Specifically, unlike vehicle sensors, roadside sensors are fixed. To further compress, you can also manually or automatically compress only the point cloud of the region of interest (PCroi). For example, you can manually remove areas that are not related to the drivable area of the road, such as trees and buildings outside the road. You can also combine it with a static high-precision map to automatically get a drivable area as ROI.
[0040] The solution proposed in the present invention samples point cloud data through a sampling weight matrix, and can adaptively compress point cloud data so that the sampling rates of point clouds at different locations are different. The original point cloud information can still be better expressed while reducing the amount of data involved in encoding.
[0041] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a laser radar point cloud compression system 400, such as Figure 7 As shown, including: An acquisition module 401 is configured to acquire point cloud data collected by a laser radar; A sampling module 402 is configured to sample the point cloud data based on a sampling weight matrix; The compression module 403 is configured to perform structured conversion and compression on the sampled point cloud data.
[0042] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Construct the first training set; Processing the data in the first training set to obtain point cloud features in a preset format; The sampling weight matrix is determined based on the reconstruction loss of the point cloud features in the preset format.
[0043] In some embodiments, determining the sampling weight matrix based on the reconstruction loss of the features of the preset format further comprises: Multiplying the initial sampling weight matrix with the data in the first training set to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; Calculating the similarity between the sampling feature and the point cloud feature and adding the similarity to the balance weight parameter to obtain the objective function; The initial sampling weight matrix is optimized based on the objective function until the objective function reaches a preset requirement, and the optimization is stopped to obtain the sampling weight matrix.
[0044] In some embodiments, calculating the similarity between the sampled features and the point cloud features and adding the similarity to the balance weight parameter to obtain the objective function further includes: Calculating the similarity between the sampling feature and the point cloud feature using the L1 or L2 norm; The sampling weight matrix is characterized by the L0 norm and multiplied by the preset weight as the balancing weight parameter.
[0045] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Construct the second training set; Processing the data in the second training set to obtain point cloud features in a preset format; Multiplying the data in the second training set by the initial sampling weight matrix to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; The sampling weight matrix is determined based on the point cloud features and the sampling features.
[0046] In some embodiments, determining the sampling weight matrix based on the point cloud features and the sampling features further includes: Inputting the point cloud features and the sampling features into the target detection network respectively and obtaining the target function using the detection results; The search is performed based on the objective function and preset search conditions until the objective function meets the preset requirements, and the search is stopped to obtain the sampling weight matrix.
[0047] In some embodiments, sampling the point cloud data based on a sampling weight matrix further includes: Selecting a region of interest from the point cloud data; The region of interest is sampled.
[0048] Based on the same inventive concept, according to another aspect of the present invention, Figure 8 As shown, an embodiment of the present invention further provides a computer device 501, including: at least one processor 520; and The memory 510 stores a computer program 511 that can be run on the processor. When the processor 520 executes the program, the steps of any one of the above-mentioned laser radar point cloud compression methods are performed.
[0049] Based on the same inventive concept, according to another aspect of the present invention, Fig. 9 As shown, an embodiment of the present invention further provides a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, the steps of any of the above lidar point cloud compression methods are performed.
[0050] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.
[0051] Furthermore, it should be appreciated that the computer-readable storage medium (eg, memory) herein may be either volatile memory or nonvolatile memory, or may include both volatile memory and nonvolatile memory.
[0052] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.
[0053] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.
[0054] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.
[0055] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0056] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0057] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.
Claims
1. A laser radar point cloud compression method, characterized in that: The following steps are involved: Get the point cloud data collected by the laser radar; Sampling the point cloud data based on a sampling weight matrix; The sampled point cloud data is structured and compressed.
2. The method according to claim 1, characterized in that Sampling the point cloud data based on a sampling weight matrix further includes: Construct the first training set; Processing the data in the first training set to obtain point cloud features in a preset format; The sampling weight matrix is determined based on the reconstruction loss of the point cloud features in the preset format.
3. The method according to claim 2, characterized in that Determining the sampling weight matrix based on the reconstruction loss of the point cloud features in the preset format further includes: Multiplying the initial sampling weight matrix with the data in the first training set to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; Calculating the similarity between the sampling feature and the point cloud feature and adding the similarity to the balance weight parameter to obtain the objective function; The initial sampling weight matrix is optimized based on the objective function until the objective function reaches a preset requirement, and the optimization is stopped to obtain the sampling weight matrix.
4. The method according to claim 3, characterized in that Calculating the similarity between the sampling feature and the point cloud feature and adding the similarity to the balance weight parameter to obtain the objective function further includes: Calculating the similarity between the sampling feature and the point cloud feature using the L1 or L2 norm; The sampling weight matrix is characterized by the L0 norm and multiplied by the preset weight as the balancing weight parameter.
5. The method according to claim 1, characterized in that Sampling the point cloud data based on a sampling weight matrix further includes: Construct the second training set; Processing the data in the second training set to obtain point cloud features in a preset format; Multiplying the data in the second training set by the initial sampling weight matrix to obtain sampling data and processing the sampling data to obtain sampling features in a preset format; The sampling weight matrix is determined based on the point cloud features and the sampling features.
6. The method according to claim 5, characterized in that Determining the sampling weight matrix based on the point cloud features and the sampling features further includes: Inputting the point cloud features and the sampling features into the target detection network respectively and obtaining the target function using the detection results; The search is performed based on the objective function and preset search conditions until the objective function meets the preset requirements, and the search is stopped to obtain the sampling weight matrix.
7. The method according to claim 1, characterized in that Sampling the point cloud data based on a sampling weight matrix further includes: Selecting a region of interest from the point cloud data; The region of interest is sampled.
8. A laser radar point cloud compression system, characterized in that: include: An acquisition module, configured to acquire point cloud data collected by the laser radar; A sampling module, configured to sample the point cloud data based on a sampling weight matrix; The compression module is configured to perform structured conversion and compression on the sampled point cloud data.
9. A computer device comprising: at least one processor; as well as A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method according to any one of claims 1 to 7 when executing the program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.