Laser radar point cloud compression method, system, device and medium

Abstract

The invention discloses a laser radar point cloud compression method and system, computer equipment and a medium. The method comprises the following steps: acquiring point cloud data and corresponding image data; performing structured conversion on the point cloud data to obtain structured data; performing compression coding on the structured data according to a preset compression ratio to obtain a first coding feature; and encoding the image data by using a first image encoder to obtain a second encoding feature, and fusing the first encoding feature and the second encoding feature by using a first feature fusion module to obtain a first fused encoding feature. According to the scheme provided by the invention, the image information is used as priori knowledge and is fused into a point cloud data compression process, so that the priori information serving as the image is utilized to reconstruct the original point cloud data under the condition of reducing the information amount of the point cloud data, and the compression ratio of the point cloud data is improved.

Description

Technical Field

[0001] The present invention relates to the field of laser radar, and in particular to a laser radar point cloud compression method, system, equipment and medium. Background Art

[0002] LiDAR determines the distance of an object by emitting laser pulses and calculating the reflection time. This imaging principle determines that the point cloud density presents the characteristics of "closer, denser, farther, sparser". When an object is close to the radar, the number of points detected per unit area increases, and the point cloud is dense; when the object is far away, the number of points per unit area decreases, and the point cloud is sparse.

[0003] This non-uniform distribution poses a challenge to compression. Existing point cloud data compression technologies are difficult to achieve a high compression ratio while maintaining high quality. In addition, the large observation range of lidar also increases the difficulty of compression. Existing compression schemes cannot effectively process point cloud data in different density areas while ensuring coverage of the entire observation range. Especially for large-scale roadside perception systems, storing and transmitting large amounts of point cloud data will bring cost and efficiency issues. Summary of the invention

[0004] In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present invention proposes a laser radar point cloud compression method, comprising the following steps: Obtain point cloud data and corresponding image data; Performing structured conversion on the point cloud data to obtain structured data; Compressing and encoding the structured data according to a preset compression ratio to obtain a first encoding feature; The image data is encoded using a first image encoder to obtain a second encoding feature, and the first encoding feature and the second encoding feature are fused using a first feature fusion module to obtain a first fused encoding feature.

[0005] In some embodiments, encoding the image data using a first image encoder to obtain a second encoding feature, and fusing the first encoding feature and the second encoding feature using a first feature fusion module to obtain a first fused encoding feature, further comprising: Using the first feature conversion module to perform a bird's-eye view conversion on the second coding feature to obtain a third coding feature; The first feature fusion module is used to fuse the first coding feature and the third coding feature to obtain the first fused coding feature.

[0006] In some embodiments, it also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, and the first feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, and the first feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement, and the optimization is stopped.

[0007] In some embodiments, compressing and encoding the structured data according to a preset compression ratio to obtain a first encoding feature further includes: Encoding the image data using a second image encoder to obtain a fourth encoding feature; Using the second feature conversion module to perform a bird's-eye view conversion on the fourth coding feature to obtain a fifth coding feature; Using a second feature fusion module to fuse the structured data and the fifth coding feature to obtain a second fused coding feature; The second fused coding feature is compressed and encoded to obtain a first coding feature.

[0008] In some embodiments, it also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement and the optimization is stopped.

[0009] In some embodiments, it also includes: Decompressing and decoding the first fused coding feature output by the first feature fusion module to obtain structured data; Performing inverse structural processing on the structured data to obtain reconstructed point cloud data; The reconstruction loss is calculated based on the similarity between the original point cloud data in the training set and the reconstructed point cloud data.

[0010] In some embodiments, constructing a training set using a plurality of point cloud data and corresponding image data further includes: Perform clock synchronization and external parameter calibration on the lidar sensor and image sensor; The point cloud data and image data collected at the same time are used as a pair of samples in the training set.

[0011] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a laser radar point cloud compression system, comprising: An acquisition module configured to acquire point cloud data and corresponding image data; A conversion module, configured to perform structured conversion on the point cloud data to obtain structured data; A compression module, configured to compress and encode the structured data according to a preset compression ratio to obtain a first encoding feature; The fusion module is configured to use the first image encoder to encode the image data to obtain a second encoding feature, and use the first feature fusion module to fuse the first encoding feature and the second encoding feature to obtain a first fused encoding feature.

[0012] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer device, including: at least one processor; and A memory storing a computer program executable on the processor, wherein the processor executes the steps of any one of the above-described laser radar point cloud compression methods when executing the program.

[0013] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the lidar point cloud compression methods described above are performed.

[0014] The present invention has one of the following beneficial technical effects: the scheme proposed in the present invention integrates image information as a priori knowledge into the point cloud data compression process. During reconstruction, the original point cloud data is reconstructed using the image as priori information. This can not only improve the compression ratio of the point cloud data and reduce the amount of information in the point cloud data, but also ensure that the point cloud within the observation range of the lidar is effectively compressed and the point cloud data in different density areas is effectively processed. In particular, for large-scale roadside perception systems, a large amount of point cloud data can be better stored and transmitted, thereby further reducing costs and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic diagram of a flow chart of a laser radar point cloud compression method provided by an embodiment of the present invention; Figure 2 It is a flowchart of the existing laser radar point cloud compression method; Figure 3 A flowchart of a laser radar point cloud compression method provided by an embodiment of the present invention; Figure 4 A flowchart of a laser radar point cloud compression method provided by another embodiment of the present invention; Figure 5 A schematic diagram of the structure of a laser radar point cloud compression system provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0018] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0019] According to one aspect of the present invention, an embodiment of the present invention provides a laser radar point cloud compression method, such as Figure 1 As shown, it may include the steps of: S1, obtain point cloud data and corresponding image data; S2, performing structured conversion on the point cloud data to obtain structured data; S3, compressing and encoding the structured data according to a preset compression ratio to obtain a first encoding feature; S4, using a first image encoder to encode the image data to obtain a second encoding feature, and using a first feature fusion module to fuse the first encoding feature and the second encoding feature to obtain a first fused encoding feature.

[0020] In some embodiments, the original laser radar point cloud includes not only a sequence of coordinate values ​​of 3D points (usually x, y, z in a Cartesian coordinate system), but also other attributes, such as reflection intensity. The corresponding image data is generally a two-dimensional image taken from a certain perspective, which can provide visual information such as the appearance and texture of the object. Point clouds and images can describe target objects or scenes from different angles. The original point cloud data may have an irregular distribution and format. It is first structured to obtain structured intermediate representation data (IR) to facilitate subsequent processing and analysis. In order to reduce the amount of data storage and improve transmission efficiency, it is necessary to IR encode the point cloud data that has undergone structured conversion. The operation can be performed according to a pre-set compression ratio, which determines the degree to which the data is compressed. Through a specific compression coding algorithm (such as kd tree, octree, grid and range image, etc.), the structured data is converted into a more compact representation, namely the first coding feature. This coding feature retains the key information of the point cloud data, but the amount of data is greatly reduced relative to the original data. For the acquired image data, the first image encoder is used to encode to obtain the second encoding feature, and finally the first feature fusion module is used to fuse the first encoding feature of the point cloud data and the second encoding feature of the image data according to a certain fusion strategy (such as weighted summation, splicing and then passing through a fully connected layer, etc.), thereby obtaining the first fused encoding feature. This fused encoding feature integrates the key information of two different modal data, and can provide a more comprehensive and accurate basis for subsequent reconstruction.

[0021] In some embodiments, Figure 2 As shown in the figure, the existing compression process is to convert the sampled point cloud data into structured intermediate representation data (IR), and apply the compression coding algorithm to the IR data to obtain IR encoded data. When reconstruction is required, the IR encoded data is first decoded to obtain IR data, and finally the IR data is converted into the original point cloud to obtain the reconstructed laser point cloud data. That is: Coding process: PC E =Encoder(IR(PC O ) Decoding process: PC D =IR inv (Decoder (PC E ) Among them, PC ORepresents the original point cloud; IR represents the intermediate representation feature building module, which converts the original point cloud into IR data; Encoder represents the encoding module based on IR, which usually selects the appropriate compression ratio parameter based on demand or experience; PC E is IR coded data, which is obtained by encoding IR data; Decoder represents a decoding module based on IR, which is used to decode IR coded data to obtain IR data; IR inv It is the inverse operation of the intermediate representation feature building module, which is used to convert IR data to PC; PC D This is the decoded point cloud. In this process, you can first select the compression ratio parameter of the Encoder, and then observe the PC D With PC O If the difference is too large, the compression ratio is reduced, but the process is not optimized by the network module. It can be seen that the existing compression scheme cannot achieve a high compression ratio in the IR encoding stage. The scheme proposed in the present invention integrates image information as a priori knowledge into the point cloud data compression process. In this way, the original point cloud data is reconstructed using the image as a priori information while reducing the amount of point cloud data information, thereby improving the compression ratio of the point cloud data.

[0022] In some embodiments, encoding the image data using a first image encoder to obtain a second encoding feature, and fusing the first encoding feature and the second encoding feature using a first feature fusion module to obtain a first fused encoding feature, further comprising: Using the first feature conversion module to perform a bird's-eye view conversion on the second coding feature to obtain a third coding feature; The first feature fusion module is used to fuse the first coding feature and the third coding feature to obtain the first fused coding feature.

[0023] Specifically, Figure 3 As shown, the image is used as prior knowledge to participate in the IR decoding stage of the point cloud. The image is first encoded by the first image encoder to obtain the second encoding feature, and then the second encoding feature is converted into a BEV feature (third encoding feature) approximating the BEV perspective (bird's eye view) by the first feature conversion module, so as to facilitate fusion with the IR encoding data (first encoding feature). Finally, the BEV feature of the first feature fusion module is fused with the IR encoding data. In this way, when reconstructing, based on the introduction of additional image information, a higher reconstruction effect can be obtained while improving the compression ratio of the IR encoding stage. That is: Coding process: PC E =Encoder(IR(PC O ) Decoding process: PC D =IR inv (Decoder (Fusion D (PC E , BEV 1 (Encoder 1 (Img)))) Among them, Encoder 1 The first image encoder may be, for example, a neural network, whose input is the image and whose output is the first encoded feature of the image. 1 The first feature conversion module, for example, can be a neural network module, whose input is the first encoding feature of the image, and whose output is the BEV feature (third encoding feature) of the BEV perspective (bird's eye view). D It is the first feature fusion module, which can be a neural network module, for example, to fuse the BEV features of the image with the IR encoded data, and finally decode it through the Decoder. inv The inverse transformation obtains the reconstructed point cloud data. Since the image information is introduced, a higher compression ratio can be selected in the IR encoding stage, thereby improving the compression ratio and obtaining a better reconstruction effect.

[0024] In some embodiments, it also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, and the first feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, and the first feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement, and the optimization is stopped.

[0025] In some embodiments, it also includes: Decompressing and decoding the first fused coding feature output by the first feature fusion module to obtain structured data; Performing inverse structural processing on the structured data to obtain reconstructed point cloud data; The reconstruction loss is calculated based on the similarity between the original point cloud data in the training set and the reconstructed point cloud data.

[0026] Specifically, a compression ratio can be set, and based on the compression ratio, the first image encoder, the first feature conversion module, and the first feature fusion module are trained, so that the point cloud data can be effectively compressed at a given compression ratio while maintaining good reconstruction quality.

[0027] During training, the reconstruction loss RL(PC O , PC D ) as the objective function, RL can use L1 or L2 norm to calculate the reconstruction loss, that is, the first fusion coding feature output by the first feature fusion module is decoded and inversely structured to obtain the reconstructed point cloud data, and then RL is used to calculate the similarity between the reconstructed point cloud data and the original point cloud data in the training set to obtain the reconstruction loss.

[0028] When optimizing, an optimization algorithm (such as stochastic gradient descent) can be used to update the parameters of the first image encoder, the first feature conversion module, and the first feature fusion module to reduce the reconstruction loss. In each iteration, the reconstruction loss under the current model parameters is calculated, and the parameters are updated according to the gradient of the loss. This process is repeated until the reconstruction loss meets the preset requirements or a certain number of iterations is reached, and the optimization is stopped.

[0029] In some embodiments, compressing and encoding the structured data according to a preset compression ratio to obtain a first encoding feature further includes: Encoding the image data using a second image encoder to obtain a fourth encoding feature; Using the second feature conversion module to perform a bird's-eye view conversion on the fourth coding feature to obtain a fifth coding feature; Using a second feature fusion module to fuse the structured data and the fifth coding feature to obtain a second fused coding feature; The second fused coding feature is compressed and encoded to obtain a first coding feature.

[0030] Specifically, Figure 4As shown, the image can be used as prior knowledge to participate in the IR encoding stage and IR decoding stage of the point cloud. In the IR encoding stage, the image is first encoded by the second image encoder to obtain the fourth encoding feature, and then the fourth encoding feature is converted into a BEV feature (fifth encoding feature) approximating the BEV perspective (bird's eye view) by the second feature conversion module, so as to facilitate fusion with the IR data (structured data) to obtain the second fused encoding feature. The IR encoding module encodes the second fused encoding feature to obtain IR encoded data (first encoding feature). In the IR decoding stage, similarly, the image is first encoded by the first image encoder to obtain the second encoding feature, and then the second encoding feature is converted into a BEV feature (third encoding feature) approximating the BEV perspective (bird's eye view) by the first feature conversion module, so as to facilitate fusion with the IR encoded data (first encoding feature). Finally, the BEV feature of the first feature fusion module is fused with the IR encoded data. In this way, by introducing image information at the same time in the encoding stage and the decoding stage, compared with introducing image information only in the decoding stage, it is possible to further improve the compression ratio while obtaining a higher reconstruction effect. That is: Coding process: PC E =Encoder(Fusion E (IR(PC O ), BEV 2 (Encoder 2 (Img)))) Decoding process: PC D =IR inv (Decoder (Fusion D (PC E , BEV 1 (Encoder 1 (Img)))) Among them, Encoder 2 The second image encoder may be, for example, a neural network, whose input is the image and whose output is the fourth encoded feature of the image. 2 The second feature conversion module may be, for example, a neural network module, whose input is the fourth encoding feature of the image and whose output is the BEV feature (fifth encoding feature) of the BEV perspective (bird's eye view). E It is the second feature fusion module image, for example, it can be a neural network module, which is used to fuse the BEV features of the image with the IR data, and finally obtain the PC after being encoded by the Encoder. E . Encoder 1The first image encoder may be, for example, a neural network, whose input is the image and whose output is the first encoded feature of the image. 1 The first feature conversion module, for example, can be a neural network module, whose input is the first encoding feature of the image, and whose output is the BEV feature (third encoding feature) of the BEV perspective (bird's eye view). D is the first feature fusion module, which can be a neural network module for example, to combine the BEV features of the image with the PC E After fusion, the IR inv The inverse transformation obtains the reconstructed point cloud data. Since the image information is introduced, a higher compression ratio can be selected in the IR encoding stage, thereby improving the compression ratio and obtaining a better reconstruction effect.

[0031] In some embodiments, it also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement and the optimization is stopped.

[0032] In some embodiments, it also includes: Decompressing and decoding the first fused coding feature output by the first feature fusion module to obtain structured data; Performing inverse structural processing on the structured data to obtain reconstructed point cloud data; The reconstruction loss is calculated based on the similarity between the original point cloud data in the training set and the reconstructed point cloud data.

[0033] Specifically, a compression ratio can be set, and based on the compression ratio, the first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module are trained, so that the point cloud data can be effectively compressed at a given compression ratio while maintaining good reconstruction quality.

[0034] During training, the reconstruction loss RL(PC O , PC D) as the objective function, RL can use L1 or L2 norm to calculate the reconstruction loss, that is, the first fusion coding feature output by the first feature fusion module is decoded and inversely structured to obtain the reconstructed point cloud data, and then RL is used to calculate the similarity between the reconstructed point cloud data and the original point cloud data in the training set to obtain the reconstruction loss.

[0035] When optimizing, an optimization algorithm (such as stochastic gradient descent) can be used to update the parameters of the first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module to reduce the reconstruction loss. In each iteration, the reconstruction loss under the current model parameters is calculated, and the parameters are updated according to the gradient of the loss. This process is repeated until the reconstruction loss meets the preset requirements or a certain number of iterations is reached, and the optimization is stopped.

[0036] In some embodiments, constructing a training set using a plurality of point cloud data and corresponding image data further includes: Perform clock synchronization and external parameter calibration on the lidar sensor and image sensor; The point cloud data and image data collected at the same time are used as a pair of samples in the training set.

[0037] Specifically, the time synchronization device can be used to synchronize the clocks of the lidar sensor and the image sensor, and the calibration board can be used to calibrate the internal parameters of the camera. Then, the dense point cloud map in the high-precision map can be used to calibrate the external parameters of the lidar sensor and the image sensor. When there is no dense point cloud, the point cloud of the lidar sensor can be projected onto the image to perform relative external parameter calibration. In this way, the point cloud data collected by the lidar sensor and the image data collected by the image sensor are synchronized in time and space. During training, based on the time-space synchronized point cloud / image data set, the first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module can be trained and optimized without label data.

[0038] The solution proposed in the present invention integrates image information as a priori knowledge into the point cloud data compression process. In this way, while reducing the amount of information in the point cloud data, the image used as priori information is used to reconstruct the original point cloud data, thereby improving the compression ratio of the point cloud data.

[0039] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a laser radar point cloud compression system 400, such as Figure 5 As shown, including: An acquisition module 401 is configured to acquire point cloud data and corresponding image data; A conversion module 402 is configured to perform structured conversion on the point cloud data to obtain structured data; The compression module 403 is configured to compress and encode the structured data according to a preset compression ratio to obtain a first encoding feature; The fusion module 404 is configured to use the first image encoder to encode the image data to obtain a second encoding feature, and use the first feature fusion module to fuse the first encoding feature and the second encoding feature to obtain a first fused encoding feature.

[0040] In some embodiments, encoding the image data using a first image encoder to obtain a second encoding feature, and fusing the first encoding feature and the second encoding feature using a first feature fusion module to obtain a first fused encoding feature, further comprising: Using the first feature conversion module to perform a bird's-eye view conversion on the second coding feature to obtain a third coding feature; The first feature fusion module is used to fuse the first coding feature and the third coding feature to obtain the first fused coding feature.

[0041] In some embodiments, it also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, and the first feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, and the first feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement, and the optimization is stopped.

[0042] In some embodiments, compressing and encoding the structured data according to a preset compression ratio to obtain a first encoding feature further includes: Encoding the image data using a second image encoder to obtain a fourth encoding feature; Using the second feature conversion module to perform a bird's-eye view conversion on the fourth coding feature to obtain a fifth coding feature; Using a second feature fusion module to fuse the structured data and the fifth coding feature to obtain a second fused coding feature; The second fused coding feature is compressed and encoded to obtain a first coding feature.

[0043] In some embodiments, it also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement and the optimization is stopped.

[0044] In some embodiments, it also includes: Decompressing and decoding the first fused coding feature output by the first feature fusion module to obtain structured data; Performing inverse structural processing on the structured data to obtain reconstructed point cloud data; The reconstruction loss is calculated based on the similarity between the original point cloud data in the training set and the reconstructed point cloud data.

[0045] In some embodiments, constructing a training set using a plurality of point cloud data and corresponding image data further includes: Perform clock synchronization and external parameter calibration on the lidar sensor and image sensor; The point cloud data and image data collected at the same time are used as a pair of samples in the training set.

[0046] Based on the same inventive concept, according to another aspect of the present invention, Figure 6 As shown, an embodiment of the present invention further provides a computer device 501, including: at least one processor 520; and The memory 510 stores a computer program 511 that can be run on the processor. When the processor 520 executes the program, the steps of any one of the above-mentioned laser radar point cloud compression methods are performed.

[0047] Based on the same inventive concept, according to another aspect of the present invention, Figure 7 As shown, an embodiment of the present invention further provides a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, the steps of any of the above-mentioned lidar point cloud compression methods are performed.

[0048] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0049] Furthermore, it should be appreciated that the computer-readable storage medium (eg, memory) herein may be either volatile memory or nonvolatile memory, or may include both volatile memory and nonvolatile memory.

[0050] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0051] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0052] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0053] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0054] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0055] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A laser radar point cloud compression method, characterized in that: The following steps are involved: Obtain point cloud data and corresponding image data; Performing structured conversion on the point cloud data to obtain structured data; Compressing and encoding the structured data according to a preset compression ratio to obtain a first encoding feature; The image data is encoded using a first image encoder to obtain a second encoding feature, and the first encoding feature and the second encoding feature are fused using a first feature fusion module to obtain a first fused encoding feature.

2. The method according to claim 1, characterized in that The method further comprises: encoding the image data by using a first image encoder to obtain a second encoding feature, and fusing the first encoding feature and the second encoding feature by using a first feature fusion module to obtain a first fused encoding feature, further comprising: Using the first feature conversion module to perform a bird's-eye view conversion on the second coding feature to obtain a third coding feature; The first feature fusion module is used to fuse the first coding feature and the third coding feature to obtain the first fused coding feature.

3. The method according to claim 2, characterized in that Also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, and the first feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, and the first feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement, and the optimization is stopped.

4. The method according to claim 2, characterized in that The structured data is compressed and encoded according to a preset compression ratio to obtain a first encoding feature, further comprising: Encoding the image data using a second image encoder to obtain a fourth encoding feature; Using the second feature conversion module to perform a bird's-eye view conversion on the fourth coding feature to obtain a fifth coding feature; Using a second feature fusion module to fuse the structured data and the fifth coding feature to obtain a second fused coding feature; The second fused coding feature is compressed and encoded to obtain a first coding feature.

5. The method according to claim 4, characterized in that Also includes: Construct a training set using multiple point cloud data and corresponding image data; Obtaining a preset compression ratio, and training the first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module using the training set according to the compression ratio; The first image encoder, the first feature conversion module, the first feature fusion module, the second image encoder, the second feature conversion module, and the second feature fusion module are optimized based on the reconstruction loss until the reconstruction loss reaches a preset requirement and the optimization is stopped.

6. The method according to claim 3 or 5, characterized in that Also includes: Decompressing and decoding the first fused coding feature output by the first feature fusion module to obtain structured data; Performing inverse structural processing on the structured data to obtain reconstructed point cloud data; The reconstruction loss is calculated based on the similarity between the original point cloud data in the training set and the reconstructed point cloud data.

7. The method according to claim 3 or 5, characterized in that Constructing a training set using multiple point cloud data and corresponding image data further includes: Perform clock synchronization and external parameter calibration on the lidar sensor and image sensor; The point cloud data and image data collected at the same time are used as a pair of samples in the training set.

8. A laser radar point cloud compression system, characterized in that: include: An acquisition module configured to acquire point cloud data and corresponding image data; A conversion module, configured to perform structured conversion on the point cloud data to obtain structured data; A compression module, configured to compress and encode the structured data according to a preset compression ratio to obtain a first encoding feature; The fusion module is configured to use the first image encoder to encode the image data to obtain a second encoding feature, and use the first feature fusion module to fuse the first encoding feature and the second encoding feature to obtain a first fused encoding feature.

9. A computer device comprising: at least one processor; as well as A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

Images