Depth image compression method, depth image compression device and storage medium

By compressing the depth image of the high-precision depth map, high- or low-level depth maps are determined, lossy compression and preprocessing are performed, and lossless compression is performed by fusing features, which solves the problem of low efficiency of traditional compression methods and achieves more efficient depth map compression.

CN114359420BActive Publication Date: 2025-05-13PEKING UNIV SHENZHEN GRADUATE SCHOOL
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111516459.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-10
Publication Date
2025-05-13
Estimated Expiration
2041-12-10

AI Technical Summary

Technical Problem

The traditional compression method has low compression efficiency for high-precision depth maps.

Method used

By determining the high-bit depth map or low-bit depth map, enter the lossy compressed network to obtain the lossy reconstruction depth map, estimate the pseudo-residue, and input it into the lossy depth preprocessing network and the pseudo-residue preprocessing network, fusing the features for lossless compression encoding.

Benefits of technology

Effectively improve the compression efficiency of high-precision depth images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114359420B_ABST
    Figure CN114359420B_ABST
Patent Text Reader

Abstract

The present invention discloses a depth image compression method, a depth image compression device and a computer-readable storage medium, the method comprising: determining a high-bit depth map or a low-bit depth map according to depth map data to be compressed; inputting the high-bit depth map or the low-bit depth map into a lossy compression network to obtain a lossy reconstructed depth map; determining an estimated pseudo residual according to the lossy reconstructed depth map; inputting the lossy reconstructed depth map into a lossy depth preprocessing network to obtain a first output feature, and inputting the estimated pseudo residual into a pseudo residual preprocessing network to obtain a second output feature; fusing the first output feature and the second output feature to obtain a fused feature; determining encoding data according to the fused feature, and performing lossless compression encoding on the depth map data based on the encoding data. The present invention aims to achieve the effect of improving compression efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a depth image compression method, a depth image compression device and a computer-readable storage medium. Background Art

[0002] Depth map is a basic data format in the field of signal processing and computer vision. Unlike natural images that represent texture information, depth map contains spatial information of the physical environment. Therefore, depth map is widely used in three-dimensional scenes such as self-driving cars. A large number of deep learning works are related to depth maps, including depth estimation, depth completion, etc. These deep learning-based works can generate a large amount of depth map data. In addition, with the rapid improvement of the accuracy and popularity of devices such as laser scanners and LiDAR, these devices have also generated a large number of high-precision depth maps. High-precision depth maps have a wider data range and more complex distribution. Therefore, when high-precision depth maps are compressed using traditional compression methods, their compression efficiency is low.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a depth image compression method, a depth image compression device and a computer-readable storage medium, aiming to solve the technical problem of low compression efficiency of high-precision depth images compressed by traditional compression methods.

[0005] To achieve the above object, the present invention provides a depth image compression method, the depth image compression method comprising the following steps:

[0006] Determining a high-bit depth map or a low-bit depth map according to the depth map data to be compressed;

[0007] Inputting the high-bit depth map or the low-bit depth map into a lossy compression network to obtain a lossy reconstructed depth map;

[0008] Determining an estimated pseudo residual according to the lossy reconstructed depth map;

[0009] Inputting the lossy reconstructed depth map into a lossy depth preprocessing network to obtain a first output feature, and inputting the estimated pseudo residual into a pseudo residual preprocessing network to obtain a second output feature;

[0010] fusing the first output feature and the second output feature to obtain a fused feature;

[0011] The encoding data is determined according to the fusion feature, and the depth map data is losslessly compressed and encoded based on the encoding data.

[0012] Optionally, the step of determining an estimated pseudo residual according to the lossy reconstructed depth map comprises:

[0013] Inputting the lossy reconstructed depth map into the lossy compression network to obtain a simulated lossy reconstructed depth map;

[0014] The estimated pseudo residual is determined according to the lossy reconstructed depth map and the simulated lossy reconstructed depth map.

[0015] Optionally, before the step of determining the encoding data according to the fusion feature and performing lossless compression encoding on the depth map data based on the encoding data, the method further includes:

[0016] Determine a true residual according to the high-bit depth map or the low-bit depth map input to the lossy compression network and the lossy reconstructed depth map;

[0017] The step of determining the encoding data according to the fusion feature and performing lossless compression encoding on the depth map data based on the encoding data comprises:

[0018] Inputting the fused features and the true residual into a deep entropy coding network based on mixed Laplace mixture distribution to obtain the coded data;

[0019] The depth map data is losslessly compressed and encoded based on the encoded data.

[0020] Optionally, the step of fusing the first output feature and the second output feature to obtain a fused feature includes:

[0021] The first output feature and the second output feature are input into a fusion network to obtain the fusion feature.

[0022] Optionally, the pseudo residual preprocessing network includes four residual blocks and an attention module, and the residual block includes two convolutional layers and two Leaky ReLU layers.

[0023] Optionally, the residual block included in the lossy depth preprocessing network has the same architecture as the residual block included in the pseudo residual preprocessing network.

[0024] The present invention also provides a depth image compression device, which includes: a memory, a processor, and a depth image compression program stored in the memory and executable on the processor, wherein the depth image compression program implements the various steps of the depth image compression method described above when executed by the processor.

[0025] The present invention also provides a computer-readable storage medium, on which a depth image compression program is stored. When the depth image compression program is executed by a processor, the various steps of the depth image compression method described above are implemented.

[0026] The embodiments of the present invention propose a depth image compression method, a depth image compression device and a computer-readable storage medium, which first determine a high-bit depth map or a low-bit depth map according to the depth map data to be compressed, and then input the high-bit depth map or the low-bit depth map into a lossy compression network to obtain a lossy reconstructed depth map, and then determine an estimated pseudo-residual according to the lossy reconstructed depth map, and input the lossy reconstructed depth map into a lossy depth preprocessing network to obtain a first output feature, and input the estimated pseudo-residual into a pseudo-residual preprocessing network to obtain a second output feature, and then fuse the first output feature and the second output feature to obtain a fused feature, and finally determine the coded data according to the fused feature, and perform lossless compression coding on the depth map data based on the coded data. The present invention effectively improves the compression efficiency of high-precision depth images. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a flow chart of an embodiment of a depth image compression method of the present invention;

[0029] Figure 3 The overall architecture diagram of the network involved in the embodiment of the present invention;

[0030] Figure 4 Schematic diagram of the network architecture of the pseudo residual preprocessing network involved in the embodiment of the present invention;

[0031] Figure 5 A schematic diagram of a network architecture of a lossy deep preprocessing network involved in an embodiment of the present invention;

[0032] Figure 6 A schematic diagram of a network architecture of a converged network involved in an embodiment of the present invention;

[0033] Figure 7 Schematic diagram of the network architecture of a deep entropy coding network based on a mixed Laplace mixture distribution involved in an embodiment of the present invention.

[0034] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0035] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0036] Depth map is a basic data format in the field of signal processing and computer vision. Unlike natural images that represent texture information, depth map contains spatial information of the physical environment. Therefore, depth map is widely used in three-dimensional scenes such as self-driving cars. A large number of deep learning works are related to depth maps, including depth estimation, depth completion, etc. These deep learning-based works can generate a large amount of depth map data. In addition, with the rapid improvement of the accuracy and popularity of devices such as laser scanners and LiDAR, these devices have also generated a large number of high-precision depth maps. High-precision depth maps have a wider data range and more complex distribution. Therefore, when high-precision depth maps are compressed using traditional compression methods, their compression efficiency is low.

[0037] In order to improve the compression efficiency of high-precision depth maps, an embodiment of the present invention provides a method for compressing depth images, first determining a high-bit depth map or a low-bit depth map according to the depth map data to be compressed, then inputting the high-bit depth map or the low-bit depth map into a lossy compression network to obtain a lossy reconstructed depth map, and then determining an estimated pseudo-residual according to the lossy reconstructed depth map, and inputting the lossy reconstructed depth map into a lossy depth preprocessing network to obtain a first output feature, and inputting the estimated pseudo-residual into a pseudo-residual preprocessing network to obtain a second output feature, and then fusing the first output feature and the second output feature to obtain a fused feature, and finally determining the coded data according to the fused feature, and performing lossless compression coding on the depth map data based on the coded data. The present invention effectively improves the compression efficiency of high-precision depth images.

[0038] like Figure 1 As shown, Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0039] The terminal in the embodiment of the present invention may be a depth image compression device, for example, a PC, a vehicle computer, or a server.

[0040] like Figure 1 As shown, the terminal may include: a processor 1001, an interface 1003, a memory 1004, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The interface 1003 is configured to communicate with other devices or other components. The memory 1004 may be a NAND. The memory 1004 may optionally be a storage device independent of the aforementioned processor 1001.

[0041] Those skilled in the art will understand that Figure 1The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0042] like Figure 1 As shown, the memory 1004 as a computer storage medium may include a control system, an interface module, and a compression program for a depth image.

[0043] exist Figure 1 In the terminal shown, the processor 1001 can be used to call the compression program of the depth image stored in the memory 1004 and perform the following operations:

[0044] Determining a high-bit depth map or a low-bit depth map according to the depth map data to be compressed;

[0045] Inputting the high-bit depth map or the low-bit depth map into a lossy compression network to obtain a lossy reconstructed depth map;

[0046] Determining an estimated pseudo residual according to the lossy reconstructed depth map;

[0047] Inputting the lossy reconstructed depth map into a lossy depth preprocessing network to obtain a first output feature, and inputting the estimated pseudo residual into a pseudo residual preprocessing network to obtain a second output feature;

[0048] fusing the first output feature and the second output feature to obtain a fused feature;

[0049] The encoding data is determined according to the fusion feature, and the depth map data is losslessly compressed and encoded based on the encoding data.

[0050] Optionally, in some embodiments, the processor 1001 may also be used to call a compression program of the depth image stored in the memory 1004, and perform the following operations:

[0051] Inputting the lossy reconstructed depth map into the lossy compression network to obtain a simulated lossy reconstructed depth map;

[0052] The estimated pseudo residual is determined according to the lossy reconstructed depth map and the simulated lossy reconstructed depth map.

[0053] Optionally, in some embodiments, the processor 1001 may also be used to call a compression program of the depth image stored in the memory 1004, and perform the following operations:

[0054] Determine a true residual according to the high-bit depth map or the low-bit depth map input to the lossy compression network and the lossy reconstructed depth map;

[0055] The step of determining the encoding data according to the fusion feature and performing lossless compression encoding on the depth map data based on the encoding data comprises:

[0056] Inputting the fused features and the true residual into a deep entropy coding network based on mixed Laplace mixture distribution to obtain the coded data;

[0057] The depth map data is losslessly compressed and encoded based on the encoded data.

[0058] Optionally, in some embodiments, the processor 1001 may also be used to call a compression program of the depth image stored in the memory 1004, and perform the following operations:

[0059] The first output feature and the second output feature are input into a fusion network to obtain the fusion feature.

[0060] Depth map is a basic data format in the field of signal processing and computer vision. Unlike natural images that represent texture information, depth maps contain spatial information of the physical environment. Therefore, depth maps are widely used in three-dimensional scenes such as self-driving cars. A large number of deep learning works are related to depth maps, including depth estimation, depth completion, etc. These deep learning-based works can generate a large amount of depth map data. In addition, with the rapid improvement in the accuracy and popularity of devices such as laser scanners and LiDAR, these devices have also generated a large number of high-precision depth maps. High-precision depth maps have a wider data range and more complex distribution.

[0061] In related technologies, learning-based image compression is impacting the traditional image coding field. Among various methods, generative models have the highest usage rate. For example, generative models based on variational autoencoders were first used for learning-based lossy image compression, such as balle2016end, cheng2020learned, and balle2018variational. Similar structures have been used for learning-based lossless image compression, such as mentzer2019practical.

[0062] Due to the ability of context modeling, the generative model based on autoregression has good performance. Among them, bai2021learning is a joint deep lossy and lossless image compression. It first uses a deep lossy compression model to compress the input image, and then uses the context model to estimate the likelihood of the residual. However, the generative model based on autoregression suffers from heavy computational cost due to the context structure. Therefore, when the above compression method is used to compress high-precision depth maps, its compression efficiency is low.

[0063] In order to solve the above defects, an embodiment of the present invention proposes a depth image compression method, which aims to solve the defect of low efficiency of traditional depth image compression and achieve the effect of improving the compression efficiency of depth image data. For ease of understanding, the depth image compression method proposed by the present invention is explained below through specific implementation methods.

[0064] Reference Figure 2 The present invention provides a first embodiment of a depth image compression method, the method comprising:

[0065] Step S10: determining a high-bit depth map or a low-bit depth map according to the depth map data to be compressed;

[0066] Step S20: inputting the high-bit depth map or the low-bit depth map into a lossy compression network to obtain a lossy reconstructed depth map;

[0067] Step S30: determining an estimated pseudo residual according to the lossy reconstructed depth map;

[0068] Step S40: inputting the lossy reconstructed depth map into a lossy depth preprocessing network to obtain a first output feature, and inputting the estimated pseudo residual into a pseudo residual preprocessing network to obtain a second output feature;

[0069] Step S50: fusing the first output feature and the second output feature to obtain a fused feature;

[0070] Step S60: determining encoding data according to the fusion features, and performing lossless compression encoding on the depth map data based on the encoding data.

[0071] In this embodiment, after the depth map data to be compressed is acquired, the depth map data may be preprocessed to obtain a high-bit depth map or a low-bit depth map.

[0072] For example, the depth map data to be compressed may be obtained firstly: x={x 1 , x 2 , …, x n}, where x i is the pixel corresponding to the depth map data to be compressed, each pixel x i is a non-negative integer. In high-bit-depth depth maps, x i The range can be very large, such as 0≤xi<2 18 Etc. Therefore, the high bit depth depth map can be split into two low bit depth depth maps. The two low bit depth depth maps (low bit depth map and high bit depth map) are represented as the most significant byte (MSB) and the least significant byte (LSB), respectively:

[0073]

[0074]

[0075] Here, d can be any positive number.

[0076] After determining the high-bit depth map or the low-bit depth map according to the depth map data to be compressed, the high-bit depth map or the low-bit depth map can be input into a lossy compression network to obtain a lossy reconstructed depth map. Then, the lossy reconstructed depth map is input into the lossy compression network to obtain a simulated lossy reconstructed depth map, and the estimated pseudo residual is determined based on the lossy reconstructed depth map and the simulated lossy reconstructed depth map. Then, the lossy reconstructed depth map is input into a lossy depth preprocessing network to obtain a first output feature, and the estimated pseudo residual is input into a pseudo residual preprocessing network to obtain a second output feature. And the real residual is determined based on the high-bit depth map or the low-bit depth map input into the lossy compression network and the lossy reconstructed depth map. And the fused feature and the real residual are input into a deep entropy coding network based on a mixed Laplace mixture distribution to obtain the encoded data. Then, the depth map data is losslessly compressed and encoded based on the encoded data.

[0077] For example, please refer to Figure 3 , this embodiment also proposes a network structure. The network structure involved in this embodiment includes a Lossy compression part and a Lossless compression part. The Lossy compression part includes a Lossy compression network, and the Lossless compression part includes a Pseudo-residual Pre-process Network, a Lossy-depth Pre-process Network, a Fusion Network, and a LMM-based deep Entropy Network.

[0078] After the high-bit depth map or the low-bit depth map is determined, the high-bit depth map or the low-bit depth map x can be input into the Lossy compression network to obtain a lossy reconstructed depth map x'. Then the lossy reconstructed depth map x' is input into the Lossy compression network again to obtain a simulated lossy reconstructed depth map x'. sim It can be understood that in this step, when the operation of the lossy compression network is defined as C(), the following relationship exists:

[0079] x'=C(x)

[0080] x' sim =C(x')

[0081] Further, when determining the lossy reconstructed depth map x' and simulating the lossy reconstructed depth map x' sim Afterwards, the estimated pseudo residual r can be determined according to the following relationship est :

[0082] r est =x'-x' sim

[0083] Optionally, the true residual r can also be defined according to the following relationship:

[0084] r = round(x-x')

[0085] It should be noted that since round is not differentiable, uniform noise can be added during training to approximate it, and round is used for calculation during inference.

[0086] When determining the lossy reconstructed depth map x' and estimating the pseudo residual r est Afterwards, the lossy reconstructed depth map x' and the estimated pseudo residual r can be est Input the Pseudo-residual Pre-process Network to obtain the first output feature. Input the lossy reconstructed depth map x' to the Lossy-depth Pre-process Network to obtain the second output feature. Then input the first output feature and the second output feature to the Fusion Network to obtain the fusion feature.

[0087] After the fused features are obtained, the fused features and the true residual r can be input into the LMM-based deep Entropy Network to obtain the encoded data through the LMM-based deep Entropy Network.

[0088] Please note that, please refer to Figure 4 , the Pseudo-residual Pre-process Network includes four residual blocks and one attention module, and the residual block includes two convolutional layers and two Leaky ReLU layers. Please refer to Figure 5, the architecture of Lossy-depth Pre-process Network is more complex than Pseudo-residual Pre-process Network and is a U-net-like network. Features are downsampled to study features at different scales and reduce computational cost. The downsampling operation is a 3x3 convolution with a stride of 2, and the upsampling operation is a 3x3 sub-pixel convolution. After the upsampling operation, features from lower scales are concatenated with features from skip connections. The residual block in the lossy depth preprocessing network is the same as the residual block in the pseudo-residual preprocessing network. The details of Fusion Network are as follows Figure 6 As shown. The block structure in the Fusion Network is the same as that in the Lossy-depth Pre-process Network. Through multiplication and addition operations, the features of the first output feature and the second output feature are fused together and sent to a deep entropy coding network called hybrid Laplace mixture distribution. Please refer to the reference Figure 7 , Figure 7 It is an architecture diagram of the LMM-based deep Entropy Network. The output of the LMM-based deep Entropy Network is used as the output of the overall network to obtain encoded data, and the depth map data is losslessly compressed and encoded based on the encoded data.

[0089] It can be understood that when the distribution of the true residual r is modeled using a mixed Laplace distribution, for a single Laplace distribution, there exists:

[0090]

[0091] The mixture Laplace distribution is defined as:

[0092]

[0093] Among them, K can be set to 3. The parameters of the Laplace mixture distribution are the outputs of the three parameter blocks. Each parameter block consists of five residual blocks. The residual blocks are the same as the blocks in the pseudo-residual preprocessing network.

[0094] The optimized loss function is defined as follows:

[0095] Based on Shannon entropy, the entropy of P is equal to the value of i The expected number of bits H(p) required for encoding is:

[0096]

[0097] in is the actual distribution of x, and p is the estimated distribution generated by the model.

[0098] Based on the above principle, it can be understood that when compression is performed using the scheme provided in this embodiment, the compression rate is reduced from lossy compression R lossy And the lossless compression ratio R lossless Since the lossy compression is performed based on the hyper-prior structure, the compression is named latent feature and super prior features Two features are required. Therefore, R lossy It consists of two parts:

[0099]

[0100] And R lossless Defined as

[0101]

[0102] where r is the true residual, r est is the estimated pseudo residual and x' is the lossy reconstruction of the input depth map x.

[0103] Based on the definition of the mixed Laplace distribution in the previous equation, according to mentzer2019practical and mentzer2020learning, the cumulative distribution probability P(r) of the discrete true residual r is evaluated by its CDF (cumulative distribution function):

[0104]

[0105] Among them, except R lossy and R lossless , two distortion terms can also be added:

[0106]

[0107]

[0108] where D(x,x′) aims to minimize the mean square error between the original input x and its lossy reconstruction x′; D(r,r est ) is used to minimize the true residual r and its estimated pseudo residual r est The gap between.

[0109] The optimization objectives of lossy and lossless residual compression of the entire depth map are:

[0110] L=R lossy +R lossless +α*D(x,x′)+β*D(r,r est )

[0111] Among them, α and β are two hyperparameters that control the reconstruction performance of lossy depth and the generation performance of pseudo residual, respectively.

[0112] In order to better reflect the effect of the compression method provided in this embodiment, experimental data conducted on the public datasets DIODE and SementicKITTI are provided below.

[0113] It should be noted that DIODE is the first public dataset that contains RGBD images from indoor and outdoor scenes. It contains thousands of accurate, dense, and long-range depth maps. The dataset was collected by the FARO Focus S350 scanner. FARO is a laser scanner that provides high-precision depth measurements ranging from 0.6m to 350m. The depth map of DIODE contains two parts, one is the raw depth acquired by the sensor, and the other is the valid mask. The mask is used to mark whether the pixel is valid. In the experiment, the pixels with invalid masks in the raw depth are set to zero. The DIODE test dataset used contains 771 depth maps.

[0114] SementicKITTI contains 22 point cloud sequences obtained by Velodyne HDL-64E LiDAR. Light Detection and Ranging (LiDAR) provides precise geometric information about the environment. The point cloud data is converted into a depth map that is the same as the range image representation. The depth range is from 0.8m to 120m. Median filtering is used during the test to handle lost pixels. The SementicKITTI test dataset used in the test contains 20351 depth maps.

[0115] The specific experimental data are as follows:

[0116]

[0117]

[0118] The calculation unit of the performance (value) in the above table is bpp, and the lower the value, the better the performance. The solution proposed in the embodiment of the present invention has achieved good performance on two public data sets. That is, the compression efficiency can be improved by using the solution proposed in this embodiment to compress the depth map.

[0119] The present invention also provides a depth image compression device, which includes: a memory, a processor, and a depth image compression program stored in the memory and executable on the processor, wherein the depth image compression program implements the various steps of the depth image compression method described above when executed by the processor.

[0120] The present invention also provides a computer-readable storage medium, on which a depth image compression program is stored. When the depth image compression program is executed by a processor, the various steps of the depth image compression method described above are implemented.

[0121] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0122] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a device to execute the methods described in each embodiment of the present invention.

[0124] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for compressing a depth image, characterized in that: The depth image compression method comprises the following steps: Determining a high-bit depth map or a low-bit depth map according to the depth map data to be compressed; Inputting the high-bit depth map or the low-bit depth map into a lossy compression network to obtain a lossy reconstructed depth map; Determining an estimated pseudo residual according to the lossy reconstructed depth map; Inputting the lossy reconstructed depth map into a lossy depth preprocessing network to obtain a first output feature, and inputting the estimated pseudo residual into a pseudo residual preprocessing network to obtain a second output feature; fusing the first output feature and the second output feature to obtain a fused feature; Determine a true residual according to the high-bit depth map or the low-bit depth map input to the lossy compression network and the lossy reconstructed depth map; Determine encoding data according to the fusion feature, and perform lossless compression encoding on the depth map data based on the encoding data, including: inputting the fusion feature and the true residual into a deep entropy coding network based on a mixed Laplace mixture distribution to obtain the encoding data, and perform lossless compression encoding on the depth map data based on the encoding data.

2. The depth image compression method according to claim 1, characterized in that: The step of determining an estimated pseudo residual according to the lossy reconstructed depth map comprises: Inputting the lossy reconstructed depth map into the lossy compression network to obtain a simulated lossy reconstructed depth map; The estimated pseudo residual is determined according to the lossy reconstructed depth map and the simulated lossy reconstructed depth map.

3. The depth image compression method according to claim 1, characterized in that: The step of fusing the first output feature and the second output feature to obtain a fused feature comprises: The first output feature and the second output feature are input into a fusion network to obtain the fusion feature.

4. The depth image compression method according to claim 1, characterized in that: The pseudo residual preprocessing network includes four residual blocks and an attention module, and the residual block includes two convolutional layers and two Leaky ReLU layers.

5. The depth image compression method according to claim 1, characterized in that: The residual block included in the lossy deep preprocessing network has the same architecture as the residual block included in the pseudo residual preprocessing network.

6. A depth image compression device, characterized in that: The depth image compression device includes: a memory, a processor, and a depth image compression program stored in the memory and executable on the processor. When the depth image compression program is executed by the processor, the steps of the depth image compression method as described in any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a depth image compression program, and when the depth image compression program is executed by a processor, the steps of the depth image compression method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Depth map compression method, device and system and storage medium

    CN113727105A

  • Real-time lossless compression of depth streams

    US20170237996A1