Binocular image compression method, device and readable storage medium

Abstract

This application discloses a binocular image compression method, device, and readable storage medium. This application uses a preset stereo matching model to extract the difference between two binocular images to obtain a disparity image. Residual data is generated based on the reconstructed disparity image and one of the binocular images. The code stream obtained by compressing the residual data is then used as the code stream for the image, thereby compressing the entire binocular image. Compared to other deep learning-based binocular image compression methods, this application significantly improves compression efficiency, reduces computational complexity, and has a wider range of applicable scenarios.

Description

Technical Field

[0001] The present application relates to the field of image compression, and in particular to a binocular image compression method, device, and readable storage medium. Background Art

[0002] In recent years, binocular images have been widely used in various fields, such as 3D (three-dimensional) movies, virtual reality, and autonomous driving. This large amount of high-quality binocular images poses new challenges to data transmission and storage, necessitating efficient image compression methods to reduce costs. Stereo image compression (SIC) is the joint compression of left and right images, aiming to achieve high compression rates for both images. By fully leveraging the mutual information between binocular images, SIC reduces redundancy between viewpoints, resulting in higher compression efficiency than methods that compress each viewpoint independently.

[0003] Some traditional SIC methods treat the similarity between left and right viewpoints as the temporal similarity between consecutive video frames and use video compression techniques to compress binocular images. Similar to motion compensation in video compression, some traditional SIC methods use disparity-compensated prediction to compress left and right images. However, these methods rely on manually designed modules that cannot be jointly optimized, which significantly limits their compression performance. Although deep learning-based binocular image compression methods have emerged with the advancement of deep learning technology, the current deep learning-based binocular image compression methods are computationally complex, resulting in slow image compression.

[0004] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a binocular image compression method, which aims to solve the technical problem that the compression process of the binocular image compression method based on deep learning is relatively complex, resulting in a slow image compression process.

[0006] To achieve the above object, the present application provides a binocular image compression method, wherein the binocular image includes a first image and a second image, and the binocular image compression method includes the following steps:

[0007] Extracting the difference between the first image and the second image based on a preset stereo matching model to obtain a disparity image;

[0008] Compress and reconstruct the first image based on a first compression network to obtain a first reconstructed image;

[0009] Compressing and reconstructing the disparity image based on a second compression network to obtain a reconstructed disparity image;

[0010] transforming the first reconstructed image based on the reconstructed disparity image to obtain a transformed image;

[0011] extracting a difference between the second image and the transformed image to obtain residual data;

[0012] A first image code stream obtained by compressing the first image based on the first compression network, a disparity image code stream obtained by compressing the disparity image based on the second compression network, and a second image code stream obtained by compressing the residual data based on the third compression network model are used as the compressed code stream corresponding to the binocular image.

[0013] Furthermore, the first compression network includes a first encoder, a first decoder, a first quantizer, and a first conditional entropy model, and the step of compressing and reconstructing the first image based on the first compression network to obtain a first reconstructed image includes:

[0014] extracting a first image feature of the first image by the first encoder;

[0015] obtaining a first quantized image feature of the first image feature by the first quantizer;

[0016] Inputting the first quantized image feature and the first image feature into the first conditional entropy model to obtain the first image code stream;

[0017] The first image code stream is decoded and reconstructed by the first decoder to obtain the first reconstructed image.

[0018] Furthermore, the second compression network includes a second encoder, a second decoder, a second quantizer, and a second conditional entropy model, and the step of compressing and reconstructing the disparity image based on the second compression network to obtain a reconstructed disparity image includes:

[0019] extracting a disparity image feature of the disparity image by the second encoder;

[0020] Obtaining a quantized disparity image feature of the disparity image feature by the second quantizer;

[0021] Inputting the quantized disparity image features and the disparity image features into the second conditional entropy model to obtain the disparity image code stream;

[0022] The disparity image code stream is decoded and reconstructed by the second decoder to obtain the reconstructed disparity image.

[0023] Furthermore, the third compression network model includes a third encoder, a third decoder, a third quantizer, and a third conditional entropy model. After the step of extracting the difference between the second image and the transformed image to obtain residual data, the method includes:

[0024] Extracting the residual data by the third encoder to obtain the second image feature;

[0025] obtaining a second quantized image feature of the second image feature by the third quantizer;

[0026] Inputting the second quantized image feature and the second image feature into the third conditional entropy model to obtain the second image code stream;

[0027] The third decoder is configured to reconstruct a second reconstructed image based on the second image code stream and the reconstructed disparity image.

[0028] Furthermore, the first conditional entropy model includes a first super-prior encoding subnetwork, a first super-prior decoding subnetwork, a first context prediction subnetwork, a first entropy parameter estimation subnetwork, a first quantizer, a first entropy encoder, and a first Gaussian model, and the step of inputting the first quantized image feature and the first image feature into the first conditional entropy model to obtain the first image code stream includes:

[0029] Inputting the first image feature into the first super-prior encoding sub-network to obtain a first super-prior parameter;

[0030] Inputting the first super-prior parameter into the first quantizer to obtain a first quantized super-prior parameter;

[0031] Inputting the first quantized super-prior parameter into the first super-prior decoding sub-network to obtain a first upsampled super-prior feature;

[0032] Inputting the first quantized image feature into the first context prediction subnetwork to obtain a first dependency result between any first pixel in the first quantized image feature and a decoded pixel adjacent to the first pixel;

[0033] Combining the first upsampled super-prior feature and the first dependency result and inputting them into the first entropy parameter estimation subnetwork to estimate the first parameter of the first Gaussian model;

[0034] Using the first Gaussian model configured with the first parameters as a first probability distribution function of the first quantized image feature;

[0035] The first quantized image feature and the first probability distribution function are input into the first entropy encoder to obtain the first image code stream.

[0036] Furthermore, the second conditional entropy model includes a second super-prior encoding subnetwork, a second super-prior decoding subnetwork, a second context prediction subnetwork, a second entropy parameter estimation subnetwork, a second quantizer, a second entropy encoder, and a second Gaussian model. The step of inputting the quantized disparity image features and the disparity image features into the second conditional entropy model to obtain the disparity image code stream includes:

[0037] Inputting the disparity image feature into the second super-prior encoding sub-network to obtain a second super-prior parameter;

[0038] Inputting the second super-prior parameter into the second quantizer to obtain a second quantized super-prior parameter;

[0039] Inputting the second quantized super-prior parameter into the second super-prior decoding sub-network to obtain a second up-sampled super-prior feature;

[0040] Inputting the quantized disparity image feature into the second context prediction subnetwork to obtain a second dependency result between any second pixel in the quantized disparity image feature and a decoded pixel adjacent to the second pixel;

[0041] Combining the second up-sampled super-prior feature and the second dependency result and inputting them into the second entropy parameter estimation subnetwork to estimate the second parameters of the second Gaussian model;

[0042] Using the second Gaussian model configured with the second parameter as a second probability distribution function of the quantized disparity image feature;

[0043] The quantized disparity image features and the second probability distribution function are input into the second entropy encoder to obtain the disparity image code stream.

[0044] Furthermore, the preset third conditional entropy model includes a third super a priori encoding subnetwork, a third super a priori decoding subnetwork, a third context prediction subnetwork, a priori fine-tuning network, a third entropy parameter estimation subnetwork, a third quantizer, a third entropy encoder and a third Gaussian model, and the step of inputting the second quantized image features and the second image features into the third conditional entropy model to obtain the second image code stream includes:

[0045] Inputting the second image feature into the third super-prior encoding sub-network to obtain a third super-prior parameter;

[0046] Inputting the third super a priori parameter into the third quantizer to obtain a third quantized super a priori parameter;

[0047] Inputting the third quantized super-prior parameter into the third super-prior decoding sub-network to obtain a third up-sampled super-prior feature;

[0048] Inputting the second quantized image feature into the third context prediction subnetwork to obtain a third dependency result between any third pixel in the second quantized image feature and a decoded pixel adjacent to the third pixel;

[0049] transforming the first quantized image feature based on the reconstructed disparity image to obtain aligned cross-viewpoint prior image features;

[0050] Inputting the aligned cross-view prior image features into the prior fine-tuning network to obtain new aligned cross-view prior image features after fine-tuning;

[0051] Combining the third up-sampled super-prior feature, the third dependency result, and the new aligned cross-viewpoint prior image feature and inputting them into the third entropy parameter estimation subnetwork to estimate the third parameters of the third Gaussian model;

[0052] Using the third Gaussian model configured with the third parameter as a third probability distribution function of the second quantized image feature;

[0053] The second quantized image feature and the third probability distribution function are input into the third entropy encoder to obtain the second image code stream.

[0054] Furthermore, the binocular image compression method further includes:

[0055] A loss function is constructed based on the first image code stream, the disparity image code stream, the second image code stream, the first image, the first reconstructed image, the second image, and the second reconstructed image, wherein the loss function is as follows:

[0056]

[0057] Where R l represents the average number of bits per pixel of the second image stream, R r represents the average number of bits per pixel of the first image stream, R d represents the average number of bits per pixel of the parallax image stream, D is the distortion function, λ is the weight, and x l For the second image, is the second reconstructed image, x r For the first image, for the first reconstructed image;

[0058] The first compression network, the second compression network, the third compression network and the preset stereo matching model are iteratively optimized based on the loss function.

[0059] In addition, to achieve the above-mentioned purpose, the present application also provides a binocular image compression device, which includes: a memory, a processor, and a binocular image compression program stored on the memory and runnable on the processor. When the binocular image compression program is executed by the processor, the steps of the binocular image compression method as described above are implemented.

[0060] In addition, to achieve the above-mentioned purpose, the present application also provides a readable storage medium, on which a binocular image compression program is stored. When the binocular image compression program is executed by a processor, the steps of the binocular image compression method as described above are implemented.

[0061] Embodiments of the present application propose a binocular image compression method, device, and readable storage medium. The method comprises extracting the difference between a first image and a second image based on a preset stereo matching model to obtain a disparity image; compressing and reconstructing the first image based on a first compression network to obtain a first reconstructed image; compressing and reconstructing the disparity image based on a second compression network to obtain a reconstructed disparity image; transforming the first reconstructed image based on the reconstructed disparity image to obtain a transformed image; extracting the difference between the second image and the transformed image to obtain residual data; and using the first image stream obtained by compressing the first image based on the first compression network, the disparity image stream obtained by compressing the disparity image based on the second compression network, and the second image stream obtained by compressing the residual data based on a third compression network model as the compressed stream corresponding to the binocular image. Specifically, the present application extracts the difference between the two images in a binocular image using a preset stereo matching model to obtain a disparity image. Compared to calculating the mutual information of a binocular image pair through dense connections in existing DSICs, transforming through disparity images can significantly reduce computational complexity. Compared to the homography-based perspective transformation used in existing HES ICs, the disparity image-based transformation has a wider range of applicability and higher matching accuracy. Furthermore, to reduce the image quality loss caused by transforming the first image based on the disparity map to obtain the transformed image, the present application also generates residual data based on the difference between the transformed image and the original second image, making the reconstructed second image clearer. Furthermore, the difference image code stream and the second image code stream (i.e., the code stream obtained by compressing the residual data) replace the code stream obtained by compressing the original second image, reducing the code stream consumption required to transmit the second image and achieving higher binocular image compression efficiency, i.e., a smaller compression size. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0063] Figure 2This is a flow chart of the first embodiment of the binocular image compression method of the present application;

[0064] Figure 3 This is a flow chart of the second embodiment of the binocular image compression method of the present application;

[0065] Figure 4 This is a flow chart of the third embodiment of the binocular image compression method of the present application;

[0066] Figure 5 This is a flow chart of the fourth embodiment of the binocular image compression method of the present application;

[0067] Figure 6 Schematic diagram of the network structure of each component in the binocular image compression method of this application;

[0068] Figure 7 Schematic diagram of the structure of the first conditional entropy model in the binocular image compression method of this application;

[0069] Figure 8 Schematic diagram of the structure of the second conditional entropy model in the binocular image compression method of this application;

[0070] Figure 9 This is a schematic diagram of the structure of the third conditional entropy model in the binocular image compression method of this application.

[0071] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0072] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0073] like Figure 1 As shown, Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0074] The device in the embodiment of the present application may be a PC, or an electronic terminal device with data processing capabilities, such as a smart phone, a tablet computer, or a portable computer.

[0075] like Figure 1As shown, the device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and optionally the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0076] Optionally, the device may also include a camera, RF (Radio Frequency) circuit, sensor, audio circuit, WiFi module, etc. Among them, sensors include light sensors, motion sensors and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor can turn off the display screen and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the mobile device can also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0077] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation of the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0078] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a binocular image compression program.

[0079] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the compression program of the binocular image stored in the memory 1005, where the binocular image includes a first image and a second image, and perform the following operations:

[0080] Extracting the difference between the first image and the second image based on a preset stereo matching model to obtain a disparity image;

[0081] Compress and reconstruct the first image based on a first compression network to obtain a first reconstructed image;

[0082] Compressing and reconstructing the disparity image based on a second compression network to obtain a reconstructed disparity image;

[0083] transforming the first reconstructed image based on the reconstructed disparity image to obtain a transformed image;

[0084] extracting a difference between the second image and the transformed image to obtain residual data;

[0085] A first image code stream obtained by compressing the first image based on the first compression network, a disparity image code stream obtained by compressing the disparity image based on the second compression network, and a second image code stream obtained by compressing the residual data based on the third compression network model are used as the compressed code stream corresponding to the binocular image.

[0086] Furthermore, the processor 1001 may call a binocular image compression program stored in the memory 1005 and perform the following operations:

[0087] The first compression network includes a first encoder, a first decoder, a first quantizer, and a first conditional entropy model. The step of compressing and reconstructing the first image based on the first compression network to obtain a first reconstructed image includes:

[0088] extracting a first image feature of the first image by the first encoder;

[0089] obtaining a first quantized image feature of the first image feature by the first quantizer;

[0090] Inputting the first quantized image feature and the first image feature into the first conditional entropy model to obtain the first image code stream;

[0091] The first image code stream is decoded and reconstructed by the first decoder to obtain the first reconstructed image.

[0092] Furthermore, the processor 1001 may call a binocular image compression program stored in the memory 1005 and perform the following operations:

[0093] The second compression network includes a second encoder, a second decoder, a second quantizer, and a second conditional entropy model. The step of compressing and reconstructing the disparity image based on the second compression network to obtain a reconstructed disparity image includes:

[0094] extracting a disparity image feature of the disparity image by the second encoder;

[0095] Obtaining a quantized disparity image feature of the disparity image feature by the second quantizer;

[0096] Inputting the quantized disparity image features and the disparity image features into the second conditional entropy model to obtain the disparity image code stream;

[0097] The disparity image code stream is decoded and reconstructed by the second decoder to obtain the reconstructed disparity image.

[0098] Furthermore, the processor 1001 may call a binocular image compression program stored in the memory 1005 and perform the following operations:

[0099] The third compression network model includes a third encoder, a third decoder, a third quantizer, and a third conditional entropy model. After the step of extracting the difference between the second image and the transformed image to obtain residual data, the method includes:

[0100] Extracting the residual data by the third encoder to obtain the second image feature;

[0101] obtaining a second quantized image feature of the second image feature by the third quantizer;

[0102] Inputting the second quantized image feature and the second image feature into the third conditional entropy model to obtain the second image code stream;

[0103] The third decoder is configured to reconstruct a second reconstructed image based on the second image code stream and the reconstructed disparity image.

[0104] Furthermore, the processor 1001 may call a binocular image compression program stored in the memory 1005 and perform the following operations:

[0105] The first conditional entropy model includes a first super-prior encoding subnetwork, a first super-prior decoding subnetwork, a first context prediction subnetwork, a first entropy parameter estimation subnetwork, a first quantizer, a first entropy encoder and a first Gaussian model, and the step of inputting the first quantized image feature and the first image feature into the first conditional entropy model to obtain the first image code stream includes:

[0106] Inputting the first image feature into the first super-prior encoding sub-network to obtain a first super-prior parameter;

[0107] Inputting the first super-prior parameter into the first quantizer to obtain a first quantized super-prior parameter;

[0108] Inputting the first quantized super-prior parameter into the first super-prior decoding sub-network to obtain a first upsampled super-prior feature;

[0109] Inputting the first quantized image feature into the first context prediction subnetwork to obtain a first dependency result between any first pixel in the first quantized image feature and a decoded pixel adjacent to the first pixel;

[0110] Combining the first upsampled super-prior feature and the first dependency result and inputting them into the first entropy parameter estimation subnetwork to estimate the first parameter of the first Gaussian model;

[0111] Using the first Gaussian model configured with the first parameters as a first probability distribution function of the first quantized image feature;

[0112] The first quantized image feature and the first probability distribution function are input into the first entropy encoder to obtain the first image code stream.

[0113] Furthermore, the processor 1001 may call a binocular image compression program stored in the memory 1005 and perform the following operations:

[0114] The second conditional entropy model includes a second super priori encoding subnetwork, a second super priori decoding subnetwork, a second context prediction subnetwork, a second entropy parameter estimation subnetwork, a second quantizer, a second entropy encoder and a second Gaussian model, and the step of inputting the quantized disparity image features and the disparity image features into the second conditional entropy model to obtain the disparity image code stream includes:

[0115] Inputting the disparity image feature into the second super-prior encoding sub-network to obtain a second super-prior parameter;

[0116] Inputting the second super-prior parameter into the second quantizer to obtain a second quantized super-prior parameter;

[0117] Inputting the second quantized super-prior parameter into the second super-prior decoding sub-network to obtain a second up-sampled super-prior feature;

[0118] Inputting the quantized disparity image feature into the second context prediction subnetwork to obtain a second dependency result between any second pixel in the quantized disparity image feature and a decoded pixel adjacent to the second pixel;

[0119] Combining the second up-sampled super-prior feature and the second dependency result and inputting them into the second entropy parameter estimation subnetwork to estimate the second parameters of the second Gaussian model;

[0120] Using the second Gaussian model configured with the second parameter as a second probability distribution function of the quantized disparity image feature;

[0121] The quantized disparity image features and the second probability distribution function are input into the second entropy encoder to obtain the disparity image code stream.

[0122] Furthermore, the processor 1001 may call a binocular image compression program stored in the memory 1005 and perform the following operations:

[0123] The preset third conditional entropy model includes a third super a priori encoding subnetwork, a third super a priori decoding subnetwork, a third context prediction subnetwork, a priori fine-tuning network, a third entropy parameter estimation subnetwork, a third quantizer, a third entropy encoder, and a third Gaussian model. The step of inputting the second quantized image feature and the second image feature into the third conditional entropy model to obtain the second image code stream includes:

[0124] Inputting the second image feature into the third super-prior encoding sub-network to obtain a third super-prior parameter;

[0125] Inputting the third super a priori parameter into the third quantizer to obtain a third quantized super a priori parameter;

[0126] Inputting the third quantized super-prior parameter into the third super-prior decoding sub-network to obtain a third up-sampled super-prior feature;

[0127] Inputting the second quantized image feature into the third context prediction subnetwork to obtain a third dependency result between any third pixel in the second quantized image feature and a decoded pixel adjacent to the third pixel;

[0128] transforming the first quantized image feature based on the reconstructed disparity image to obtain aligned cross-viewpoint prior image features;

[0129] Inputting the aligned cross-view prior image features into the prior fine-tuning network to obtain new aligned cross-view prior image features after fine-tuning;

[0130] Combining the third up-sampled super-prior feature, the third dependency result, and the new aligned cross-viewpoint prior image feature and inputting them into the third entropy parameter estimation subnetwork to estimate the third parameters of the third Gaussian model;

[0131] Using the third Gaussian model configured with the third parameter as a third probability distribution function of the second quantized image feature;

[0132] The second quantized image feature and the third probability distribution function are input into the third entropy encoder to obtain the second image code stream.

[0133] Furthermore, the processor 1001 may call a binocular image compression program stored in the memory 1005 and perform the following operations:

[0134] The binocular image compression method further includes:

[0135] A loss function is constructed based on the first image code stream, the disparity image code stream, the second image code stream, the first image, the first reconstructed image, the second image, and the second reconstructed image, wherein the loss function is as follows:

[0136]

[0137] Where R l represents the average number of bits per pixel of the second image stream, R r represents the average number of bits per pixel of the first image stream, R d represents the average number of bits per pixel of the parallax image stream, D is the distortion function, λ is the weight, and x l For the second image, is the second reconstructed image, x r For the first image, for the first reconstructed image;

[0138] The first compression network, the second compression network, the third compression network and the preset stereo matching model are iteratively optimized based on the loss function.

[0139] Reference Figure 2 The first embodiment of the binocular image compression method of the present application includes:

[0140] Step S10, extracting the difference between the first image and the second image based on a preset stereo matching model to obtain a disparity image;

[0141] It should be noted that a binocular image generally refers to the images seen by each of a person's two eyes, and can be divided into a left-eye image and a right-eye image. The left-eye image and the right-eye image are actually images of the same scene observed from different angles, so the left-eye image and the right-eye image are relatively similar. In this embodiment, the binocular image includes a first image and a second image, where the first image can represent either the left-eye image or the right-eye image, and the corresponding second image can represent either the right-eye image or the left-eye image. Because the first and second images of the binocular image are relatively similar, when compressing the binocular image, the original image of one image can be compressed, while for the other image, only the difference between the two images needs to be compressed, without compressing the original image, thereby achieving a significant reduction in image size.

[0142] Specifically, the first image is used as the reference image, that is, the first image will be compressed from its original image. The difference between the first image and the second image is extracted through a preset stereo matching model to obtain a disparity map. The process of extracting the disparity map based on the stereo matching model can refer to the existing technology and will not be repeated here. The extracted disparity image refers to the coordinate difference of similar features of the first and second images in the binocular image, which can reflect the pixel-level correlation between the first and second images. Therefore, the disparity map can be used as a transformation basis for transforming the first image to obtain the second image.

[0143] It should be noted that there are two main types of binocular image compression methods currently available based on deep learning. One is a method based on dense connection of feature domains (Deep Stereo Image Compression, DSIC). DSIC densely transforms the features of the left image to the right image at all levels of the encoder and decoder to reduce the joint bit rate by utilizing the redundancy between the binocular image pairs. However, due to the limitations of dense connection, this method has high computational complexity. The other is a method based on homography (Deep Homography for Efficient Stereo Image Compression, HESIC). HESI C uses a 3×3 homography matrix to transform the left image to the right viewpoint. For the right viewpoint, only the residual image is encoded, reducing computational complexity. However, the homography matrix is used to describe the relationship between two images on the same plane in space. This condition is difficult to meet for most binocular image pairs, which limits the scope of application of HESI C. In this embodiment, the disparity map obtained based on the stereo matching model is transformed (i.e., the second image is obtained based on the transformation of the first image). Compared with the dense connection in DSIC to calculate the mutual information of the binocular image pair, the transformation based on the disparity image can greatly reduce the computational complexity. Compared with the homography-based perspective transformation used in HESI C, the transformation based on disparity images has a wider range of applications and higher matching accuracy.

[0144] Step S20, compressing and reconstructing the first image based on a first compression network to obtain a first reconstructed image;

[0145] Specifically, in this embodiment, the first compression network includes a first encoder and a first decoder. The first image will serve as the reference image, so the first encoder encodes and compresses the first image (i.e., the original left or right image) to produce a first image stream. The first encoder then decodes and reconstructs the first image stream to produce a first reconstructed image. It is understood that the first reconstructed image is the image restored after compression of the first image.

[0146] Step S30, compressing and reconstructing the disparity image based on a second compression network to obtain a reconstructed disparity image;

[0147] Specifically, the second compression network includes a second encoder and a second decoder. The disparity image, which is the difference information between the second image and the first image, will also be compressed, that is, the difference image is encoded and compressed by the second encoder to obtain a difference image code stream, and then the difference image code stream is decoded and reconstructed based on the second decoder to obtain a reconstructed disparity image. It should be noted that for the above-mentioned first encoder, first decoder, second encoder and second decoder, technical personnel can set the encoder type according to different needs. For example, an auto-encoder (AE) can remove redundant information in an image, so the auto-encoder can be used to compress the image (remove redundancy) or extract the features of the image (the information retained by removing the redundancy is the feature of the image).

[0148] Step S40, transforming the first reconstructed image based on the reconstructed disparity image to obtain a transformed image;

[0149] The first reconstructed image is transformed based on the reconstructed disparity image to obtain a transformed image (it can be understood that the transformed image obtained at this time can actually be used as the reconstructed second image, but the image clarity is low). The process of transforming the first reconstructed image based on the reconstructed disparity image to obtain the transformed image can be to use backward mapping to map the pixel points in the first reconstructed image to the angle of the second image based on the pixel point correspondence provided by the reconstructed disparity image, thereby obtaining the above-mentioned transformed image. Specifically, based on the reconstructed disparity image, a sampling grid can be generated to correspond each pixel point (u, v) in the second image to the coordinate value (u', v') in the first image. Since the coordinates (u', v') are usually not integers, interpolation fitting processing is required to interpolate and fit the pixel points around the coordinates (u', v') in the first image to calculate the pixel value corresponding to the pixel point (u, v) of the second image. Exemplarily, bilinear interpolation can be selected to fit the pixels.

[0150] Step S50, extracting the difference between the second image and the transformed image to obtain residual data;

[0151] Specifically, the transformed image extracted above can be considered a second image with lower clarity. To further improve the clarity of the reconstructed second image, the difference between the transformed image and the second image is extracted to obtain residual data. It is understood that the residual data can further improve the clarity of the transformed image, thereby improving the quality of the reconstructed second image. Extraction of the residual data can also be implemented based on the aforementioned autoencoder (and can be directly obtained via the third encoder in the third compression network).

[0152] Step S60: Using a first image code stream obtained by compressing the first image based on the first compression network, a disparity image code stream obtained by compressing the disparity image based on the second compression network, and a second image code stream obtained by compressing the residual data based on the third compression network model as a compressed code stream corresponding to the binocular image.

[0153] Specifically, in this embodiment, the third compression network includes a third encoder, and the third encoder compresses the residual data to obtain a second image stream. The disparity image stream obtained by compressing the disparity image, the second image stream obtained by compressing the residual data, and the first image stream obtained by compressing the first image are used as the compressed binocular image stream.

[0154] In this embodiment, the difference between the first image and the second image is extracted based on a preset stereo matching model to obtain a disparity image; the first image is compressed and reconstructed based on a first compression network to obtain a first reconstructed image; the disparity image is compressed and reconstructed based on a second compression network to obtain a reconstructed disparity image; the first reconstructed image is transformed based on the reconstructed disparity image to obtain a transformed image; the difference between the second image and the transformed image is extracted to obtain residual data; and the first image stream obtained by compressing the first image based on the first compression network, the disparity image stream obtained by compressing the disparity image based on the second compression network, and the second image stream obtained by compressing the residual data based on a third compression network model are used as the compressed stream corresponding to the binocular image. That is, the present application extracts the difference between the two images in the binocular image using a preset stereo matching model to obtain a disparity image. Compared with the dense connection method used in existing DSI C to calculate the mutual information of the binocular image pair, transforming based on the disparity image can significantly reduce computational complexity. Compared with the homography-based perspective transformation used in existing HESI C, the transformation based on the disparity image has a wider range of applicability and higher matching accuracy. To reduce image quality loss caused by transforming the first image based on the disparity map to obtain a transformed image, this application also generates residual data based on the difference between the transformed image and the original second image, making the reconstructed second image clearer. Furthermore, replacing the compressed bitstream of the original second image with the difference image bitstream and the second image bitstream (i.e., the bitstream obtained by compressing the residual data) reduces the bitstream consumption required to transmit the second image, resulting in higher binocular image compression efficiency and a smaller compressed binocular image.

[0155] Further, refer to Figure 3 Based on the first embodiment of the binocular image compression method of the present application, a second embodiment of the binocular image compression method of the present application is proposed.

[0156] The first compression network includes a first encoder, a first decoder, a first quantizer, and a first conditional entropy model, and the compressing and reconstructing the first image based on the first compression network to obtain a first reconstructed image includes:

[0157] Step S210: extracting a first image feature of the first image by using the first encoder;

[0158] Specifically, the first image feature of the first image is obtained by removing redundancy in the first image through a preset first encoder (such as an autoencoder). It can be understood that the first image feature obtained at this time is a compressed image, but the compression degree is low. Take the first image as the right eye image in the first binocular image as an example for explanation, as shown in FIG. Figure 6 , is a schematic diagram of the network structure of each component in the binocular image compression method of this application. Right eye image x r The right eye image feature y is extracted by the first encoder l .

[0159] Step S220, obtaining a first quantized image feature of the first image feature by the first quantizer;

[0160] Step S230: Inputting the first quantized image feature and the first image feature into the first conditional entropy model to obtain the first image code stream of the first quantized image feature;

[0161] Specifically, based on the above example, refer to Figure 6 , the right eye image feature will be input into the first quantizer for quantization to obtain the quantized right eye image feature (ie, the first quantized image feature). The quantized right eye image feature and the right eye image feature (ie, the first image feature) are input into the first conditional entropy model to generate a first probability distribution function of the quantized right eye image feature.

[0162] Furthermore, the first conditional entropy model includes a first super priori encoding subnetwork, a first super priori decoding subnetwork, a first context prediction subnetwork, a first entropy parameter estimation subnetwork, a first quantizer, a first entropy encoder and a first Gaussian model, and step S230 includes:

[0163] Step S231: inputting the first image feature into the first super-prior encoding sub-network to obtain a first super-prior parameter;

[0164] Step S232, inputting the first super-prior parameter into the first quantizer to obtain a first quantization super-prior parameter;

[0165] Step S233: inputting the first quantized super-prior parameter into the first super-prior decoding sub-network to obtain a first up-sampled super-prior feature;

[0166] Step S234: input the first quantized image feature into the first context prediction subnetwork to obtain a first dependency result between any first pixel in the first quantized image feature and a decoded pixel adjacent to the first pixel;

[0167] Step S235: Combining the first up-sampled super-prior feature and the first dependency result and inputting them into the first entropy parameter estimation subnetwork to estimate the first parameters of the first Gaussian model;

[0168] Step S236: using the first Gaussian model configured with the first parameter as a first probability distribution function of the first quantized image feature;

[0169] Step S237: input the first quantized image feature and the first probability distribution function into the first entropy encoder to obtain the first image code stream.

[0170] Specifically, refer to Figure 7 , is a structural diagram of the first conditional entropy model in the binocular image compression method of the present application, and the dotted area in the figure is the first conditional entropy model. The first image feature is input into the first super-prior encoding sub-network to obtain the first super-prior parameter. The first super-prior parameter is input into the first quantizer to obtain the first quantization super-prior parameter. The first quantization super-prior parameter is input into the first super-prior decoding sub-network to obtain the first up-sampled super-prior feature. The first quantized image feature is input into the first context prediction sub-network to obtain the first dependency result between any first pixel in the first quantized image feature and the decoded pixel adjacent to the first pixel. The first up-sampled super-prior feature and the first dependency result are combined and input into the first entropy parameter estimation sub-network to estimate the first parameter of the first Gaussian model. Let the first parameter be {μ r ,σ r The first Gaussian model configured with the first parameter is used as the first probability distribution function of the first quantized image feature (the first probability distribution function can be expressed as:

[0171]

[0172] Where, is the first quantized hyper-prior parameter, {μ r ,σ r} are the parameters of the first Gaussian model, The first quantized image feature is inputted into the first entropy encoder and the first probability distribution function to obtain the first image code stream of the first quantized image feature, wherein the first entropy encoder can be an arithmetic encoder, the first super priori encoder can be a convolutional layer, the first super priori encoder can be a deconvolutional layer, and the first context prediction sub-network can be a mask convolutional neural network. In addition, referring to Figure 7 Between the first quantizer and the first super-a priori decoding sub-network, a first entropy encoder and a first entropy decoder are included, which may also correspond to an arithmetic decoder and an arithmetic encoder. In this embodiment, the conditional entropy model can flexibly design a network structure based on actual needs, and the present invention does not impose specific limitations on the network structure.

[0173] Step S240: decoding and reconstructing the first image code stream by the first decoder to obtain the first reconstructed image.

[0174] Specifically, refer to Figure 7 Before obtaining the first reconstructed image, the first image code stream needs to be decoded by the first entropy decoder to obtain the first quantized image feature The first decoder then reconstructs a first reconstructed image based on the first quantized image feature (refer to Figure 6 is the reconstructed right eye image).

[0175] Further, refer to Figure 4 Based on the second embodiment of the binocular image compression method of the present application, a third embodiment of the binocular image compression method of the present application is proposed.

[0176] The second compression network includes a second encoder, a second decoder, a second quantizer, and a second conditional entropy model. The step of compressing and reconstructing the disparity image based on the second compression network to obtain a reconstructed disparity image includes:

[0177] Step S310: extracting disparity image features of the disparity image through the second encoder;

[0178] Step S320, obtaining a quantized disparity image feature of the disparity image feature by the second quantizer;

[0179] Step S330 , inputting the quantized disparity image features and the disparity image features into the second conditional entropy model to obtain the disparity image bitstream;

[0180] Furthermore, the second conditional entropy model includes a second super a priori encoding subnetwork, a second super a priori decoding subnetwork, a second context prediction subnetwork, a second entropy parameter estimation subnetwork, a second quantizer, a second entropy encoder and a second Gaussian model, and step S330 includes:

[0181] Step S331, inputting the disparity image features into the second super-prior encoding sub-network to obtain second super-prior parameters;

[0182] Step S332: inputting the second super a priori parameter into the second quantizer to obtain a second quantization super a priori parameter;

[0183] Step S333: inputting the second quantized super-prior parameter into the second super-prior decoding sub-network to obtain a second up-sampled super-prior feature;

[0184] Step S334: inputting the quantized disparity image feature into the second context prediction subnetwork to obtain a second dependency result between any second pixel in the quantized disparity image feature and a decoded pixel adjacent to the second pixel;

[0185] Step S335: Combining the second up-sampled super-prior feature and the second dependency result and inputting them into the second entropy parameter estimation subnetwork to estimate the second parameters of the second Gaussian model;

[0186] Step S336, using the second Gaussian model configured with the second parameter as a second probability distribution function of the quantized disparity image feature;

[0187] Step S337 : Input the quantized disparity image features and the second probability distribution function into the second entropy encoder to obtain the disparity image code stream.

[0188] Step S340 : decoding and reconstructing the disparity image code stream by the second decoder to obtain the reconstructed disparity image.

[0189] Specifically, refer to Figure 8 , is a schematic diagram of the structure of the second conditional entropy model in the binocular image compression method of the present application. The dotted area in the figure is the second conditional entropy model. It can be understood that in this embodiment, the process of compressing and reconstructing the disparity image based on the second compression network to obtain the reconstructed disparity image is actually similar to the second embodiment, only the input is changed from the first image to the disparity image, and the first compression model is changed to the second compression model. Therefore, it will not be repeated here. Correspondingly, let the second parameter of the above-mentioned second Gaussian model be {μ d ,σ d}, then the second probability distribution function can be expressed as:

[0190] Where, is the second quantized hyper-prior parameter, {μ d ,σ d} are the parameters of the second Gaussian model, The second quantized image feature.

[0191] Further, refer to Figure 5 Based on the third embodiment of the binocular image compression method of the present application, a fourth embodiment of the binocular image compression method of the present application is proposed.

[0192] The third compression network model includes a third encoder, a third decoder, a third quantizer, and a third conditional entropy model, wherein the third decoder is configured to reconstruct a second reconstructed image based on the second image code stream and the reconstructed disparity image. After the step of extracting the difference between the second image and the transformed image to obtain residual data, the method includes:

[0193] Step B510: extracting the residual data through the third encoder to obtain the second image feature;

[0194] It should be noted that the above residual data can also be obtained by extracting the difference between the second image and the transformed image based on the third encoder. Similarly, the first image is a right-eye image and the second image is a left-eye image. Figure 6 , based on the left image x l and rollover images The difference between them is extracted to obtain residual data, and the second image feature y is extracted from the residual data based on the third encoder. l .

[0195] Step B520, obtaining a second quantized image feature of the second image feature through the third quantizer;

[0196] Step B530: inputting the second quantized image feature and the second image feature into the third conditional entropy model to obtain the second image code stream;

[0197] Specifically, the second image feature and the second quantized image feature obtained by quantizing the second image feature through the third quantizer are combined and input into the third conditional entropy model for encoding to obtain the second image code stream.

[0198] Furthermore, the preset third conditional entropy model includes a third super a priori encoding subnetwork, a third super a priori decoding subnetwork, a third context prediction subnetwork, a priori fine-tuning network, a third entropy parameter estimation subnetwork, a third quantizer, a third entropy encoder and a third Gaussian model, and step B530 includes:

[0199] Step B531: input the second image feature into the third super-a priori encoding sub-network to obtain a third super-a priori parameter;

[0200] Step B532, inputting the third super a priori parameter into the third quantizer to obtain a third quantization super a priori parameter;

[0201] Step B533: input the third quantized super-prior parameter into the third super-prior decoding sub-network to obtain a third up-sampled super-prior feature;

[0202] Step B534: input the second quantized image feature into the third context prediction subnetwork to obtain a third dependency result between any third pixel in the second quantized image feature and a decoded pixel adjacent to the third pixel;

[0203] Step B535 , transforming the first quantized image feature based on the reconstructed disparity image to obtain aligned cross-viewpoint priori image features;

[0204] Step B536: inputting the aligned cross-view prior image features into the prior fine-tuning network to obtain the fine-tuned new aligned cross-view prior image features;

[0205] Step B537: Combining the third up-sampled super-prior feature, the third dependency result, and the new aligned cross-viewpoint prior image feature into the third entropy parameter estimation subnetwork to estimate the third parameters of the third Gaussian model;

[0206] Step B538: using the third Gaussian model configured with the third parameter as a third probability distribution function of the second quantized image feature;

[0207] Step B539: Input the second quantized image features and the third probability distribution function into the third entropy encoder to obtain the second image code stream.

[0208] Specifically, refer to Figure 9 , which is a structural diagram of the third conditional entropy model in the binocular image compression method of the present application. The dotted area in the figure is the third conditional entropy model. The second image feature is input into the third super-prior encoding sub-network to obtain a third super-prior parameter. The third super-prior parameter is input into the third quantizer to obtain a third quantized super-prior parameter. The third quantized super-prior parameter is input into the third super-prior decoding sub-network to obtain a third up-sampled super-prior feature. The second quantized image feature is input into the third context prediction sub-network to obtain a third dependency result between any third pixel in the second quantized image feature and the decoded pixel adjacent to the third pixel. Based on the reconstructed disparity image The first quantitative image feature Transformation to obtain aligned cross-view prior image features The aligned cross-view prior image features are input into the prior fine-tuning network for fine-tuning to obtain new aligned cross-view prior image features. The third up-sampled super-prior feature, the third dependency result and the new aligned cross-viewpoint prior image feature are all input into the third entropy parameter estimation sub-network to estimate the third parameter of the third Gaussian model. Let the third parameter be {μ l ,σ l Thus, the compression degree of the second image code stream obtained by compressing the second quantized image feature can be further improved. The third Gaussian model configured with the third parameter is used as the third probability distribution function of the second quantized image feature (the third probability distribution function can be expressed as:

[0209]

[0210] Where, is the third quantized hyper-prior parameter, is the new aligned cross-viewpoint prior image feature obtained through fine-tuning, {μ l ,σ l} is the third parameter. The second quantized image features and the third probability distribution function are input into the third entropy encoder to obtain the second image code stream. It should be noted that, in this embodiment, in addition to the third upsampling super-prior features and the third dependent results, new aligned cross-viewpoint prior image features are added to the third entropy parameter estimation subnetwork, wherein the new aligned cross-viewpoint prior image features are obtained by changing the decoded first quantized image features based on the disparity image, which is equivalent to adding new features. Therefore, in the case of adding features, the third parameter of the third Gaussian model estimated by the third entropy parameter estimation subnetwork can be estimated more accurately, that is, the third probability distribution function of the second quantized image features is more accurate, and correspondingly, the degree of compression of the second quantized image features by the third entropy encoder will be further improved. In addition, the third compression network includes a third decoder, with reference to Figure 6 , the third decoder will be based on the second quantized image features obtained by decoding and the transformed image obtained by reconstruction Reconstruct the second reconstructed image (reconstruct the right eye image in the figure), and the second reconstructed image can be used to construct the subsequent loss function.

[0211] Further, refer to Figure 5 Based on the fourth embodiment of the binocular image compression method of the present application, a fifth embodiment of the binocular image compression method of the present application is proposed.

[0212] The binocular image compression method further includes:

[0213] Step S70, constructing a loss function based on the first image stream, the disparity image stream, the second image stream, the first image, the first reconstructed image, the second image, and the second reconstructed image;

[0214] The loss function is as follows:

[0215]

[0216] Where R l represents the average number of bits per pixel of the second image stream, R r represents the average number of bits per pixel of the first image stream, R d represents the average number of bits per pixel of the parallax image stream, D is the distortion function, λ is the weight, and x l For the second image, is the second reconstructed image, x r For the first image, is the first reconstructed image.

[0217] Step S80: Iteratively optimize the first compression network, the second compression network, the third compression network, and the preset stereo matching model based on the loss function.

[0218] Specifically, in order to balance the efficiency of binocular image compression and the benefits brought by compression (i.e., the quality of the image obtained by decoding and reconstructing the compressed code stream), in this embodiment, a loss function is constructed based on the first image code stream, the parallax image code stream, the second image code stream, the first image, the first reconstructed image, the second image, and the second reconstructed image. The loss function is as follows:

[0219]

[0220] Where R l represents the average number of bits per pixel of the second image stream, R r represents the average number of bits per pixel of the first image stream, R d represents the average number of bits per pixel of the parallax image stream, D is the distortion function, λ is the weight, and x l For the second image, is the second reconstructed image, x r For the first image, is the first reconstructed image.

[0221] Among them, R r The formula is as follows:

[0222]

[0223] In the formula, E represents the expected value in arithmetic coding, is the first probability distribution function, is the first quantitative image feature, is the first quantized hyper-prior parameter, A probability distribution function representing the first quantized hyper-prior parameter.

[0224] Among them, R d The formula is as follows:

[0225]

[0226] In the formula, E represents the expected value in arithmetic coding, is the second probability distribution function, is the second quantitative image feature, is the second quantized hyper-prior parameter, represents the probability distribution function of the second hyper-prior parameter.

[0227] Among them, R l The formula is as follows:

[0228]

[0229] In the formula, E represents the expected value in arithmetic coding, is the third probability distribution function, is the second quantitative image feature, is the third quantized hyper-prior parameter, is the new aligned cross-viewpoint prior image feature obtained through fine-tuning, A probability distribution function representing the third quantized hyper-prior parameter.

[0230] It should be noted that, under normal circumstances, the greater the degree of compression of the binocular image, the lower the quality of the corresponding image obtained by decoding and reconstruction will be. Technicians can set the weight λ of the above loss function according to the requirements for binocular image compression and image quality requirements. Based on the above loss function, the first compression network, the second compression network, the third compression network and the preset stereo matching model can be iteratively optimized. The iterative optimization process can be to continuously adjust the parameters of the above network or model according to the results of the loss function until the results of the loss function meet the preset requirements, and the first compression network, the second compression network, the third compression network and the preset stereo matching model that meet the compression requirements and image quality requirements can be obtained. In addition, referring to Figure 9 Between the third quantizer and the third super-a priori decoding sub-network, a third entropy encoder and a third entropy decoder are included, which may also correspond to an arithmetic decoder and an arithmetic encoder. In this embodiment, the conditional entropy model can flexibly design a network structure based on actual needs, and the present invention does not impose specific limitations on the network structure.

[0231] It can be understood that, compared with the prior art, all components (networks or modules) in this application can be trained end-to-end. By jointly optimizing the components through the loss function, the network can learn how to adaptively assign corresponding bit rates to the first image stream, the disparity image stream, and the second image, and can achieve better rate-distortion performance of the binocular image pair at the cost of the minimum disparity map bit rate.

[0232] In addition, an embodiment of the present application also proposes a binocular image compression device, which includes: a memory, a processor, and a binocular image compression program stored on the memory and runnable on the processor. When the binocular image compression program is executed by the processor, the steps of the binocular image compression method described above are implemented.

[0233] In addition, an embodiment of the present application further proposes a readable storage medium, on which a binocular image compression program is stored. When the binocular image compression program is executed by a processor, the steps of the binocular image compression method as described above are implemented.

[0234] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0235] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0236] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0237] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A binocular image compression method, characterized in that: The binocular image includes a first image and a second image, and the binocular image compression method includes the following steps: Extracting the difference between the first image and the second image based on a preset stereo matching model to obtain a disparity image; Compress and reconstruct the first image based on a first compression network to obtain a first reconstructed image; Compressing and reconstructing the disparity image based on a second compression network to obtain a reconstructed disparity image; transforming the first reconstructed image based on the reconstructed disparity image to obtain a transformed image; extracting a difference between the second image and the transformed image to obtain residual data; A first image code stream obtained by compressing the first image based on the first compression network, a disparity image code stream obtained by compressing the disparity image based on the second compression network, and a second image code stream obtained by compressing the residual data based on the third compression network model are used as the compressed code stream corresponding to the binocular image.

2. The binocular image compression method according to claim 1, wherein: The first compression network includes a first encoder, a first decoder, a first quantizer, and a first conditional entropy model. The step of compressing and reconstructing the first image based on the first compression network to obtain a first reconstructed image includes: extracting a first image feature of the first image by the first encoder; obtaining a first quantized image feature of the first image feature by the first quantizer; Inputting the first quantized image feature and the first image feature into the first conditional entropy model to obtain the first image code stream; The first image code stream is decoded and reconstructed by the first decoder to obtain the first reconstructed image.

3. The binocular image compression method according to claim 2, wherein: The second compression network includes a second encoder, a second decoder, a second quantizer, and a second conditional entropy model. The step of compressing and reconstructing the disparity image based on the second compression network to obtain a reconstructed disparity image includes: extracting a disparity image feature of the disparity image by the second encoder; Obtaining a quantized disparity image feature of the disparity image feature by the second quantizer; Inputting the quantized disparity image features and the disparity image features into the second conditional entropy model to obtain the disparity image code stream; The disparity image code stream is decoded and reconstructed by the second decoder to obtain the reconstructed disparity image.

4. The binocular image compression method according to claim 3, wherein: The third compression network model includes a third encoder, a third decoder, a third quantizer, and a third conditional entropy model. After the step of extracting the difference between the second image and the transformed image to obtain residual data, the method includes: Extracting the residual data by the third encoder to obtain the second image feature; obtaining a second quantized image feature of the second image feature by the third quantizer; Inputting the second quantized image feature and the second image feature into the third conditional entropy model to obtain the second image code stream; The third decoder is configured to reconstruct a second reconstructed image based on the second image code stream and the reconstructed disparity image.

5. The binocular image compression method according to claim 4, wherein: The first conditional entropy model includes a first super-prior encoding subnetwork, a first super-prior decoding subnetwork, a first context prediction subnetwork, a first entropy parameter estimation subnetwork, a first quantizer, a first entropy encoder and a first Gaussian model, and the step of inputting the first quantized image feature and the first image feature into the first conditional entropy model to obtain the first image code stream includes: Inputting the first image feature into the first super-prior encoding sub-network to obtain a first super-prior parameter; Inputting the first super-prior parameter into the first quantizer to obtain a first quantized super-prior parameter; Inputting the first quantized super-prior parameter into the first super-prior decoding sub-network to obtain a first upsampled super-prior feature; Inputting the first quantized image feature into the first context prediction subnetwork to obtain a first dependency result between any first pixel in the first quantized image feature and a decoded pixel adjacent to the first pixel; Combining the first upsampled super-prior feature and the first dependency result and inputting them into the first entropy parameter estimation subnetwork to estimate the first parameter of the first Gaussian model; Using the first Gaussian model configured with the first parameters as a first probability distribution function of the first quantized image feature; The first quantized image feature and the first probability distribution function are input into the first entropy encoder to obtain the first image code stream.

6. The binocular image compression method according to claim 5, wherein: The second conditional entropy model includes a second super priori encoding subnetwork, a second super priori decoding subnetwork, a second context prediction subnetwork, a second entropy parameter estimation subnetwork, a second quantizer, a second entropy encoder and a second Gaussian model, and the step of inputting the quantized disparity image features and the disparity image features into the second conditional entropy model to obtain the disparity image code stream includes: Inputting the disparity image feature into the second super-prior encoding sub-network to obtain a second super-prior parameter; Inputting the second super-prior parameter into the second quantizer to obtain a second quantized super-prior parameter; Inputting the second quantized super-prior parameter into the second super-prior decoding sub-network to obtain a second up-sampled super-prior feature; Inputting the quantized disparity image feature into the second context prediction subnetwork to obtain a second dependency result between any second pixel in the quantized disparity image feature and a decoded pixel adjacent to the second pixel; Combining the second up-sampled super-prior feature and the second dependency result and inputting them into the second entropy parameter estimation subnetwork to estimate the second parameters of the second Gaussian model; Using the second Gaussian model configured with the second parameter as a second probability distribution function of the quantized disparity image feature; The quantized disparity image features and the second probability distribution function are input into the second entropy encoder to obtain the disparity image code stream.

7. The binocular image compression method according to claim 6, wherein: The third conditional entropy model includes a third super a priori encoding subnetwork, a third super a priori decoding subnetwork, a third context prediction subnetwork, a priori fine-tuning network, a third entropy parameter estimation subnetwork, a third quantizer, a third entropy encoder, and a third Gaussian model. The step of inputting the second quantized image feature and the second image feature into the third conditional entropy model to obtain the second image code stream includes: Inputting the second image feature into the third super-prior encoding sub-network to obtain a third super-prior parameter; Inputting the third super a priori parameter into the third quantizer to obtain a third quantized super a priori parameter; Inputting the third quantized super-prior parameter into the third super-prior decoding sub-network to obtain a third up-sampled super-prior feature; Inputting the second quantized image feature into the third context prediction subnetwork to obtain a third dependency result between any third pixel in the second quantized image feature and a decoded pixel adjacent to the third pixel; transforming the first quantized image feature based on the reconstructed disparity image to obtain aligned cross-viewpoint prior image features; Inputting the aligned cross-view prior image features into the prior fine-tuning network to obtain new aligned cross-view prior image features after fine-tuning; Combining the third up-sampled super-prior feature, the third dependency result, and the new aligned cross-viewpoint prior image feature and inputting them into the third entropy parameter estimation subnetwork to estimate the third parameters of the third Gaussian model; Using the third Gaussian model configured with the third parameter as a third probability distribution function of the second quantized image feature; The second quantized image feature and the third probability distribution function are input into the third entropy encoder to obtain the second image code stream.

8. The binocular image compression method according to claim 7, wherein: The binocular image compression method further includes: A loss function is constructed based on the first image code stream, the disparity image code stream, the second image code stream, the first image, the first reconstructed image, the second image, and the second reconstructed image, wherein the loss function is as follows: Where R l represents the average number of bits per pixel of the second image stream, R r represents the average number of bits per pixel of the first image stream, R d represents the average number of bits per pixel of the parallax image stream, D is the distortion function, λ is the weight, and x l For the second image, is the second reconstructed image, x r For the first image, for the first reconstructed image; The first compression network, the second compression network, the third compression network and the preset stereo matching model are iteratively optimized based on the loss function.

9. A binocular image compression device, characterized in that: The binocular image compression device includes: a memory, a processor, and a binocular image compression program stored in the memory and executable on the processor. When the binocular image compression program is executed by the processor, the steps of the binocular image compression method according to any one of claims 1 to 8 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a binocular image compression program, and when the binocular image compression program is executed by a processor, the steps of the binocular image compression method according to any one of claims 1 to 8 are implemented.

Images