A dark light field image compression method based on probability representation

Abstract

The application discloses a dark light field image compression method based on a probability representation, and comprises the following steps: acquiring a to-be-processed light field image, inputting the to-be-processed light field image into a deep compression model based on a probability representation for processing, and outputting a processed light field image; wherein the deep compression model based on the probability representation comprises a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low-light enhancement module. The application improves the performance of dark light field image compression and reduces the consumption of image data storage resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image signal processing, in particular to a dark light field image compression method based on probability representation. BACKGROUND

[0002] Light field imaging can capture four-dimensional information of a scene, including spatial and angular information of light rays, which is crucial for the reproduction of three-dimensional visual effects. For decades, in the field of image signal processing, dark light field image compression has always been a basic and important topic that people continue to explore and research, aiming to achieve efficient dark light field image compression and storage. However, some traditional dark light field image compression methods have defects, such as the traditional method of image partitioning, which can cause block effect problems. Secondly, when the compression rate is high, the traditional compression method will show artifacts. At the same time, the traditional method is difficult to improve, and its performance depends on the level of a single module, which increases the difficulty of optimization and improvement. Therefore, a better compression method is needed to obtain better image quality while reducing bit consumption, which also helps to save file storage space on the network, and to achieve faster, more efficient image sharing and exchange, and to obtain better quality of experience.

[0003] However, the existing dark light field image compression model based on deep learning only considers eliminating the redundancy between features from the spatial dimension, while ignoring the feature redundancy between angles, channels and epipolar planes, which limits the improvement of dark light field image compression performance. SUMMARY

[0004] To solve the above technical problems existing in the prior art, the present application provides a dark light field image compression method based on probability representation, which shows that dark light field images can not only be analyzed visually, but also useful information can be dynamically selected by introducing probability distribution, thereby providing a novel and effective solution for dark light field image compression.

[0005] To achieve the above purpose, the present application provides a dark light field image compression method based on probability representation, comprising:

[0006] obtaining a to-be-processed light field image, inputting the to-be-processed light field image into a deep compression model based on probability representation for processing, and outputting a processed light field image;

[0007] The deep compression model based on probability representation comprises a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low light enhancement module connected in sequence.

[0008] Preferably, inputting the to-be-processed light field image into the deep compression model based on probability representation for processing comprises:

[0009] extracting features of the to-be-processed light field image through the probabilistic feature representation module, wherein the features include spatial features, angular features, and epipolar plane features;

[0010] initializing the extracted features based on the encoder, mapping low-dimensional features to high-dimensional features, and performing compression encoding to generate a latent feature encoding matrix;

[0011] performing quantization encoding on the latent feature encoding matrix using the quantization module to obtain a quantized latent feature representation;

[0012] calculating the latent feature encoding matrix through the entropy evaluation module and estimating the total number of bits required for the quantized latent feature representation under the specific Gaussian distribution;

[0013] performing decoding on the quantized latent feature representation based on the decoder to generate a reconstructed image, performing brightening on the reconstructed image through the low-light enhancement module, and outputting the processed light field image.

[0014] Preferably, the features of the to-be-processed light field image are extracted, including:

[0015] segmenting the to-be-processed light field image into a plurality of small block images, and inputting the plurality of small block images into the probabilistic feature representation module for feature extraction; wherein the probabilistic feature representation module includes a first feature embedding unit, a second feature embedding unit, and a feature mixed convolution unit, the first feature embedding unit dynamically extracts first feature information by following a learnable independent Bernoulli distribution binary mask, the second feature embedding unit dynamically extracts second feature information by following a learnable independent Bernoulli distribution binary mask, and the feature mixed convolution unit is used to convolve the second feature information to generate a processed feature matrix.

[0016] Preferably, the encoder includes an Angular Transformer unit, a non-overlapping sliding window attention unit, a down-sampling convolution unit, and a generated split normalization unit, wherein the Angular Transformer unit is used to transform the processed feature matrix into feature data through a rearrange function; the non-overlapping sliding window attention unit is used to allow information to pass between adjacent windows by introducing a displacement window mechanism, and to maintain the flow of global information within the local window; the down-sampling convolution unit and the generated split normalization unit are used to perform compression encoding of dark light field picture features until the latent feature encoding matrix is generated.

[0017] Preferably, the quantized latent feature representation is obtained, including:

[0018] Adding uniformly distributed noise in the quantization encoding operation stage makes the quantization process derivable, and converts the floating-point numbers into integers through the rounding method to obtain the latent feature representation.

[0019] Preferably, the entropy evaluation module is used to calculate the latent feature encoding matrix, including:

[0020] The latent feature encoding matrix is input into the entropy evaluation module for bit rate estimation to obtain Gaussian model parameters.

[0021] The entropy evaluation module includes a hyper-encoder, a quantization unit, and a hyper-decoder. The hyper-encoder is composed of a 2D convolution unit, two down-sampling convolution units, and two LeakyReLU activation functions. The hyper-decoder is composed of two up-sampling convolution units, two LeakyReLU activation functions, and a 2D convolution unit.

[0022] The hyper-encoder is used to estimate the edge distribution of the latent feature encoding matrix, i.e., the prior information, and compress the prior information. The quantization unit is used to reduce the precision of the prior information. The hyper-decoder is used to reconstruct the prior information.

[0023] Preferably, the total number of bits required by the quantized latent feature representation under a specific Gaussian distribution is estimated, including:

[0024] The quantized latent feature representation is compressed using an arithmetic encoder. A learning-based hyper-prior compression method is used to model the compressed latent feature representation as a Gaussian distribution. Based on the Gaussian model parameters calculated by the hyper-encoder and the hyper-decoder, a specific Gaussian distribution is constructed for each data, and the total number of bits required by the quantized latent feature representation under the specific distribution is estimated.

[0025] Preferably, the reconstructed image is generated, including:

[0026] The quantized latent feature representation is input into an arithmetic decoder for decompression, and then input into the decoder for latent feature decoding. The decoder is composed of an Angular Transformer unit, a non-overlapping sliding window attention unit, an up-sampling convolution unit, and an inverse generative split normalization unit. The Angular Transformer unit is used to capture the panoramic angle dependence that traditional convolution cannot capture. The non-overlapping sliding window attention unit is used to introduce residual connection to improve the quality of intermediate features. The up-sampling convolution unit is used to increase the resolution of the picture and gradually reconstruct the picture. The inverse generative split normalization unit is used for inverse normalization processing to restore the data distribution characteristics.

[0027] Preferably, output the processed light field image, comprising:

[0028] The high-dimensional feature matrix output by the decoder is input into the low-light enhancement module for brightening operation to generate a light field picture under normal lighting, i.e., the processed light field image.

[0029] The low-light enhancement module is composed of a LayerNorm unit, a processing convolution unit, an inverse processing convolution unit, a depth separable convolution unit, and a plurality of linear layers; the LayerNorm unit is used for standardization operation on image features; the processing convolution unit is used for increasing image color channels; the inverse processing convolution unit is used for reducing image color channel numbers, integrating and refining feature information, and generating the reconstructed image; the depth separable convolution unit is used for layered processing with each color channel to isolate noise interference between different channels; and the linear layer is used for generating query, key, and value components of a self-attention mechanism.

[0030] Compared with the prior art, the present application has the following advantages and technical effects:

[0031] The method of the present application dynamically selects useful features based on adaptive feature probability, which is quite different from the existing method of combining spatial and angular features based on experience. The present application improves the performance of dark light field image compression and reduces the consumption of image data storage resources. The low-light enhancement module (LLE) is used to perform global color enhancement and local refinement to achieve the purpose of dark light image enhancement. The present application significantly improves the efficiency and accuracy of dark light field image compression, and has important application value and broad application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which form a part of this application, are intended to provide further understanding of the application and are incorporated herein for a purpose of explanation and are not intended to constitute an undue limitation of the application. In the drawings:

[0033] Figure 1 A flow chart of a dark light field image compression method based on probability representation according to an embodiment of the present application;

[0034] Figure 2 A schematic diagram of a deep compression model based on probability representation according to an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a probability feature representation module according to an embodiment of the present application;

[0036] Figure 4 A schematic diagram of an Angular Transformer unit according to an embodiment of the present application;

[0037] Figure 5 A non-overlapping sliding window attention unit structure diagram of an embodiment of the present application;

[0038] Figure 6 A low-light enhancement module structure diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0039] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0041] The embodiment proposes a dark light field image compression method based on probability representation, comprising:

[0042] Obtaining a to-be-processed light field image, inputting the to-be-processed light field image into a deep compression model based on probability representation for processing, and outputting a processed light field image;

[0043] The deep compression model based on probability representation comprises a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low-light enhancement module.

[0044] The embodiment method follows a learnable independent Bernoulli distribution binary mask to dynamically select useful features, improves the performance of dark light field image compression through model parameter sharing and loss weight optimization, effectively solves the shortcomings of the prior art, such as that the existing deep compression model does not process the local detail lines of the dark light field image well, only considers eliminating the redundancy between features from the spatial dimension, ignores the feature redundancy between angles, channels and polar planes, and the improvement of the dark light field image compression performance is limited. The embodiment shows that the dark light field image can not only be analyzed visually, but also dynamically select useful information by introducing probability distribution, thereby providing a novel and effective solution for dark light field image compression.

[0045] Further, the to-be-processed light field image is input into a deep compression model based on probability representation (such as Figures 1-2 ) for processing, comprising:

[0046] The probability feature representation module extracts the features of the to-be-processed light field image, including spatial features, angle features and polar plane features;

[0047] Based on the initialization of the extracted features by the encoder, the low-dimensional features are mapped to high-dimensional features and compressed encoding is performed to generate a latent feature encoding matrix;

[0048] The latent feature encoding matrix is quantized and encoded using the quantization module to obtain a quantized latent feature representation.

[0049] The latent feature encoding matrix is calculated by the entropy evaluation module, and the total number of bits required for the quantized latent feature representation under the specific Gaussian distribution is estimated.

[0050] Based on the decoder, the quantized latent feature representation is decoded to generate a reconstructed image, and the low-light enhancement module is used to brighten the reconstructed image to output the processed light field image.

[0051] Further, the features of the light field image to be processed are extracted, including:

[0052] The light field image to be processed is divided into a plurality of small block images, and the plurality of small block images are input into the probability feature representation module for feature extraction. The probability feature representation module includes a first feature embedding unit, a second feature embedding unit and a feature mixed convolution unit. The first feature embedding unit dynamically extracts first feature information following a learnable independent Bernoulli distribution binary mask. The second feature embedding unit dynamically extracts second feature information following a learnable independent Bernoulli distribution binary mask. The feature mixed convolution unit is used to convolve the second feature information to generate a processed feature matrix.

[0053] Specifically, the dark light and high bit light field image to be processed is input into the probability feature representation module, and spatial, angular and EPI useful features are extracted respectively.

[0054] The light field image to be processed can be represented as L(u, v, h, w, c), where (u, v) represents the angular dimension (sub-aperture view) of the light field image, (h, w) is the spatial dimension, and c is the number of color channels (RGB channels). In order to enable the image to be processed by the network and reduce the amount of calculation during model training and inference, the image data is first preprocessed, and the light field image to be processed is cropped or adjusted to the input size required by the network. The processed dark light field picture x has an input dimension of (5, 5, 3, 64, 64).

[0055] The obtained processed dark light field picture data is input into the probability feature representation module, such as Figure 3As shown, the probability feature representation module (PFR) has a total of 8 feature embedding units, and the front feature embedding units dynamically provide useful information for the rear feature embedding units by following a learnable independent Bernoulli distribution binary mask. Each feature embedding unit is composed of Conv_spa, Conv_ang, Conv_epi_h, Conv_epi_w, Conv_vsi_v and Conv_vsi_h, six carefully designed convolutional operations for extracting angle, space, horizontal epipolar plane, vertical epipolar plane, horizontal virtual slit image, and vertical virtual slit image features, respectively, to roughly remove unnecessary redundancy. Similarly, by following a learnable independent Bernoulli distribution binary mask, more useful features are dynamically selected to further remove unnecessary redundancy, and finally a 3x3 mixed convolution is used to mix the extracted features to enhance the compression capability of the model. Finally, the dark light field picture processed by the probability feature representation module (PFR) has a dimension of (batch_size, 3, 64, 64), where batch_size = u*v.

[0056] Further, based on the encoder, the extracted features are initialized, the low-dimensional features are mapped to high-dimensional features, and compressed encoding is performed to generate a latent feature encoding matrix.

[0057] The encoder includes an Angular Transformer unit, a non-overlapping sliding window attention unit, a down-sampling convolution unit, and a generated split normalization unit. The Angular Transformer unit is used to transform the processed feature matrix into feature data through a rearrange function. The non-overlapping sliding window attention unit allows information to pass between adjacent windows by introducing a displacement window mechanism, and maintains the flow of global information within the local window. The down-sampling convolution unit and the generated split normalization unit are used to perform compression encoding of the dark light field picture features until the latent feature encoding matrix is generated.

[0058] Specifically, the encoder (Encoder) first sends each dark light field picture into an initialization convolution, which maps low-dimensional features to high-dimensional features. This not only extracts and retains the key features of the image, but also enhances the feature expression capability of the model, improves the efficiency and accuracy of the model, and enhances the generalization ability of the model. The processed feature matrix has a dimension of (batch_size, 192, 64, 64).

[0059] The encoder is composed of 2 Angular Transformer units, 2 non-overlapping sliding window attention units, 4 5x5 down-sampling convolution units and 3 generated split normalization units. In this embodiment, except for the last down-sampling convolution unit, there is a generated split normalization unit after each down-sampling convolution unit to achieve the maximum compression bit rate of the dark light image.

[0060] As shown in Figure 4 Each Angular Transformer unit contains 2 LayerNorm, 1 Multi-Head Self-Attention (MHSA), 1 Feed-Forward Network (FFN) and 2 residual connections, similar to the traditional Transformer. However, unlike the traditional Transformer, the Angular Transformer unit focuses on the relationship between the pixel points at the same position in multiple angles, so it needs to transform the feature matrix into a feature data with dimensions of (64*64, 192, 5, 5) through the rearrange function, where (u, v) is (5, 5) representing the angle dimension (sub-aperture view) of the light field image. The feature dimension after angle feature enhancement extraction by the Angular Transformer unit is (64*64, 192, 5, 5), and then it is transformed into a feature data with dimensions of (batch_size, 192, 64, 64) through the rearrange function again and sent to the non-overlapping sliding window attention unit (NOSWA). Based on the non-overlapping spatial adjacent elements, the attention mask can effectively extract spatial features with less computational cost, thereby improving the compression performance, so this embodiment proposes as Figure 5The non-overlapping sliding window attention unit (NOSWA) shown, which contains 1 sliding window attention, 2 residual blocks (ResBlock) and a Sigmoid function, to realize by introducing displacement window mechanism, allows information to pass between adjacent windows, so as to maintain the flow of global information within the local window. Similarly, the feature dimension after spatial feature enhancement extraction by the non-overlapping sliding window attention unit (NOSWA) is (batch_size, 192, 64, 64). The features extracted by angle and space are sent to the first 5x5 down-sampling convolution unit and the generated split normalization unit (GDN) for the first feature compression encoding of the dark light field picture. The dimension of the feature matrix after processing is (batch_size, 192, 32, 32). The above is the whole process of the first feature compression encoding of the dark light field picture. The next 3 compression encoding processes are similar to the first one. The feature matrix dimensions after each compression encoding processing are (batch_size, 192, 16, 16), (batch_size, 192, 8, 8), and (batch_size, 192, 4, 4), respectively. The feature matrix dimension of the latent feature representation (y) after the encoder compression encoding is (batch_size, 192, 4, 4), that is, the latent feature encoding matrix.

[0061] Further, obtaining the quantized latent feature representation includes: adding uniformly distributed noise to the quantization encoding operation stage to make the quantization process derivable, and then converting the floating-point number into an integer by rounding to obtain the latent feature representation.

[0062] Specifically, the generated encoded latent feature representation (y) is sent to the quantization module for quantization encoding. Considering that the quantization process itself is not derivable, this embodiment makes the quantization process derivable by adding uniformly distributed noise, and then converts the floating-point number into an integer by rounding to realize the quantization operation, further improving the compression efficiency. Finally, the latent representation after quantization encoding is denoted as The dimension is (batch_size, 192, 4, 4), that is, the quantized latent feature representation.

[0063] Further, the latent feature encoding matrix is calculated by an entropy evaluation module, including:

[0064] The latent feature encoding matrix is input into the entropy evaluation module for bit rate estimation to obtain Gaussian model parameters;

[0065] The entropy evaluation module comprises a hyper encoder, a quantization unit and a hyper decoder, the hyper encoder is composed of a 2D convolution unit, two down-sampling convolution units and two LeakyReLU activation functions; the hyper decoder is composed of two up-sampling convolution units, two LeakyReLU activation functions and a 2D convolution unit.

[0066] The hyper encoder is used for estimating the edge distribution of the latent feature coding matrix, i.e. prior information, and compressing the prior information; the quantization unit is used for reducing the precision of the prior information and significantly reducing the inference speed of the model; the hyper decoder is used for reconstructing the prior information, so as to eliminate the spatial redundancy in the prior information and calculate the parameters of the Gaussian distribution of the latent feature.

[0067] Specifically, the latent feature coding matrix is input into the entropy evaluation model to perform bit rate estimation. In the embodiment, it is considered that the target of the image compression method is to obtain a high-quality reconstructed image under a given bit rate target, and the entropy model is used to estimate the bit rate, so it is crucial to establish an accurate entropy model.

[0068] The entropy evaluation model proposed in the embodiment is similar to a variational autoencoder (VAE), which applies a hyper encoder, a quantization unit and a hyper decoder to estimate the corresponding parameters Φ in the entropy model. The hyper encoder comprises one 3x3 2D convolution unit, two down-sampling 5x5 convolution units and two LeakyReLU activation functions; the hyper decoder comprises two up-sampling 5x5 convolution units, two LeakyReLU activation functions and one 3x3 2D convolution unit. A hyper-prior compression scheme is adopted, and the latent feature representation is modeled as a Gaussian distribution, the parameters μ and σ of the Gaussian model are estimated through the hyper encoder and the hyper decoder, a specific Gaussian distribution is constructed, and the quantized latent feature representation is approximately estimated under the specific Gaussian distribution.

[0069] Further, the total number of bits required by the quantized latent feature representation under the specific Gaussian distribution is estimated, which comprises:

[0070] The quantized latent feature representation is compressed using an arithmetic encoder, which is modeled as a Gaussian distribution by adopting a learning-based hyper-prior compression scheme. According to the Gaussian model parameters μ and σ estimated by the hyper encoder and the hyper decoder, a specific Gaussian distribution is constructed for each data, and the quantized latent feature representation is approximately estimated under the specific distribution.

[0071] Further, the reconstructed image is generated, comprising:

[0072] The quantized latent feature representation is sent to an arithmetic decoder for decompression and then to a decoder for latent feature decoding; wherein the decoder is composed of an Angular Transformer unit, a non-overlapping sliding window attention unit, an up-sampling convolution unit, and an inverse generative split normalization unit. The Angular Transformer unit is used to capture the panoramic angle dependence that traditional convolution cannot capture. The non-overlapping sliding window attention unit is used to introduce residual connection to improve the quality of intermediate features. The up-sampling convolution unit is used to increase the resolution of the picture and gradually reconstruct the picture. The inverse generative split normalization unit is used for inverse normalization processing to restore the data distribution characteristics.

[0073] Specifically, in the present embodiment, the decoder (Decoder) corresponds to the structure of the encoder (Encoder) and is composed of 2 Angular Transformer units, 2 non-overlapping sliding window attention units, 4 5x5 up-sampling convolution units, and 3 inverse generative split normalization units (IGDN) to maximize the dark light image reconstruction performance. The dimensions of the feature matrix after each decoding process are (batch_size, 192, 8, 8), (batch_size, 192, 16, 16), (batch_size, 192, 32, 32), and (batch_size, 192, 64, 64), respectively. Therefore, the final feature matrix dimension after decoding module reconstruction is (batch_size, 192, 64, 64).

[0074] Further, the processed light field image is output, comprising:

[0075] The high-dimensional feature matrix output by the decoder is sent to a low-light enhancement module for brightening operation to generate a light field picture under normal lighting, i.e., the processed light field image;

[0076] The low-light enhancement module is composed of a LayerNorm unit, a processing convolution unit, an inverse processing convolution unit, a depth separable convolution unit, and a plurality of linear layers. The LayerNorm unit is used for standardization operation on image features. The processing convolution unit is used to increase the number of image color channels, thereby increasing the expression ability of color features. The inverse processing convolution unit is used to reduce the number of image color channels, further integrate and refine feature information, and generate a reconstructed image. The depth separable convolution unit is used for layered processing with each color channel to isolate noise interference between different channels and provide purer processing features for the subsequent self-attention mechanism composed of a plurality of linear layers. The linear layer is used to generate the query, key, and value three core components of the self-attention mechanism.

[0077] Specifically, the reconstructed high-dimensional feature matrix is decoded by the decoder and sent to the low-light enhancement module (LLE) to realize the brightening operation of the reconstructed image. Due to the globally shared settings such as ambient lighting and color space specification, the color of the image is mainly enhanced or converted to another color space through channel matrix transformation. Following this principle, as shown in the figure, the embodiment introduces a low-light enhancement module (LLE) to perform global color enhancement and local refinement. The design of this module benefits from the latest progress of self-attention by extracting Query, Key, Value in self-attention through 3 linear layers (Liner), where the dimensions of Query, Key, Value are all (HW, C). The matrix multiplication of the transposed Query and Key obtains a feature matrix of (HW, HW), and the matrix multiplication of the (HW, HW) feature matrix and Value obtains a feature matrix of (HW, C), thereby realizing the operation of light enhancement. Finally, the enhanced image after the low-light enhancement module (LLE) has a dimension of (batch_size, 3, 64, 64), which is the same as the size of the original image. Figure 6

[0078] The difference between the reconstructed image and the ground truth (Ground Truth) and the bit rate of the latent feature representation jointly constitute the loss function of the model, which is specifically described as:

[0079]

[0080] Where D and R represent distortion and bit rate respectively, λ is the trade-off parameter, represents the picture after reconstruction and enhancement, Y is the ground truth (Ground Truth) of the picture under normal lighting, the bit rate of the latent feature representation estimated by the entropy estimation module.

[0081] By continuously reducing the loss of the model through training, the performance of the entire model is optimized.

[0082] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application can be easily thought of by those skilled in the art, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.​​

Claims

1. A probabilistic representation based dim light light field image compression method, characterized in that, The method comprises the following steps: obtaining a to-be-processed light field image, inputting the to-be-processed light field image into a deep compression model based on a probability representation for processing, and outputting a processed light field image; wherein the deep compression model based on the probability representation comprises a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low-light enhancement module connected in sequence; inputting the to-be-processed light field image into the deep compression model based on the probability representation for processing comprises: extracting features of the to-be-processed light field image by the probability feature representation module, wherein the features include spatial features, angle features and epipolar plane features; initializing the extracted features based on the encoder, mapping low-dimensional features to high-dimensional features, and performing compression encoding to generate a latent feature encoding matrix; performing quantization encoding operation on the latent feature encoding matrix using the quantization module to obtain a quantized latent feature representation; calculating the latent feature encoding matrix by the entropy evaluation module and estimating the total number of bits required for the quantized latent feature representation under the specific Gaussian distribution; performing decoding operation on the quantized latent feature representation based on the decoder to generate a reconstructed image, and performing brightening operation on the reconstructed image by the low-light enhancement module to output the processed light field image; obtaining the quantized latent feature representation comprises: adding uniformly distributed noise to the quantization process to make it differentiable, and then converting the floating-point number to an integer by rounding method to obtain the latent feature representation; generating a reconstructed image comprises: sending the quantized latent feature representation into an arithmetic decoder for decompression, and then sending it into the decoder for latent feature decoding; wherein the decoder is composed of an Angular Transformer unit, a non-overlapping sliding window attention unit, an up-sampling convolution unit and an inverse generation split normalization unit; the Angular Transformer unit is used to capture the panoramic angle dependence that traditional convolution cannot capture; the non-overlapping sliding window attention unit is used to introduce residual connection to improve the quality of intermediate features; the up-sampling convolution unit is used to increase the resolution of the picture and gradually reconstruct the picture; the inverse generation split normalization unit is used for inverse normalization processing to restore the data distribution characteristics; outputting the processed light field image comprises: sending the high-dimensional feature matrix output by the decoder into the low-light enhancement module for brightening operation to generate a light field picture under normal lighting, i.e. the processed light field image; The low-light enhancement module is composed of a LayerNorm unit, a processing convolution unit, an inverse processing convolution unit, a depth separable convolution unit, and a plurality of linear layers; the LayerNorm unit is used for standardization operation on image features; the processing convolution unit is used for increasing the image color channel; the inverse processing convolution unit is used for reducing the image color channel number, integrating and refining feature information, and generating the reconstructed image; the depth separable convolution unit is used for layered processing with each color channel to isolate noise interference between different channels; and the linear layer is used to generate query, key, and value components of the self-attention mechanism.

2. The method of claim 1, wherein, The features of the to-be-processed light field image are extracted, including: The to-be-processed light field image is segmented into a plurality of small block images, and the plurality of small block images are input into the probability feature representation module for feature extraction; wherein the probability feature representation module includes a first feature embedding unit, a second feature embedding unit, and a feature mixed convolution unit, the first feature embedding unit dynamically extracts first feature information by following a learnable independent Bernoulli distribution binary mask, the second feature embedding unit dynamically extracts second feature information by following a learnable independent Bernoulli distribution binary mask, and the feature mixed convolution unit is used to convolve the second feature information to generate a processed feature matrix.

3. The method of claim 2, wherein, The encoder includes an Angular Transformer unit, a non-overlapping sliding window attention unit, a down-sampling convolution unit, and a generated split normalization unit, wherein the Angular Transformer unit is used to transform the processed feature matrix into feature data through a rearrange function; the non-overlapping sliding window attention unit is used to allow information to pass between adjacent windows and maintain the flow of global information within the local window by introducing a displacement window mechanism; and the down-sampling convolution unit and the generated split normalization unit are used for compression encoding of dark light field picture features until the latent feature encoding matrix is generated.

4. The method of claim 1, wherein, The latent feature encoding matrix is calculated by the entropy evaluation module, including: The latent feature encoding matrix is input into the entropy evaluation module for bit rate estimation to obtain Gaussian model parameters; The entropy evaluation module includes a hyper-encoder, a quantization unit, and a hyper-decoder, the hyper-encoder is composed of a 2D convolution unit, two down-sampling convolution units, and two LeakyReLU activation functions; the hyper-decoder is composed of two up-sampling convolution units, two LeakyReLU activation functions, and a 2D convolution unit; The hyper-encoder is used to estimate the edge distribution of the latent feature encoding matrix, i.e., prior information, and compress the prior information; the quantization unit is used to reduce the accuracy of the prior information; and the hyper-decoder is used to reconstruct the prior information.

5. The method of claim 4, wherein, The total number of bits required by the quantized latent feature representation under a specific Gaussian distribution is estimated, including: The quantized latent feature representation is compressed using an arithmetic encoder, by employing a learning-based hyper-prior compression method, modeling the compressed latent feature representation as a Gaussian distribution, and constructing a specific Gaussian distribution for each data according to the Gaussian model parameters calculated by the hyper-encoder and the hyper-decoder, estimating the total number of bits required for the quantized latent feature representation under a specific distribution.

Images