Dark light field image compression method based on probability representation

Abstract

The invention discloses a dark light field image compression method based on probability representation, and the method comprises the steps: obtaining a to-be-processed light field image, inputting the to-be-processed light field image into a depth compression model based on probability representation for processing, and outputting the processed light field image; the depth compression model based on probability representation comprises a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low light enhancement module. According to the method, the dark light field image compression performance is improved, and the consumption of image data storage resources is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image signal processing, and in particular to a dark light field image compression method based on probability representation. Background Art

[0002] Light field imaging can capture the four-dimensional information of a scene, including the spatial and angular information of light, which is crucial for the reproduction of three-dimensional visual effects. For decades, in the field of image signal processing, dark light field image compression has always been a fundamental and important topic that people have been exploring and researching, with the aim of achieving efficient dark light field image compression and storage. However, some traditional dark light field image compression methods have defects. For example, the traditional method partitions the image into blocks, which will produce block effects. Secondly, when the compression rate is high, traditional compression methods will show artifacts. At the same time, traditional methods are difficult to improve, and their performance is highly dependent on the level of a single module, which increases the difficulty of optimization and improvement. Therefore, a better compression method is needed to obtain better image quality while reducing bit consumption, which also helps save file storage space on the network and achieve faster and more efficient image sharing and exchange, as well as better experience quality.

[0003] However, the existing deep learning-based dark-light light field image compression model only considers eliminating the redundancy between features from the spatial dimension, but ignores the feature redundancy between angles, channels, and polar planes, which limits the improvement of dark-light light field image compression performance. Summary of the invention

[0004] In order to solve the technical problems existing in the above-mentioned prior art, the present invention proposes a dark-light light field image compression method based on probability representation, which shows that dark-light light field images can not only be analyzed visually but also can dynamically select useful information by introducing probability distribution, thereby providing a novel and effective solution for dark-light light field image compression.

[0005] To achieve the above object, the present invention provides a dark light field image compression method based on probability representation, comprising:

[0006] Acquire a light field image to be processed, input the light field image to be processed into a deep compression model based on probability representation for processing, and output a processed light field image;

[0007] Among them, the deep compression model based on probability representation includes a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low-light enhancement module which are connected in sequence.

[0008] Preferably, the light field image to be processed is input into a deep compression model based on probability representation for processing, comprising:

[0009] Extracting features of the light field image to be processed by the probabilistic feature representation module, including spatial features, angle features and extreme plane features;

[0010] Initializing the extracted features based on the encoder, mapping the low-dimensional features to high-dimensional features, and performing compression encoding to generate a potential feature encoding matrix;

[0011] Using the quantization module to perform a quantization encoding operation on the potential feature encoding matrix to obtain a quantized potential feature representation;

[0012] The entropy evaluation module calculates the potential feature encoding matrix and estimates the total number of bits required for the quantized potential feature representation under the specific Gaussian distribution;

[0013] The decoder performs a decoding operation on the quantized latent feature representation to generate a reconstructed image, and the low-light enhancement module performs a brightening operation on the reconstructed image to output the processed light field image.

[0014] Preferably, extracting the features of the light field image to be processed includes:

[0015] The light field image to be processed is divided into a number of small images, and the several small images are input into the probabilistic feature representation module for feature extraction; wherein the probabilistic feature representation module includes a first feature embedding unit, a second feature embedding unit and a feature mixing convolution unit, the first feature embedding unit dynamically extracts first feature information by following a learnable independent Bernoulli distribution binary mask, the second feature embedding unit dynamically extracts second feature information by following a learnable independent Bernoulli distribution binary mask, and the feature mixing convolution unit is used to perform convolution processing on the second feature information to generate a processed feature matrix.

[0016] Preferably, the encoder comprises an Angular Transformer unit, a non-overlapping sliding window attention unit, a downsampling convolution unit and a generating splitting normalization unit, wherein the Angular Transformer unit is used to transform the processed feature matrix into feature data through a rearrange function; the non-overlapping sliding window attention unit is used to allow information to be transferred between adjacent windows by introducing a displacement window mechanism, thereby maintaining the flow of global information within a local window; the downsampling convolution unit and the generating splitting normalization unit are used to perform compression encoding of dark light field image features until the potential feature encoding matrix is ​​generated.

[0017] Preferably, obtaining the quantized potential feature representation includes:

[0018] In the quantization coding operation stage, uniformly distributed noise is added to make the quantization process differentiable, and then the floating point number is converted into an integer by rounding to obtain the potential feature representation.

[0019] Preferably, calculating the potential feature encoding matrix by the entropy evaluation module includes:

[0020] Inputting the potential feature coding matrix into the entropy evaluation module to perform bit rate estimation to obtain Gaussian model parameters;

[0021] The entropy evaluation module includes a super encoder, a quantization unit and a super decoder, wherein the super encoder is composed of a 2D convolution unit, two downsampling convolution units and two LeakyReLU activation functions; the super decoder is composed of two upsampling convolution units, two LeakyReLU activation functions and a 2D convolution unit;

[0022] The super encoder is used to estimate the marginal distribution of the potential feature coding matrix, that is, the prior information, and compress the prior information; the quantization unit is used to reduce the accuracy of the prior information; and the super decoder is used to reconstruct the prior information.

[0023] Preferably, estimating the total number of bits required for the quantized potential feature representation under a specific Gaussian distribution comprises:

[0024] The quantized latent feature representation is compressed using an arithmetic encoder, and the compressed latent feature representation is modeled as a Gaussian distribution by adopting a learning-based super-prior compression method. A specific Gaussian distribution is constructed for each data based on the Gaussian model parameters calculated by the super-encoder and the super-decoder, and the total number of bits required for the quantized latent feature representation under the specific distribution is estimated.

[0025] Preferably, generating a reconstructed image comprises:

[0026] The quantized latent feature representation is sent to an arithmetic decoder for decompression, and then sent to the decoder for latent feature decoding; wherein the decoder is composed of an Angular Transformer unit, a non-overlapping sliding window attention unit, an upsampling convolution unit and an anti-generation split normalization unit; the Angular Transformer unit is used to capture the panoramic angle dependency that is difficult to capture by traditional convolution; the non-overlapping sliding window attention unit is used to introduce residual connections to improve the quality of intermediate features; the upsampling convolution unit is used to increase the image resolution and gradually reconstruct the image; the anti-generation split normalization unit is used for inverse normalization processing to restore data distribution characteristics.

[0027] Preferably, outputting the processed light field image comprises:

[0028] The high-dimensional feature matrix output by the decoder is sent to the low-light enhancement module for brightening operation to generate a light field image under normal lighting, that is, the processed light field image;

[0029] Among them, the low-light enhancement module is composed of a LayerNorm unit, a processing convolution unit, a deprocessing convolution unit, a depth-separable convolution unit and several linear layers; the LayerNorm unit is used to perform standardization operations on image features; the processing convolution unit is used to increase the image color channels; the deprocessing convolution unit is used to reduce the number of image color channels, integrate and refine feature information, and generate the reconstructed image; the depth-separable convolution unit is used to perform layered processing with each color channel to isolate noise interference between different channels; the linear layer is used to generate the query, key, and value components of the self-attention mechanism.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] The method of the present invention dynamically selects useful features based on adaptive feature probability, which is in sharp contrast to the existing method of empirically combining spatial and angular features. The present invention improves the performance of dark light field image compression and reduces the consumption of image data storage resources. A low light enhancement module (LLE) is used to perform global color enhancement and local refinement to achieve the purpose of dark light image enhancement. The present invention significantly improves the efficiency and accuracy of dark light field image compression, and has important application value and broad promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0033] Figure 1 This is a flow chart of a dark light field image compression method based on probability representation according to an embodiment of the present invention;

[0034] Figure 2 Schematic diagram of the overall structure of a deep compression model based on probability representation according to an embodiment of the present invention;

[0035] Figure 3 Schematic diagram of the structure of a probability feature representation module according to an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the Angular Transformer unit structure of an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the structure of a non-overlapping sliding window attention unit according to an embodiment of the present invention;

[0038] Figure 6 FIG. 4 is a schematic diagram of the structure of a low-light enhancement module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] This embodiment proposes a dark light field image compression method based on probability representation, including:

[0042] Acquire a light field image to be processed, input the light field image to be processed into a deep compression model based on probability representation for processing, and output a processed light field image;

[0043] Among them, the deep compression model based on probability representation includes a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low-light enhancement module.

[0044] The method of this embodiment follows the learnable independent Bernoulli distribution binary mask to dynamically select useful features, and improves the performance of dark light field image compression through model parameter sharing and loss weight optimization; by introducing probabilistic representation, it effectively solves the shortcomings of the existing deep compression model for the local detail texture of dark light field images, and only considers eliminating the redundancy between features from the spatial dimension, while ignoring the feature redundancy between angles, channels, and polar planes, which limits the improvement of dark light field image compression performance. This embodiment shows that dark light field images can not only be analyzed visually, but also can dynamically select useful information by introducing probability distribution, thereby providing a novel and effective solution for dark light field image compression.

[0045] Furthermore, the light field image to be processed is input into a deep compression model based on probability representation (such as Figure 1-Figure 2 ) for processing, including:

[0046] Extracting features of the light field image to be processed by a probabilistic feature representation module, including spatial features, angle features and extreme plane features;

[0047] Initializing the extracted features based on the encoder, mapping the low-dimensional features to high-dimensional features, and performing compression encoding to generate a potential feature encoding matrix;

[0048] Using the quantization module to perform a quantization encoding operation on the potential feature encoding matrix to obtain a quantized potential feature representation;

[0049] The entropy evaluation module calculates the potential feature encoding matrix and estimates the total number of bits required for the quantized potential feature representation under the specific Gaussian distribution;

[0050] The decoder performs a decoding operation on the quantized latent feature representation to generate a reconstructed image, and the low-light enhancement module performs a brightening operation on the reconstructed image to output the processed light field image.

[0051] Furthermore, the features of the light field image to be processed are extracted, including:

[0052] The light field image to be processed is divided into a number of small images, and the several small images are input into the probabilistic feature representation module for feature extraction; wherein the probabilistic feature representation module includes a first feature embedding unit, a second feature embedding unit and a feature mixing convolution unit, the first feature embedding unit dynamically extracts the first feature information by following a learnable independent Bernoulli distribution binary mask, the second feature embedding unit dynamically extracts the second feature information by following a learnable independent Bernoulli distribution binary mask, and the feature mixing convolution unit is used to perform convolution processing on the second feature information to generate a processed feature matrix.

[0053] Specifically, the dark and high-bit light field image to be processed is input into the probabilistic feature representation module to extract useful spatial, angular, and epipolar plane (EPI) features respectively.

[0054] The light field image to be processed can be expressed as L(u, v, h, w, c), where (u, v) represents the angular dimension of the light field image (subaperture view), (h, w) is the spatial dimension, and c is the number of color channels (RGB channels). In order to enable the image to be processed by the network and reduce the amount of computation during model training and inference, the image data is first preprocessed to crop or adjust the light field image to be processed to the input size required by the network. The processed dark light field image x input dimension is (5, 5, 3, 64, 64).

[0055] The acquired processed dark light field image data is sent to the probability feature representation module, such as Figure 3As shown in the figure, the probability feature representation module (PFR) has a total of 8 feature embedding units. The previous feature embedding unit provides useful information to the subsequent feature embedding unit by following the learnable independent Bernoulli distribution binary mask dynamics. Each feature embedding unit is composed of Conv_spa, Conv_ang, Conv_epi_h, Conv_epi_w, Conv_vsi_v and Conv_vsi_h. Six carefully designed convolutions are used to extract angle, space, horizontal polar plane, vertical polar plane, horizontal virtual slit image, and vertical virtual slit image features respectively, and roughly remove unnecessary redundancy. Similarly, more useful features are selected by following the learnable independent Bernoulli distribution binary mask to further remove unnecessary redundancy, and the final 3×3 mixed convolution is used to mix the extracted features to enhance the compression capability of the model. Finally, the dimension of the dark light field image processed by the probability feature representation module (PFR) is (batch_size, 3, 64, 64), where batch_size = u*v.

[0056] Furthermore, the extracted features are initialized based on the encoder, the low-dimensional features are mapped to high-dimensional features, and compression encoding is performed to generate a potential feature encoding matrix;

[0057] The encoder includes an Angular Transformer unit, a non-overlapping sliding window attention unit, a downsampling convolution unit and a generating splitting normalization unit, wherein the Angular Transformer unit is used to transform the processed feature matrix into feature data through a rearrange function; the non-overlapping sliding window attention unit is used to allow information to be transferred between adjacent windows by introducing a displacement window mechanism, and to maintain the flow of global information within a local window; the downsampling convolution unit and the generating splitting normalization unit are used to perform compression encoding of dark light field image features until the potential feature encoding matrix is ​​generated.

[0058] Specifically, the encoder first feeds each dark light field image into the initialization convolution to map low-dimensional features to high-dimensional features, which can not only extract and retain the key features of the image, but also enhance the model's feature expression ability, improve the model's efficiency and accuracy, and enhance the model's generalization ability. The processed feature matrix dimension is (batch_size, 192, 64, 64).

[0059] The encoder consists of 2 Angular Transformer units, 2 non-overlapping sliding window attention units, 4 5×5 downsampling convolution units and 3 generation split normalization units. In this embodiment, except for the last downsampling convolution unit, each of the remaining downsampling convolution units is followed by a generation split normalization unit to maximize the compression bit rate for dark light images.

[0060] like Figure 4 As shown in the figure, similar to the traditional Transformer, each Angular Transformer unit contains 2 LayerNorms, 1 Multi-Head Self-Attention (MHSA), 1 Feed-Forward Network (FFN) and 2 residual connections. However, unlike the traditional Transformer, the Angular Transformer unit focuses on the relationship between pixels at the same position in multiple angles, so it needs to transform the feature matrix into feature data with a dimension of (64*64, 192, 5, 5) through the rearrange function, where (u, v) is (5, 5) representing the angular dimension of the light field image (sub-aperture view). After the Angular Transformer unit performs angle feature enhancement extraction, the feature dimension is (64*64, 192, 5, 5), which is transformed again through the rearrange function into feature data with a dimension of (batch_size, 192, 64, 64) and sent to the No Overlap Shift WindowAttention (NOSWA) unit. Generating an attention mask based on non-overlapping spatial adjacent elements can effectively extract spatial features with less computational cost, thereby improving compression performance. Therefore, this embodiment proposes the following Figure 5The non-overlapping sliding window attention unit (NOSWA) shown in the figure includes 1 sliding window attention, 2 residual blocks (ResBlock) and a Sigmoid function to allow information to be transferred between adjacent windows by introducing a displacement window mechanism, thereby maintaining the flow of global information within the local window. The feature dimension after spatial feature enhancement extraction by the non-overlapping sliding window attention unit (NOSWA) is (batch_size, 192, 64, 64). The features after angle and spatial feature extraction are sent to the first 5×5 downsampling convolution unit and the generation split normalization unit (GDN) for the first feature compression encoding of the dark light field image. The dimension of the feature matrix after processing is (batch_size, 192, 32, 32). The above is the whole process of the first feature compression encoding of the dark light field image. The following three compression encoding processes are similar to the first one. The dimensions of the feature matrix after each compression encoding process are (batch_size, 192, 16, 16), (batch_size, 192, 8, 8), and (batch_size, 192, 4, 4). Finally, the feature matrix dimension of the potential feature representation (y) after compression encoding by the encoder is (batch_size, 192, 4, 4), that is, the potential feature encoding matrix.

[0061] Furthermore, obtaining the potential feature representation after quantization includes: adding uniformly distributed noise in the quantization coding operation stage to make the quantization process differentiable, and then converting the floating point number into an integer by rounding to obtain the potential feature representation.

[0062] Specifically, the generated encoded latent feature representation (y) is sent to the quantization module for quantization encoding. Considering that the quantization process itself is not differentiable, this embodiment makes the quantization process differentiable by adding uniformly distributed noise, and then converts the floating point number into an integer by rounding to implement the quantization operation, thereby further improving the compression efficiency. Finally, the latent representation after quantization encoding is recorded as The dimension is (batch_size, 192, 4, 4), which is the quantized potential feature representation.

[0063] Furthermore, the potential feature encoding matrix is ​​calculated by the entropy evaluation module, including:

[0064] Inputting the potential feature coding matrix into the entropy evaluation module to perform bit rate estimation to obtain Gaussian model parameters;

[0065] Among them, the entropy evaluation module includes a super encoder, a quantization unit and a super decoder. The super encoder is composed of a 2D convolution unit, two downsampling convolution units and two LeakyReLU activation functions; the super decoder is composed of two upsampling convolution units, two LeakyReLU activation functions and a 2D convolution unit.

[0066] The super encoder is used to estimate the marginal distribution of the potential feature coding matrix, that is, the prior information, and compress the prior information; the quantization unit is used to reduce the accuracy of the prior information and significantly reduce the inference speed of the model; the super decoder is used to reconstruct the prior information, thereby eliminating the spatial redundancy in the prior information, and based on this, calculate the parameters of the Gaussian distribution of the potential features.

[0067] Specifically, the potential feature coding matrix is ​​fed into the entropy estimation model to perform bit rate estimation. In this embodiment, considering that the goal of the image compression method is to obtain a high-quality reconstructed image under a given bit rate target, and the entropy model is used to estimate the bit rate, it is crucial to establish an accurate entropy model.

[0068] The entropy estimation model proposed in this embodiment is similar to the variational autoencoder (VAE). It uses a hyperencoder, a quantization unit, and a hyperdecoder to estimate the corresponding parameter Φ in the entropy model. The hyperencoder (Hyperencoder) includes a 3×3 2D convolution unit, 2 downsampling 5×5 convolution units, and 2 LeakyReLU activation functions; the hyperdecoder (Hyperdecoder) includes 2 upsampling 5×5 convolution units, 2 LeakyReLU activation functions, and a 3×3 2D convolution unit. A hyper-prior compression scheme is adopted, and the potential feature representation is Modeled as a Gaussian distribution, the parameters μ and σ of the Gaussian model are estimated through the super encoder and super decoder, and a specific Gaussian distribution is constructed to approximate the potential feature representation after quantization. The total number of bits required under a specific Gaussian distribution.

[0069] Furthermore, the total number of bits required to represent the quantized latent features under a specific Gaussian distribution is estimated, including:

[0070] The quantized latent feature representation is compressed using an arithmetic encoder and modeled as a Gaussian distribution by adopting a learning-based super-prior compression scheme. A specific Gaussian distribution is constructed for each data based on the Gaussian model parameters μ and σ estimated by the super-encoder and super-decoder, and the quantized latent feature representation is approximated based on this. The total number of bits required under a particular distribution.

[0071] Further, generating a reconstructed image includes:

[0072] The quantized latent feature representation is sent to the arithmetic decoder for decompression, and then sent to the decoder for latent feature decoding; wherein the decoder is composed of an Angular Transformer unit, a non-overlapping sliding window attention unit, an upsampling convolution unit, and an anti-generation split normalization unit. The Angular Transformer unit is used to capture the panoramic angle dependency that is difficult to capture with traditional convolution; the non-overlapping sliding window attention unit is used to introduce residual connections to improve the quality of intermediate features; the upsampling convolution unit is used to increase the image resolution and gradually reconstruct the image; the anti-generation split normalization unit is used for inverse normalization processing to restore the data distribution characteristics.

[0073] Specifically, in this embodiment, the decoder (Decoder) corresponds to the encoder (Encoder) structure, and is composed of 2 Angular Transformer units, 2 non-overlapping sliding window attention units, 4 5×5 upsampling convolution units and 3 inverse generation split normalization units (IGDN) to maximize the performance of dark light image reconstruction. The dimensions of the feature matrix after each decoding process are (batch_size, 192, 8, 8), (batch_size, 192, 16, 16), (batch_size, 192, 32, 32) and (batch_size, 192, 64, 64). Therefore, the dimension of the feature matrix reconstructed by the decoding module is (batch_size, 192, 64, 64).

[0074] Furthermore, the processed light field image is output, including:

[0075] The high-dimensional feature matrix output by the decoder is sent to the low-light enhancement module for brightening operation to generate a light field image under normal lighting, that is, the processed light field image;

[0076] The low-light enhancement module is composed of a LayerNorm unit, a processing convolution unit, a deprocessing convolution unit, a depth-separable convolution unit, and several linear layers. The LayerNorm unit is used to standardize image features; the processing convolution unit is used to increase the image color channels, thereby increasing the expressiveness of color features; the deprocessing convolution unit is used to reduce the number of image color channels, further integrate and refine feature information, and generate a reconstructed image; the depth-separable convolution unit is used to perform layered processing with each color channel to isolate noise interference between different channels, and provide purer processing features for the self-attention mechanism composed of several subsequent linear layers; the linear layer is used to generate the three core components of the self-attention mechanism: query, key, and value.

[0077] Specifically, the high-dimensional feature matrix after decoding by the decoder is sent to the low-light enhancement module (LLE) to achieve brightening of the reconstructed image. Due to the globally shared settings, such as ambient lighting and color space specifications, the color of the image is mainly enhanced or converted to another color space through channel matrix transformation. Following this principle, Figure 6 As shown, this embodiment introduces a low-light enhancement module (LLE) to perform global color enhancement and local refinement. The design of this module benefits from the latest progress in self-attention. The Query, Key, and Value components in the self-attention are extracted through three linear layers (Liner), where the dimensions of Query, Key, and Value are all (HW, C). The Query is transposed and matrix multiplied with the Key to obtain a feature matrix of (HW, HW). The (HW, HW) feature matrix is ​​matrix multiplied with the Value to obtain a feature matrix of dimension (HW, C), thereby realizing the illumination enhancement operation. Finally, the image is enhanced after the low-light enhancement module (LLE). The dimension is (batch_size, 3, 64, 64), which is the same as the original image size.

[0078] The difference and potential features between the reconstructed image and the ground truth are represented The bit rates of the model jointly constitute the loss function, which is specifically described as:

[0079]

[0080] Where D and R represent distortion and bit rate respectively, and λ is a trade-off parameter. represents the image after reconstruction and enhancement, Y is the ground truth of the image under normal lighting, Latent feature representation estimated by the entropy estimation module bit rate.

[0081] The loss of the model is continuously reduced through training, thereby optimizing the performance of the entire model.

[0082] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A dark light field image compression method based on probability representation, characterized in that: include: Acquire a light field image to be processed, input the light field image to be processed into a deep compression model based on probability representation for processing, and output a processed light field image; Among them, the deep compression model based on probability representation includes a probability feature representation module, an encoder, a quantization module, an entropy evaluation module, a decoder and a low-light enhancement module which are connected in sequence.

2. The method according to claim 1, characterized in that Inputting the light field image to be processed into a deep compression model based on probability representation for processing, including: Extracting features of the light field image to be processed by the probabilistic feature representation module, including spatial features, angle features and extreme plane features; Initializing the extracted features based on the encoder, mapping the low-dimensional features to high-dimensional features, and performing compression encoding to generate a potential feature encoding matrix; Using the quantization module to perform a quantization encoding operation on the potential feature encoding matrix to obtain a quantized potential feature representation; The entropy evaluation module calculates the potential feature encoding matrix and estimates the total number of bits required for the quantized potential feature representation under the specific Gaussian distribution; The decoder performs a decoding operation on the quantized latent feature representation to generate a reconstructed image, and the low-light enhancement module performs a brightening operation on the reconstructed image to output the processed light field image.

3. The method according to claim 2, characterized in that Extracting features of the light field image to be processed includes: The light field image to be processed is divided into a number of small images, and the several small images are input into the probabilistic feature representation module for feature extraction; wherein the probabilistic feature representation module includes a first feature embedding unit, a second feature embedding unit and a feature mixing convolution unit, the first feature embedding unit dynamically extracts first feature information by following a learnable independent Bernoulli distribution binary mask, the second feature embedding unit dynamically extracts second feature information by following a learnable independent Bernoulli distribution binary mask, and the feature mixing convolution unit is used to perform convolution processing on the second feature information to generate a processed feature matrix.

4. The method according to claim 3, characterized in that The encoder includes an Angular Transformer unit, a non-overlapping sliding window attention unit, a downsampling convolution unit and a generating split normalization unit, wherein the Angular Transformer unit is used to transform the processed feature matrix into feature data through a rearrange function; the non-overlapping sliding window attention unit is used to allow information to be transferred between adjacent windows by introducing a displacement window mechanism, and to maintain the flow of global information within a local window; the downsampling convolution unit and the generating split normalization unit are used to perform compression encoding of dark light field image features until the potential feature encoding matrix is ​​generated.

5. The method according to claim 2, characterized in that: Obtaining the quantized potential feature representation includes: In the quantization coding operation stage, uniformly distributed noise is added to make the quantization process differentiable, and then the floating point number is converted into an integer by rounding to obtain the potential feature representation.

6. The method according to claim 1, characterized in that The potential feature encoding matrix is ​​calculated by the entropy evaluation module, including: Inputting the potential feature coding matrix into the entropy evaluation module to perform bit rate estimation to obtain Gaussian model parameters; The entropy evaluation module includes a super encoder, a quantization unit and a super decoder, wherein the super encoder is composed of a 2D convolution unit, two downsampling convolution units and two LeakyReLU activation functions; the super decoder is composed of two upsampling convolution units, two LeakyReLU activation functions and a 2D convolution unit; The super encoder is used to estimate the marginal distribution of the potential feature coding matrix, that is, the prior information, and compress the prior information; the quantization unit is used to reduce the accuracy of the prior information; and the super decoder is used to reconstruct the prior information.

7. The method according to claim 6, characterized in that The total number of bits required to represent the quantized latent feature under a specific Gaussian distribution is estimated, comprising: The quantized latent feature representation is compressed using an arithmetic encoder, and the compressed latent feature representation is modeled as a Gaussian distribution by adopting a learning-based super-prior compression method. A specific Gaussian distribution is constructed for each data based on the Gaussian model parameters calculated by the super-encoder and the super-decoder, and the total number of bits required for the quantized latent feature representation under the specific distribution is estimated.

8. The method according to claim 2, characterized in that: Generate a reconstructed image, including: The quantized latent feature representation is sent to an arithmetic decoder for decompression, and then sent to the decoder for latent feature decoding; wherein the decoder is composed of an Angular Transformer unit, a non-overlapping sliding window attention unit, an upsampling convolution unit and an anti-generation split normalization unit; the Angular Transformer unit is used to capture the panoramic angle dependency that is difficult to capture by traditional convolution; the non-overlapping sliding window attention unit is used to introduce residual connections to improve the quality of intermediate features; the upsampling convolution unit is used to increase the image resolution and gradually reconstruct the image; the anti-generation split normalization unit is used for inverse normalization processing to restore data distribution characteristics.

9. The method according to claim 8, characterized in that Outputting the processed light field image includes: The high-dimensional feature matrix output by the decoder is sent to the low-light enhancement module for brightening operation to generate a light field image under normal lighting, that is, the processed light field image; Among them, the low-light enhancement module is composed of a LayerNorm unit, a processing convolution unit, a deprocessing convolution unit, a depth-separable convolution unit and several linear layers; the LayerNorm unit is used to perform standardization operations on image features; the processing convolution unit is used to increase the image color channels; the deprocessing convolution unit is used to reduce the number of image color channels, integrate and refine feature information, and generate the reconstructed image; the depth-separable convolution unit is used to perform layered processing with each color channel to isolate noise interference between different channels; the linear layer is used to generate the query, key, and value components of the self-attention mechanism.

Images