A light field image enhancement method for remote sensing images

Abstract

The present invention relates to the technical field of light field image enhancement, and in particular to a light field image enhancement method for remote sensing images. The present invention discloses a light field image enhancement method for remote sensing images, comprising: acquiring a light field image, and acquiring atmospheric illumination data and priori data based on the light field image; using a transmittance estimation method for the priori data to acquire the transmittance of the light field image; and using a data enhancement algorithm based on the light field image, the atmospheric illumination data, and the transmittance of the light field image to acquire an enhanced light field image. The present invention significantly improves the quality and detail expression of light field images for remote sensing images by combining multi-dimensional feature extraction, priori information utilization, and an efficient enhancement algorithm, thereby providing strong support for the digital recording and dissemination of remote sensing images and having important practical application value.

Description

Technical Field

[0001] The present invention relates to the technical field of light field image enhancement, and in particular to a light field image enhancement method for remote sensing images. Background Art

[0002] In the field of remote sensing imaging, the capture and presentation of four-dimensional light field imaging technology is extremely valuable. However, traditional imaging methods have significant shortcomings in meeting the multi-view, high-quality imaging requirements of remote sensing imaging. Four-dimensional light field imaging technology, with its multi-angle scene capture capabilities and inherent geometric information advantages, provides a new solution for high-quality remote sensing imaging. Despite this, remote sensing imaging still faces many challenges, such as the complexity of ambient lighting, the rapid movement of targets, and the diverse surface features. These issues lead to widespread quality issues in captured images, such as insufficient contrast, blurred textures, and color deviations. Therefore, developing light field image enhancement technology specifically for remote sensing imagery has important research significance and application value for promoting the development of remote sensing imaging technology. Traditional two-dimensional RGB imaging technology exhibits significant limitations in capturing remote sensing images. It can only capture images from a single perspective, lacking depth and multi-angle information, and cannot fully display the three-dimensional structure and details of the surface. Furthermore, two-dimensional imaging technology is prone to motion blur and exposure issues when dealing with fast-moving targets and changing ambient lighting, further affecting image quality. In contrast, four-dimensional light field imaging technology, by simultaneously recording the spatial and angular information of light, can reconstruct scenes from multiple perspectives, providing richer information for analysis. This technology not only enables three-dimensional reconstruction but also allows for post-processing operations such as perspective switching and focus adjustment, significantly enhancing the depth and breadth of data analysis. Despite its numerous advantages, light field imaging technology still faces a number of challenges in practical applications. First, the massive data volume of light field images places higher demands on storage and transmission. Second, the dynamic changes and high-contrast ambient light in remote sensing imaging complicate the acquisition and processing of light field images. For example, rapid movement of a target object can cause image blur, while strong contrast in ambient light can result in over- or underexposure. Furthermore, existing light field image processing algorithms often fail to fully exploit the correlation between different perspectives when processing complex scenes, resulting in suboptimal enhancement results. Therefore, developing an effective enhancement method for light field images for remote sensing is of great practical significance for improving data analysis and applications in remote sensing imaging. Summary of the Invention

[0003] To solve the above technical problems, the present invention proposes a light field image enhancement method for remote sensing images to improve the quality and detail expression of light field images under remote sensing imaging.

[0004] To achieve the above objectives, the present invention provides a light field image enhancement method for remote sensing images, comprising:

[0005] Acquire light field images, and obtain atmospheric illumination data and prior data based on the light field images;

[0006] For the prior data, a transmittance estimation method is used to obtain the transmittance of the light field image;

[0007] Based on the light field image, atmospheric illumination data and light field image transmittance, a data enhancement algorithm is used to obtain an enhanced light field image.

[0008] Preferably, obtaining atmospheric illumination data based on the light field image includes:

[0009] Performing convolution processing on the light field image to extract initial features of the light field image;

[0010] Convolution processing is performed on the initial features to obtain atmospheric illumination data.

[0011] Preferably, obtaining prior data based on the light field image includes:

[0012] Converting the light field image into a two-dimensional feature map;

[0013] Dark channel prior data, bright channel prior data, and average channel prior data of the light field image are calculated based on the two-dimensional feature map.

[0014] Preferably, performing convolution processing on the light field image to extract initial features of the light field image includes:

[0015] The light field image is sequentially subjected to four layers of convolution processing; wherein the second convolution, the third convolution, and the fourth convolution are all followed by a LeakyReLU activation function;

[0016] The output of the first convolution process is added to the output of the fourth convolution process to obtain the initial feature.

[0017] Preferably, performing convolution processing on the initial features to obtain atmospheric illumination data includes:

[0018] The initial features are subjected to four alternating feature extractions in sequence, and the results of the first alternating feature extraction are added to the results of the fourth alternating feature extraction to obtain atmospheric illumination data.

[0019] Preferably, the method of alternating feature extraction comprises:

[0020] Performing EPI feature extraction on the initial features to obtain EPI feature data;

[0021] Perform space-angle feature extraction based on EPI feature data to obtain alternating feature extraction results.

[0022] Preferably, performing EPI feature extraction on the initial features includes:

[0023] Performing disparity information highlighting processing on the initial features, and then performing convolution processing after global feature extraction;

[0024] The convolution-processed data is added to the parallax-processed data to obtain the first EPI feature;

[0025] The first EPI feature is processed to highlight the disparity information, and then subjected to convolution processing after global feature extraction;

[0026] The data after the second convolution processing is added to the data after the first disparity information highlighting processing to obtain EPI feature data.

[0027] Preferably, for the prior data, a transmittance estimation method is used to obtain the transmittance of the light field image, including:

[0028] The prior data is subjected to 2D convolution processing to obtain preliminary features;

[0029] Perform residual convolution processing on the preliminary features in sequence to obtain the first convolution output data;

[0030] The convolution output data is then subjected to 2D convolution processing to obtain second convolution output data;

[0031] The priori data is added to the second convolution output data, and a Sigmoid activation function is performed to obtain the light field image transmittance.

[0032] Preferably, the data enhancement algorithm includes:

[0033]

[0034] in, is the light field image, is the atmospheric illumination data, is the light field image transmittance, is the enhanced light field image.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] The present invention proposes a light field image enhancement method for remote sensing images, which obtains a light field image, and obtains atmospheric illumination data and prior data of the light field image based on the light field image; for the prior data, a transmittance estimation method is used to obtain the transmittance of the light field image; based on the light field image, the atmospheric illumination data and the transmittance of the light field image, a data enhancement algorithm is used to obtain an enhanced light field image; this method aims to solve the problems of reduced contrast, blurred texture, color distortion and other problems of light field images in remote sensing imaging caused by factors such as complex ambient light, rapid movement of target objects and diverse surface features, and realizes efficient and high-quality image enhancement through multi-dimensional feature extraction and prior information utilization; by combining multi-dimensional feature extraction, prior information utilization and efficient enhancement algorithm, the quality and detail expression of light field images for remote sensing images are significantly improved, which provides strong support for the digital recording and dissemination of remote sensing images and has important practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0038] Figure 1 This is a flowchart of a light field image enhancement method for remote sensing images according to an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the overall structure of a light field image enhancement method for remote sensing images according to an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of the initial feature extraction structure of an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of the structure of a transmittance estimation module according to an embodiment of the present invention;

[0042] Figure 5 Schematic diagram of the residual convolution block structure of an embodiment of the present invention;

[0043] Figure 6 Schematic diagram of the structure of the global-local feature extraction module according to an embodiment of the present invention;

[0044] Figure 7 Schematic diagram of the structure of the EPI feature extraction module according to an embodiment of the present invention;

[0045] Figure 8 Schematic diagram of the structure of the space-angle feature extraction module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] like Figure 1 and Figure 2 The present invention proposes a light field image enhancement method for remote sensing images, comprising:

[0049] Acquire light field images, and obtain atmospheric illumination data and prior data based on the light field images;

[0050] For the prior data, the transmittance estimation method is used to obtain the transmittance of the light field image;

[0051] Based on the light field image, atmospheric illumination data and light field image transmittance, a data enhancement algorithm is used to obtain the enhanced light field image.

[0052] Further, if Figure 3 The atmospheric illumination data is obtained based on the light field image, including:

[0053] Perform convolution processing on the light field image to extract the initial features of the light field image;

[0054] Perform convolution processing on the initial features to obtain atmospheric lighting data.

[0055] Specifically, in this embodiment, the extracted light field image is , performing multi-layer convolution operations on the light field image to extract initial features and enhancing the feature representation through residual connections. The input light field image has dimensions [b, u, v, c, h, w], where b is the batch size, u and v are the angular dimensions (typically 5), c is the RGB channel (c=3), and h and w are the spatial dimensions. The extracted light field image is the unenhanced light field subaperture array image.

[0056] Furthermore, prior data is obtained based on the light field image, including:

[0057] Convert light field images into two-dimensional feature maps;

[0058] The dark channel prior data, bright channel prior data and average channel prior data of the light field image are calculated based on the two-dimensional feature map.

[0059] Specifically, in this embodiment, the prior extraction includes extracting the dark channel prior, bright channel prior, and average channel prior of the light field image. The specific steps include:

[0060] Convert the high-dimensional light field image into a two-dimensional feature map. The input light field image dimension is [b, u, v, c, h, w], and the rearranged two-dimensional feature map dimension is [b, c, uh, vw].

[0061] The dark channel prior, bright channel prior and average channel prior are calculated respectively, and the dimensions of the output dark channel prior, bright channel prior and average channel prior are all [b, 1, uh, vw].

[0062] Furthermore, convolution processing is performed on the light field image to extract the initial features of the light field image, including:

[0063] The light field image is processed with four layers of convolution in sequence; the second, third, and fourth convolutions are all followed by the LeakyReLU activation function;

[0064] The output of the first convolution process is added to the output of the fourth convolution process to obtain the initial features.

[0065] Specifically, in this embodiment, a multi-layer convolution operation is performed on the input light field image to extract the initial features of the light field image, which specifically includes:

[0066] Use 4 convolutional layers, each followed by a LeakyReLU activation function;

[0067] The first convolution layer uses a convolution layer with a kernel size of 3×3 and 64 output channels.

[0068] The second convolution layer uses a convolutional layer with a kernel size of 3×3 and 64 output channels, followed by a LeakyReLU activation function.

[0069] The third convolution layer uses a convolutional layer with a kernel size of 3×3 and 64 output channels, followed by a LeakyReLU activation function.

[0070] The fourth convolution layer uses a convolutional layer with a kernel size of 3×3 and 64 output channels, followed by a LeakyReLU activation function.

[0071] Residual connections are used to enhance feature representation, adding the output of the first convolution layer to the output of the fourth convolution layer. The dimensions of the initial output features are [b, u, v, 64, h, w].

[0072] Further, if Figure 6As shown, the initial features are convolved to obtain atmospheric illumination data, including:

[0073] The initial features are subjected to four alternating feature extractions in sequence, and the results of the first alternating feature extraction are added to the results of the fourth alternating feature extraction to obtain atmospheric illumination data.

[0074] Furthermore, the method of alternating feature extraction includes:

[0075] Perform EPI feature extraction on the initial features to obtain EPI feature data;

[0076] Perform space-angle feature extraction based on EPI feature data to obtain alternating feature extraction results.

[0077] Specifically, in this embodiment, the four-pass alternating feature extraction uses a global-local feature extraction network for feature extraction, and the initial features are fed into the global-local feature extraction network. Specifically, four alternating feature extraction modules are used to extract features of light field images, with an input dimension of [b, u, v, 64, h, w] and an output atmospheric illumination dimension of [b, u, v, 64, h, w].

[0078] Further, if Figure 7 As shown, EPI feature extraction is performed on the initial features, including:

[0079] The initial features are processed by highlighting the disparity information, and then convolution is performed after global feature extraction;

[0080] The convolution-processed data is added to the parallax-processed data to obtain the first EPI feature;

[0081] The first EPI feature is processed to highlight the disparity information, and then subjected to convolution processing after global feature extraction;

[0082] The data after the second convolution processing is added to the data after the first disparity information highlighting processing to obtain EPI feature data.

[0083] Specifically, in this embodiment, EPI feature extraction utilizes the EPI feature extraction module:

[0084] Rearrange the light field image into a vertical Epipolar Plane Image (EPI-V) to highlight the parallax information;

[0085] Use the State Space Model to extract global features;

[0086] Use a convolutional layer with a kernel size of 3×3 and 64 output channels;

[0087] Use SiLU activation function to enhance feature expression ability;

[0088] Rearrange the light field image into a horizontal Epipolar Plane Image (EPI-H) to highlight the parallax information;

[0089] Use the State Space Model to extract global features;

[0090] Use a convolutional layer with a kernel size of 3×3 and 64 output channels;

[0091] Use SiLU activation function to enhance feature expression ability.

[0092] Specifically, in this embodiment, Figure 8 As shown, after the global features are extracted by the state space model, convolution processing is performed using the space-angle feature extraction module:

[0093] Use the State Space Model to extract global features;

[0094] Use a convolutional layer with a kernel size of 3×3 and 64 output channels;

[0095] Use SiLU activation function;

[0096] Rearrange the light field image into an angle view to highlight the perspective information;

[0097] Use the State Space Model to extract global features;

[0098] Use a convolutional layer with a kernel size of 3×3 and 64 output channels;

[0099] Use SiLU activation function;

[0100] The input features are added to the processed features through residual connections to enhance the feature expression capability. The dimension of the output global-local features is [b, u, v, 64, h, w].

[0101] Further, if Figure 4 and Figure 5 As shown, for the prior data, the transmittance estimation method is used to obtain the transmittance of the light field image, including:

[0102] The prior data is processed by 2D convolution to obtain preliminary features;

[0103] Perform residual convolution processing on the preliminary features in sequence to obtain the first convolution output data;

[0104] The convolution output data is then processed by 2D convolution to obtain the second convolution output data;

[0105] The prior data is added to the second convolution output data, and the Sigmoid activation function is performed to obtain the light field image transmittance.

[0106] Specifically, the dimension of the input light field image is [b, u, v, c, h, w]. The high-dimensional light field image is converted into a two-dimensional feature map with the dimension [b, c, uh, vw].

[0107] Calculate the dark channel prior, bright channel prior and average channel prior respectively:

[0108] Dark channel prior: calculate the minimum value Max of each pixel in all color channels;

[0109] Bright channel prior: calculate the maximum value Min of each pixel in all color channels;

[0110] Average channel prior: Calculate the average value of each pixel in all color channels;

[0111] The output dark channel prior, bright channel prior, and average channel prior dimensions are all [b, 1, uh, vw].

[0112] Specifically, in this embodiment, the dark channel prior, the bright channel prior, and the average channel prior are fed into the transmittance estimation module, the transmittance of the light field image; the dimension of the input prior feature is [b, 3, uh, vw], and the transmittance estimation module includes:

[0113] 2 convolutional layers, each followed by a ReLU activation function;

[0114] 3 residual convolution blocks: Each residual convolution block uses 2 convolution layers, and each convolution layer is followed by a ReLU activation function;

[0115] Sigmoid activation function: The output of the convolution block is passed through the Sigmoid activation function to obtain the transmittance;

[0116] Residual connection: adds the input features to the output of the convolutional block to enhance feature transfer;

[0117] The dimension of the output transmittance is [b, u, v, 3, h, w];

[0118] Furthermore, the data enhancement algorithm includes:

[0119]

[0120] in, is the light field image, is the atmospheric illumination data, is the light field image transmittance, is the enhanced light field image.

[0121] Specifically, in this embodiment, Input light field image, dimension is [b, u, v, c, h, w]; Estimated atmospheric illumination, in the form of [b, u, v, c, h, w]; Transmittance, of dimension [b, u, v, c, h, w].

[0122] Specifically, in this embodiment, the light field image enhancement method for remote sensing images further includes adopting a label-supervised training strategy for training, wherein the loss function during the training process is:

[0123]

[0124] Where, represents the loss value, represents the true value label, represents the perceptual loss calculated from high-level features of the VGG network pre-trained on ImageNet, represents the structural similarity loss, 、 and is the weight coefficient.

[0125] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A light field image enhancement method for remote sensing images, characterized in that: include: Acquire light field images, and obtain atmospheric illumination data and prior data based on the light field images; For the prior data, a transmittance estimation method is used to obtain the transmittance of the light field image; Based on the light field image, the atmospheric illumination data and the light field image transmittance, a data enhancement algorithm is used to obtain an enhanced light field image; Acquire atmospheric illumination data based on light field images, including: Performing convolution processing on the light field image to extract initial features of the light field image; Performing convolution processing on the initial features to obtain atmospheric illumination data; Performing convolution processing on the light field image to extract initial features of the light field image includes: The light field image is sequentially subjected to four layers of convolution processing; wherein the second convolution, the third convolution, and the fourth convolution are all followed by a LeakyReLU activation function; Adding the output of the first convolution process and the output of the fourth convolution process to obtain the initial feature; The initial features are convolved to obtain atmospheric illumination data, including: Perform four alternating feature extractions on the initial features, add the first alternating feature extraction result and the fourth alternating feature extraction result to obtain atmospheric illumination data; The method for alternate feature extraction comprises: Performing EPI feature extraction on the initial features to obtain EPI feature data; Perform space-angle feature extraction based on EPI feature data to obtain alternating feature extraction results; Performing EPI feature extraction on the initial features, including: Performing disparity information highlighting processing on the initial features, and then performing convolution processing after global feature extraction; Adding the convolution-processed data to the parallax information-highlighted data to obtain a first EPI feature; The first EPI feature is processed to highlight the disparity information, and then subjected to convolution processing after global feature extraction; The data after the second convolution processing is added to the data after the first disparity information highlighting processing to obtain EPI feature data.

2. The light field image enhancement method for remote sensing images according to claim 1, characterized in that: Acquire prior data based on light field images, including: Converting the light field image into a two-dimensional feature map; Dark channel prior data, bright channel prior data, and average channel prior data of the light field image are calculated based on the two-dimensional feature map.

3. The light field image enhancement method for remote sensing images according to claim 1, characterized in that: For the prior data, a transmittance estimation method is used to obtain the transmittance of the light field image, including: The prior data is subjected to 2D convolution processing to obtain preliminary features; Perform residual convolution processing on the preliminary features in sequence to obtain the first convolution output data; The convolution output data is then subjected to 2D convolution processing to obtain second convolution output data; The priori data is added to the second convolution output data, and a Sigmoid activation function is performed to obtain the light field image transmittance.

4. The light field image enhancement method for remote sensing images according to claim 1, characterized in that: The data enhancement algorithm is: in, is the light field image, is the atmospheric illumination data, is the light field image transmittance, is the enhanced light field image.

Images