Light field image enhancement method for remote sensing image

Abstract

The invention relates to the technical field of light field image enhancement, in particular to a light field image enhancement method for remote sensing images. The invention discloses a light field image enhancement method for a remote sensing image, and the method comprises the steps: obtaining a light field image, and obtaining atmosphere illumination data and priori data based on the light field image; for the prior data, obtaining the light field image transmissivity by adopting a transmissivity estimation method; based on the light field image, the atmosphere illumination data and the light field image transmissivity, acquiring an enhanced light field image by adopting a data enhancement algorithm; according to the method, by combining multi-dimensional feature extraction, prior information utilization and an efficient enhancement algorithm, the quality and detail representation of the light field image facing the remote sensing image are remarkably improved, powerful support is provided for digital recording and propagation of the remote sensing image, and the method has important practical application value.

Description

Technical Field

[0001] The present invention relates to the technical field of light field image enhancement, and particularly to a light field image enhancement method for remote sensing images. Background Art

[0002] In the field of remote sensing imaging, the capture and presentation of four-dimensional light field imaging technology have extremely important value. However, traditional imaging means have obvious deficiencies in meeting the requirements of multi-view and high-quality remote sensing imaging. The four-dimensional light field imaging technology, with its multi-angle scene capture ability and inherent geometric information advantages, provides a new solution for high-quality remote sensing imaging. Nevertheless, remote sensing imaging still faces many challenges, such as the complexity of environmental light, the rapid movement of target objects, and diverse surface features. These problems result in common quality problems in the captured images, such as insufficient contrast, blurred texture, and color deviation. Therefore, developing a light field image enhancement technology specifically for remote sensing images has important research significance and application value for promoting the development of remote sensing imaging technology. Traditional two-dimensional RGB imaging technology shows obvious limitations when capturing remote sensing images. It can only obtain images from a single perspective, lacking depth and multi-angle information, and it is difficult to fully display the three-dimensional structure and details of the ground surface. In addition, two-dimensional imaging technology is prone to motion blur and exposure problems when dealing with fast-moving target objects and changing environmental light, further affecting the imaging quality. In contrast, the four-dimensional light field imaging technology can reconstruct scenes from multiple perspectives by simultaneously recording the spatial and angular information of light, bringing richer information for analysis. This technology can not only achieve three-dimensional reconstruction but also perform operations such as perspective switching and focus adjustment on the image in the later stage, significantly improving the depth and breadth of data analysis. Although the light field imaging technology has many advantages, it still faces a series of challenges in practical applications. First, the huge data volume of light field images poses higher requirements for storage and transmission. Second, the dynamic changes and high-contrast environmental light in remote sensing imaging make the acquisition and processing of light field images more difficult. For example, the rapid movement of target objects may cause image blur, while the strong contrast of environmental light may result in overexposure or underexposure. In addition, existing light field image processing algorithms often cannot fully exploit the correlation information between different perspectives when dealing with complex scenes, resulting in unsatisfactory enhancement effects. Therefore, studying an enhancement method that can effectively process light field images for remote sensing images has important practical significance for improving the data analysis and application of remote sensing imaging. Summary of the Invention

[0003] To solve the above technical problems, the present invention proposes a light field image enhancement method for remote sensing images to improve the quality and detail performance of light field images in remote sensing imaging.

[0004] To achieve the above object, the present invention provides a light field image enhancement method for remote sensing images, including:

[0005] Obtain a light field image, and obtain atmospheric illumination data and prior data based on the light field image;

[0006] For the prior data, use a transmittance estimation method to obtain the transmittance of the light field image;

[0007] Based on the light field image, atmospheric illumination data, and the transmittance of the light field image, use a data enhancement algorithm to obtain the enhanced light field image.

[0008] Preferably, obtaining atmospheric illumination data based on the light field image includes:

[0009] Perform convolution processing on the light field image to extract the initial features of the light field image;

[0010] Perform convolution processing on the initial features to obtain the atmospheric illumination data.

[0011] Preferably, obtaining prior data based on the light field image includes:

[0012] Convert the light field image into a two-dimensional feature map;

[0013] Calculate the dark channel prior data, bright channel prior data, and average channel prior data of the light field image based on the two-dimensional feature map.

[0014] Preferably, performing convolution processing on the light field image to extract the initial features of the light field image includes:

[0015] Perform four-layer convolution processing on the light field image in sequence; among them, LeakyReLU activation functions are connected after the second convolution, the third convolution, and the fourth convolution;

[0016] Add the output of the first convolution processing to the output of the fourth convolution processing to obtain the initial features.

[0017] Preferably, performing convolution processing on the initial features to obtain the atmospheric illumination data includes:

[0018] Perform four alternating feature extractions on the initial features in sequence, and add the result of the first alternating feature extraction to the result of the fourth alternating feature extraction to obtain the atmospheric illumination data.

[0019] Preferably, the method for alternating feature extraction includes:

[0020] Perform EPI feature extraction on the initial features to obtain EPI feature data;

[0021] Spatial-angle feature extraction is performed based on the EPI feature data to obtain the alternating feature extraction result.

[0022] Preferably, the EPI feature extraction of the initial feature includes:

[0023] Perform disparity information highlighting processing on the initial feature, and then perform convolution processing after global feature extraction;

[0024] Add the data after convolution processing to the data after disparity information highlighting processing to obtain the first EPI feature;

[0025] Perform disparity information highlighting processing on the first EPI feature, and then perform convolution processing after global feature extraction;

[0026] Add the data after the second convolution processing to the data after the first disparity information highlighting processing to obtain the EPI feature data.

[0027] Preferably, for the prior data, a transmittance estimation method is used to obtain the light field image transmittance, including:

[0028] Perform 2D convolution processing on the prior data to obtain preliminary features;

[0029] Perform residual convolution processing on the preliminary features in sequence to obtain the first convolution output data;

[0030] The convolution output data is further processed by 2D convolution to obtain the second convolution output data;

[0031] Add the prior data to the second convolution output data and perform Sigmoid activation function processing to obtain the light field image transmittance.

[0032] Preferably, the data enhancement algorithm includes:

[0033]

[0034] Among them, is the light field image, is the atmospheric illumination data, is the light field image transmittance, is the enhanced light field image.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] The present invention proposes a light field image enhancement method for remote sensing images, which obtains a light field image and acquires the atmospheric illumination data and prior data of the light field image based on the light field image; for the prior data, a transmittance estimation method is used to obtain the transmittance of the light field image; based on the light field image, the atmospheric illumination data and the transmittance of the light field image, a data enhancement algorithm is used to obtain the enhanced light field image; this method aims at the problems of reduced contrast, blurred texture, color distortion, etc. caused by complex environmental light, rapid movement of target objects, diverse surface features, etc. in remote sensing imaging, and realizes efficient and high-quality image enhancement through multi-dimensional feature extraction and utilization of prior information; by combining multi-dimensional feature extraction, utilization of prior information and an efficient enhancement algorithm, the quality and detail performance of the light field image for remote sensing images are significantly improved, providing strong support for the digital recording and dissemination of remote sensing images, and having important practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0038] Figure 1 It is a schematic flowchart of a light field image enhancement method for remote sensing images according to an embodiment of the present invention;

[0039] Figure 2 It is a schematic overall structure diagram of a light field image enhancement method for remote sensing images according to an embodiment of the present invention;

[0040] Figure 3 It is a schematic initial feature extraction structure diagram according to an embodiment of the present invention;

[0041] Figure 4 It is a schematic transmittance estimation module structure diagram according to an embodiment of the present invention;

[0042] Figure 5 It is a schematic residual convolution block structure diagram according to an embodiment of the present invention;

[0043] Figure 6 It is a schematic global-local feature extraction module structure diagram according to an embodiment of the present invention;

[0044] Figure 7 It is a schematic EPI feature extraction module structure diagram according to an embodiment of the present invention;

[0045] Figure 8 It is a schematic spatial-angle feature extraction module structure diagram according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0048] As Figure 1 and Figure 2 shown, the present invention proposes a light field image enhancement method for remote sensing images, including:

[0049] Obtain a light field image, and obtain atmospheric illumination data and prior data based on the light field image;

[0050] For the prior data, use a transmittance estimation method to obtain the transmittance of the light field image;

[0051] Based on the light field image, atmospheric illumination data, and the transmittance of the light field image, use a data enhancement algorithm to obtain the enhanced light field image.

[0052] Furthermore, as Figure 3 shown, obtaining atmospheric illumination data based on the light field image includes:

[0053] Perform convolution processing on the light field image to extract the initial features of the light field image;

[0054] Perform convolution processing on the initial features to obtain the atmospheric illumination data.

[0055] Specifically, in this embodiment, the extracted light field image is , perform multi-layer convolution operations on the light field image to extract the initial features of the light field image, and enhance the feature expression ability through residual connections. The dimension of the input light field image is [b, u, v, c, h, w], where b is the batch size, u and v are the angular dimensions (usually 5), c is the RGB channel (c = 3), and h and w are the spatial dimensions. The extracted light field image is the unenhanced light field sub-aperture array image.

[0056] Furthermore, obtaining prior data based on the light field image includes:

[0057] Convert the light field image into a two-dimensional feature map;

[0058] Calculate the dark channel prior data, bright channel prior data, and average channel prior data of the light field image based on the two-dimensional feature map.

[0059] Specifically, in this embodiment, prior extraction: extract the dark channel prior, bright channel prior, and average channel prior of the light field image. The specific steps include:

[0060] Convert the high-dimensional light field image into a two-dimensional feature map. The input light field image has a dimension of [b, u, v, c, h, w], and the rearranged two-dimensional feature map has a dimension of [b, c, uh, vw].

[0061] Calculate the dark channel prior, bright channel prior, and average channel prior respectively. The output dimensions of the dark channel prior, bright channel prior, and average channel prior are all [b, 1, uh, vw].

[0062] Furthermore, perform convolution processing on the light field image to extract the initial features of the light field image, including:

[0063] Perform four-layer convolution processing on the light field image in sequence; among them, LeakyReLU activation functions are connected after the second convolution, third convolution, and fourth convolution;

[0064] Add the output of the first convolution processing to the output of the fourth convolution processing to obtain the initial features.

[0065] Specifically, in this embodiment, perform multi-layer convolution operations on the input light field image to extract the initial features of the light field image, specifically including:

[0066] Use 4 convolutional layers, and each convolution is followed by a LeakyReLU activation function;

[0067] The first layer of convolution: use a convolutional layer with a kernel size of 3×3, and the number of output channels is 64;

[0068] The second layer of convolution: use a convolutional layer with a kernel size of 3×3, the number of output channels is 64, and it is followed by a LeakyReLU activation function;

[0069] The third layer of convolution: use a convolutional layer with a kernel size of 3×3, the number of output channels is 64, and it is followed by a LeakyReLU activation function;

[0070] The fourth layer of convolution: use a convolutional layer with a kernel size of 3×3, the number of output channels is 64, and it is followed by a LeakyReLU activation function;

[0071] Enhance the feature expression ability through residual connection, and add the output of the first convolution to the output of the fourth convolution. The output dimension of the initial features is [b, u, v, 64, h, w].

[0072] Furthermore, as Figure 6As shown, the initial features are convolved to obtain the atmospheric illumination data, including:

[0073] The initial features are subjected to four consecutive alternating feature extractions, and the result of the first alternating feature extraction is added to the result of the fourth alternating feature extraction to obtain the atmospheric illumination data.

[0074] Furthermore, the method of alternating feature extraction includes:

[0075] Perform EPI feature extraction on the initial features to obtain EPI feature data;

[0076] Based on the EPI feature data, perform spatial-angle feature extraction to obtain the result of alternating feature extraction.

[0077] Specifically, in this embodiment, four alternating feature extractions use a global-local feature extraction network for feature extraction. The initial features are fed into the global-local feature extraction network, and specifically, 4 alternating feature extraction modules are used: to extract the features of the light field image, with an input dimension of [b, u, v, 64, h, w], and the output dimension of the atmospheric illumination is [b, u, v, 64, h, w].

[0078] Furthermore, as Figure 7 shown, performing EPI feature extraction on the initial features includes:

[0079] Perform disparity information highlighting processing on the initial features, and then perform convolution processing after global feature extraction;

[0080] Add the data after convolution processing to the data after disparity information highlighting processing to obtain the first EPI feature;

[0081] Perform disparity information highlighting processing on the first EPI feature, and then perform convolution processing after global feature extraction;

[0082] Add the data after the second convolution processing to the data after the first disparity information highlighting processing to obtain the EPI feature data.

[0083] Specifically, in this embodiment, the EPI feature extraction utilizes an EPI feature extraction module:

[0084] Rearrange the light field image into a vertical Epipolar Plane Image (EPI-V) to highlight the disparity information;

[0085] Use the State Space Model to perform global feature extraction;

[0086] Use a convolutional layer with a convolutional kernel size of 3×3 and an output channel number of 64;

[0087] Enhance the feature expression ability using the SiLU activation function;

[0088] Rearrange the light field image into a horizontal Epipolar Plane Image (EPI-H) to highlight the parallax information;

[0089] Use the State Space Model to extract global features;

[0090] Use a convolutional layer with a kernel size of 3×3 and an output channel number of 64;

[0091] Enhance the feature expression ability using the SiLU activation function.

[0092] Specifically, in this embodiment, as Figure 8 shown, after extracting global features through the state space model, perform convolutional processing and utilize the spatial-angle feature extraction module:

[0093] Use the State Space Model to extract global features;

[0094] Use a convolutional layer with a kernel size of 3×3 and an output channel number of 64;

[0095] Use the SiLU activation function;

[0096] Rearrange the light field image into angular views to highlight the viewing angle information;

[0097] Use the State Space Model to extract global features;

[0098] Use a convolutional layer with a kernel size of 3×3 and an output channel number of 64;

[0099] Use the SiLU activation function;

[0100] Add the input features and the processed features through residual connection to enhance the feature expression ability. The dimension of the output global-local features is [b, u, v, 64, h, w].

[0101] Furthermore, as Figure 4 and Figure 5 shown, for the prior data, adopt a transmittance estimation method to obtain the light field image transmittance, including:

[0102] Perform 2D convolutional processing on the prior data to obtain preliminary features;

[0103] The preliminary features are sequentially subjected to residual convolution processing to obtain the first convolution output data;

[0104] The convolution output data is then subjected to 2D convolution processing to obtain the second convolution output data;

[0105] The prior data is added to the second convolution output data and processed by the Sigmoid activation function to obtain the transmittance of the light field image.

[0106] Specifically, the dimension of the input light field image is [b, u, v, c, h, w]. The high-dimensional light field image is converted into a two-dimensional feature map with the dimension of [b, c, uh, vw];

[0107] Calculate the dark channel prior, bright channel prior, and average channel prior respectively:

[0108] Dark channel prior: Calculate the minimum value Max of each pixel in all color channels;

[0109] Bright channel prior: Calculate the maximum value Min of each pixel in all color channels;

[0110] Average channel prior: Calculate the average value Mean of each pixel in all color channels;

[0111] The dimensions of the output dark channel prior, bright channel prior, and average channel prior are all [b, 1, uh, vw].

[0112] Specifically, in this embodiment, the dark channel prior, bright channel prior, and average channel prior are fed into the transmittance estimation module, the transmittance of the light field image; the dimension of the input prior feature is [b, 3, uh, vw], and the transmittance estimation module includes:

[0113] 2 convolutional layers, followed by the ReLU activation function after each convolution;

[0114] 3 residual convolution blocks: Each residual convolution block uses 2 convolutional layers, followed by the ReLU activation function after each convolution;

[0115] Sigmoid activation function: Pass the output of the convolution block through the Sigmoid activation function to obtain the transmittance;

[0116] Residual connection: Add the input feature to the output of the convolution block to enhance feature transmission;

[0117] The dimension of the output transmittance is [b, u, v, 3, h, w];

[0118] Furthermore, the data augmentation algorithm includes:

[0119]

[0120] Among them, is the light field image, is the atmospheric light illumination data, is the transmittance of the light field image, is the enhanced light field image.

[0121] Specifically, in this embodiment, The input light field image has a dimension of [b, u, v, c, h, w]; The estimated atmospheric light illumination has a dimension of [b, u, v, c, h, w]; The transmittance has a dimension of [b, u, v, c, h, w].

[0122] Specifically, in this embodiment, the light field image enhancement method for remote sensing images further includes training using a label supervision training strategy, where the loss function during training is:

[0123]

[0124] In the formula, represents the loss value, represents the true value label, represents the perceptual loss calculated from the high-level features of the VGG network pre-trained on ImageNet, represents the structural similarity loss, , and are weight coefficients.

[0125] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A light field image enhancement method for remote sensing images, characterized in that, Including: Obtain a light field image, and obtain atmospheric illumination data and prior data based on the light field image; For the prior data, use a transmittance estimation method to obtain the transmittance of the light field image; Based on the light field image, atmospheric illumination data, and the transmittance of the light field image, use a data enhancement algorithm to obtain an enhanced light field image; Obtaining atmospheric illumination data based on the light field image includes: Perform convolution processing on the light field image to extract the initial features of the light field image; Perform convolution processing on the initial features to obtain atmospheric illumination data; Performing convolution processing on the light field image to extract the initial features of the light field image includes: Perform four-layer convolution processing on the light field image in sequence; among them, LeakyReLU activation functions are connected after the second convolution, the third convolution, and the fourth convolution; Add the output of the first convolution processing and the output of the fourth convolution processing to obtain the initial features.

2. The light field image enhancement method for remote sensing images according to claim 1, characterized in that, Obtaining prior data based on the light field image includes: Convert the light field image into a two-dimensional feature map; Calculate the dark channel prior data, bright channel prior data, and average channel prior data of the light field image based on the two-dimensional feature map.

3. The light field image enhancement method for remote sensing images according to claim 1, characterized in that, Performing convolution processing on the initial features to obtain atmospheric illumination data includes: Perform four alternating feature extractions on the initial features in sequence, and add the result of the first alternating feature extraction and the result of the fourth alternating feature extraction to obtain atmospheric illumination data.

4. The light field image enhancement method for remote sensing images according to claim 3, characterized in that, The method of the alternating feature extraction includes: Perform EPI feature extraction on the initial features to obtain EPI feature data; Perform spatial-angle feature extraction based on the EPI feature data to obtain the result of the alternating feature extraction.

5. The light field image enhancement method for remote sensing images according to claim 4, characterized in that, Performing EPI feature extraction on the initial features includes: Perform parallax information highlighting processing on the initial features, and then perform convolution processing after global feature extraction; Add the data after convolution processing and the data after parallax information highlighting processing to obtain the first EPI feature; Perform parallax information highlighting processing on the first EPI feature, and then perform convolution processing after global feature extraction; Add the data after the second convolution processing and the data after the first parallax information highlighting processing to obtain the EPI feature data.

6. The light field image enhancement method for remote sensing images according to claim 1, characterized in that, For the prior data, using a transmittance estimation method to obtain the transmittance of the light field image includes: Perform 2D convolution processing on the prior data to obtain preliminary features; Perform residual convolution processing on the preliminary features in sequence to obtain the first convolution output data; The convolution output data is further processed by 2D convolution to obtain the second convolution output data; Add the prior data and the second convolution output data, and perform Sigmoid activation function processing to obtain the transmittance of the light field image.

7. The light field image enhancement method for remote sensing images according to claim 1, characterized in that, The data enhancement algorithm is: Among them, is the light field image, is the atmospheric light illumination data, is the transmittance of the light field image, is the enhanced light field image.

Images