A Hyperspectral Eigendecomposition Method Based on Physics Knowledge-Driven and Unsupervised Learning

Abstract

This invention presents a hyperspectral intrinsic decomposition method based on physics-driven and unsupervised learning, belonging to the field of remote sensing image processing. This invention addresses the problem of poor accuracy in reflectance components obtained using existing intrinsic decomposition methods. The main technical solution adopted in this invention is as follows: Step 1, decompose the hyperspectral remote sensing image based on a physical model to obtain reflectance components; Step 2, construct a reflectance component estimation sub-network and its output loss term; Step 3, construct a brightness component estimation sub-network and its output loss term; Step 4, combine the physical model output with the original hyperspectral data to generate input data with hybrid physics knowledge, and initialize the model parameters; Step 5, optimize the network model by incorporating the loss function term, and output the reflectance components. This invention can be applied to the intrinsic decomposition of hyperspectral remote sensing images.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing, specifically relating to a hyperspectral intrinsic decomposition method based on physical knowledge-driven and unsupervised learning. Background Technology

[0002] Hyperspectral remote sensing images contain rich spectral information, which is of great significance for applications such as accurate classification of ground features, target identification, and agricultural monitoring. However, due to the influence of various environmental factors during the hyperspectral image imaging process, the reflectivity of the same material can vary, resulting in the phenomenon of heterospectral representation of the same material. This can lead to a decrease in the accuracy of hyperspectral remote sensing images in subsequent applications.

[0003] In recent years, to address the issue of different spectra for the same object, intrinsic eigenvalue decomposition (IED) techniques have gradually developed in the hyperspectral field. However, current IED methods in remote sensing are mainly based on optimized models. These methods establish reasonable constraint assumptions based on physical observation models and obtain iterative or non-iterative estimates of intrinsic components by simplifying the model. However, due to the simplification of the model, optimized models have certain limitations in their IED capabilities, resulting in poor accuracy of the reflectance components decomposed using existing methods. With the continuous development of deep learning technology, deep learning-based models have also been applied to remote sensing to better explore the potential of remote sensing image data in subsequent tasks. However, there are currently no precedents for using deep learning frameworks to solve the hyperspectral IED problem; therefore, using deep learning networks to solve the hyperspectral IED problem remains a technical bottleneck. Summary of the Invention

[0004] The purpose of this invention is to address the problem of poor accuracy in reflectance components decomposed using existing intrinsic decomposition methods. It proposes a hyperspectral intrinsic decomposition method based on physical knowledge-driven and unsupervised learning. By combining knowledge from the output of the physical model, a dual-channel intrinsic decomposition unsupervised network consisting of two estimation sub-networks is constructed. This fully mines the intrinsic information from hyperspectral remote sensing image data, completing the task of decomposing high-quality reflectance components from a single hyperspectral remote sensing image.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0006] A hyperspectral eigenvalue decomposition method based on physics knowledge-driven and unsupervised learning, the method specifically includes the following steps:

[0007] Step 1: Decompose the hyperspectral remote sensing image based on the physical model to obtain the reflectance component;

[0008] Step 2: Construct the reflectivity component estimation subnetwork and the output loss term of the reflectivity component estimation subnetwork;

[0009] Step 3: Construct the light and dark component estimation subnetwork and the output loss term of the light and dark component estimation subnetwork;

[0010] Step 4: Generate input data based on the decomposition results in Step 1, construct a reconstruction loss term by combining the output loss terms of the reflectance component estimation subnetwork and the brightness component estimation subnetwork, and pre-train the reflectance component estimation subnetwork and the brightness component estimation subnetwork according to the input data and the reconstruction loss term. Stop the pre-training when the set number of pre-training times M1 is reached, and use the network parameters at the end of the pre-training as the initialization parameters when training the reflectance component estimation subnetwork and the brightness component estimation subnetwork.

[0011] Step 5: Construct a loss function term based on the reconstruction loss term, the output loss term of the reflectance component estimation subnetwork, and the output loss term of the brightness component estimation subnetwork. Use the input data and the loss function term to train the parameter-initialized reflectance component estimation subnetwork and the brightness component estimation subnetwork. Stop training when the set number of training times M2 is reached. Use the output of the reflectance component estimation subnetwork at the time of stopping training as the final reflectance component.

[0012] The beneficial effects of this invention are:

[0013] This invention incorporates a knowledge input network from a physical model, enabling the network to learn knowledge that cannot be learned from raw hyperspectral remote sensing images. Furthermore, this invention proposes a reasonable loss function term for the two intrinsic component estimation subnetworks based on physical model observations of hyperspectral remote sensing images. Compared to existing technologies, this invention improves intrinsic decomposition performance, obtains highly accurate reflectance components, and is more conducive to subsequent performance in fine-grained observations and other applications.

[0014] To verify the performance of the proposed method, an experiment was conducted using a set of hyperspectral remote sensing data captured by the ZY-1 02D satellite. The experimental results verified the effectiveness of the proposed hyperspectral intrinsic decomposition method based on physical knowledge-driven and unsupervised learning. Attached Figure Description

[0015] Figure 1 This is a flowchart of a hyperspectral intrinsic decomposition method based on physical knowledge-driven and unsupervised learning according to the present invention;

[0016] Figure 2 This is a structural diagram of the reflectivity component estimation subnetwork constructed in this invention;

[0017] Figure 3 This is a structural diagram of the light and dark component estimation subnetwork constructed in this invention. Detailed Implementation

[0018] The present application will now be described in further detail with reference to specific embodiments and accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are all within the scope of protection of the present invention.

[0019] Specific Implementation Method 1: Combination Figure 1 This embodiment describes a hyperspectral intrinsic decomposition method based on physics knowledge-driven and unsupervised learning. The method specifically includes the following steps:

[0020] Step 1: Decompose the hyperspectral remote sensing image based on the physical model to obtain the reflectance component;

[0021] Step 2: Construct the reflectivity component estimation subnetwork and the output loss term of the reflectivity component estimation subnetwork;

[0022] Step 3: Construct the light and dark component estimation subnetwork and the output loss term of the light and dark component estimation subnetwork;

[0023] Step 4: Generate input data based on the decomposition results in Step 1, construct a reconstruction loss term by combining the output loss terms of the reflectance component estimation subnetwork and the brightness component estimation subnetwork, and pre-train the reflectance component estimation subnetwork and the brightness component estimation subnetwork according to the input data and the reconstruction loss term. Stop the pre-training when the set number of pre-training times M1 is reached (set to 1000 times in this invention), and use the network parameters at the end of the pre-training as the initialization parameters when training the reflectance component estimation subnetwork and the brightness component estimation subnetwork.

[0024] Step 5: Construct a loss function term based on the reconstruction loss term, the output loss term of the reflectance component estimation subnetwork, and the output loss term of the brightness component estimation subnetwork. Use the input data and the loss function term to train the parameter-initialized reflectance component estimation subnetwork and brightness component estimation subnetwork. Stop training when the set number of training times M2 is reached (set to 9000 times in this invention). Use the output of the reflectance component estimation subnetwork at the time of stopping training as the final reflectance component.

[0025] During pre-training and training, the inputs to both the reflectance component estimation subnetwork and the brightness component estimation subnetwork are generated input data. This invention combines physical model knowledge with a deep learning model, and for the first time applies a deep learning framework to the intrinsic decomposition of hyperspectral remote sensing images. It fully leverages the role of knowledge from the physical model and the network's feature extraction capabilities to improve the intrinsic decomposition of hyperspectral images, estimating high-quality intrinsic reflectance components. Furthermore, the method of this invention can optimize the best network estimation model for a single hyperspectral remote sensing image, generating high-quality reflectance components, has no sample size limitation, and is easy to implement.

[0026] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the specific process of step 1 is as follows:

[0027] The physical model for decomposing hyperspectral remote sensing images into brightness components and reflectance components that are only related to the material itself (this physical model is represented in the logarithmic domain by pixel-by-pixel addition) is as follows:

[0028] I = ρ + S

[0029] Where I represents the hyperspectral remote sensing image, ρ represents the reflectance component, and S represents the brightness component. Let n represent the real number field, n represent the total number of pixels in the hyperspectral remote sensing image, and d represent the spectral number of the hyperspectral remote sensing image.

[0030] Based on the fundamental model of hyperspectral eigenvalue decomposition, the eigencomponents can be calculated in a non-iterative manner. The expression for the reflectance component is as follows:

[0031]

[0032] in, This represents a hyperspectral remote sensing image after averaging in both the spectral and spatial dimensions. E n Let I represent an n×n identity matrix. d Denotes a d×d identity matrix, 1 d Let d×1 be a vector of all 1s, 1 n Let G represent an n×1 vector of all 1s, where the superscript T indicates transpose, the superscript -1 indicates the inverse of the matrix, and G indicates the decomposition matrix.

[0033]

[0034] in, This represents the similarity matrix, which describes the degree of similarity between pixels in an input hyperspectral remote sensing image and can be measured by the spectral angle between pixels.

[0035] The other steps and parameters are the same as in Specific Implementation Method 1.

[0036] Specific implementation method three: Combining Figure 2 This embodiment is described below. The difference between this embodiment and specific embodiments one or two is that, in step 2, the constructed reflectivity component estimation sub-network includes an encoder, a decoder, a first connection module, a second connection module, a third connection module, a fourth connection module, and a fifth connection module.

[0037] For the encoder part of the reflectivity component estimation subnetwork:

[0038] Starting from the input end of the encoder, the encoder section sequentially includes a first downsampling module, a second downsampling module, a first residual module, a third downsampling module, a fourth downsampling module, and a fifth downsampling module, wherein:

[0039] The first downsampling module includes a 7×7 convolutional layer with a stride of 2, a batch normalization (BN) layer, and a linear rectified function (ReLU) layer;

[0040] The second downsampling module contains a 3×3 max pooling layer with a stride of 2;

[0041] The first residual module contains three residual units A;

[0042] The third downsampling module includes four downsampling residual units A';

[0043] The fourth downsampling module includes 6 downsampling residual units A';

[0044] The fifth downsampling module includes three downsampling residual units A';

[0045] The residual unit A includes a 1×1 convolutional layer with a stride of 1, a 3×3 convolutional layer with a stride of 2, and a 1×1 convolutional layer with a stride of 1; that is, the residual unit A is obtained by removing the average pooling layer and the fully connected layer in the Restnet-50 structure.

[0046] The downsampling residual unit A' is a convolutional layer with a size of 1×1 and a stride of 2 following the residual unit A;

[0047] For the decoder part of the reflectivity component estimation subnetwork:

[0048] Starting from the input of the decoder, the decoder section sequentially includes a first upsampling module, a second upsampling module, a third upsampling module, a fourth upsampling module, a second residual module, a fifth upsampling module, and a sigmoid activation function module, wherein:

[0049] The first upsampling module contains 6 upsampling residual units B';

[0050] The second upsampling module contains four upsampling residual units B';

[0051] The third upsampling module includes three upsampling residual units B';

[0052] The fourth upsampling module includes three upsampling residual units B';

[0053] The second residual module contains three residual units B;

[0054] The fifth upsampling module includes a convolutional layer of size 2×2 with a stride of 0.5;

[0055] The residual unit B includes a 3×3 convolutional layer with a stride of 1 and a 3×3 convolutional layer with a stride of 0.5; the upsampling residual unit B' is a convolutional layer with a size of 2×2 and a stride of 0.5 following the residual unit B.

[0056] The output of the first downsampling module passes through the first connection module, and then the output of the first connection module is concatenated with the output of the fourth upsampling module. The concatenation result is used as the input of the second residual module.

[0057] The output of the first residual module passes through the second connection module, and then the output of the second connection module is concatenated with the output of the third upsampling module. The concatenation result is used as the input of the fourth upsampling module.

[0058] The output of the third downsampling module is then passed through the third connection module, and the output of the third connection module is then concatenated with the output of the second upsampling module. The concatenation result is then used as the input of the third upsampling module.

[0059] The output of the fourth downsampling module is then passed through the fourth connection module, and the output of the fourth connection module is then concatenated with the output of the first upsampling module. The concatenation result is then used as the input of the second upsampling module.

[0060] The output of the fifth downsampling module is then passed through the fifth connection module, and the output of the fifth connection module is concatenated with the output of the fifth downsampling module. The concatenation result is used as the input of the first upsampling module.

[0061] The first connection module, the second connection module, the third connection module, the fourth connection module, and the fifth connection module have the same structure, and each connection module includes a 1×1 convolutional layer with a stride of 1, a batch normalization layer, and a linear rectified function layer.

[0062] Other steps and parameters are the same as in specific implementation method one or two.

[0063] The working process of the reflectivity component estimation subnetwork is as follows:

[0064] The input data is used as the input to the encoder's first downsampling module. Within the encoder, the input data sequentially passes through the first downsampling module, the second downsampling module, the first residual module, the third downsampling module, the fourth downsampling module, and the fifth downsampling module. The output of the fifth downsampling module passes through the fifth connection module, and the output of the fifth connection module and the output of the fifth downsampling module are concatenated as the input to the decoder's first upsampling module. Within the decoder, after passing through the first upsampling module, the output of the first upsampling module is concatenated with the output of the fourth connection module, and the concatenated result is input to the second upsampling module. The second upsampling module... The output of the block is concatenated with the output of the third connection module, and the concatenation result is input into the third upsampling module. The output of the third upsampling module is concatenated with the output of the second connection module, and the concatenation result is input into the fourth upsampling module. The output of the first downsampling module passes through the first connection module, and the output of the first connection module is concatenated with the output of the fourth upsampling module. The concatenation result is used as the input of the second residual module. The input of the second residual module then passes through the fifth upsampling module and the sigmoid activation function module in sequence. The output of the sigmoid activation function module is used as the output of the reflectance component estimation subnetwork.

[0065] The downsampling module downsamples the feature map and increases the number of feature channels. A sigmoid activation function is applied at the end of the reflectance estimation subnetwork to ensure that the output reflectance component value is within the range of 0 to 1. The connection module transforms the feature map from the encoding structure into a feature map with a smaller number of channels and passes it to the decoding structure.

[0066] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the output loss term of the reflectivity component estimation subnetwork is:

[0067]

[0068] Among them, L ref (ρ) is the output loss term of the reflectance component estimation subnetwork, N(·) represents the normalization operation of the energy amplitude of pixels in the hyperspectral remote sensing image, A(·) represents the spectral angle of adjacent pixels in the hyperspectral remote sensing image after energy amplitude normalization, e is the base of the natural logarithm, ||·||2 represents the 2-norm, and |·| represents the absolute value. Represents the dot product operator, ▽2 represents the gradient operator vector, ▽2=[▽ 2x ,▽ 2y ],▽2x The gradient operator in the horizontal direction, ▽ 2y This represents the gradient operator in the vertical direction. The gradient operator can be the Laplacian operator.

[0069] Based on the assumption of physical observation, pixels with similar intensities in a local neighborhood also have similar reflectance spectra, and this similarity can be measured by the spectral angle (SAM) index. However, since hyperspectral remote sensing imagery is affected by many environmental factors, such as the non-uniform distribution of illumination conditions at different spatial locations within the same scene, the same type of land cover may exhibit similar spectral curves in hyperspectral images but differ in energy amplitude. To eliminate the influence of energy amplitude differences, amplitude normalization is performed before the spectral angle index to more accurately measure the spectral similarity between pixels.

[0070] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0071] Specific Implementation Method Five: Combining Figure 3 This embodiment is described below. The difference between this embodiment and one of the specific embodiments one to four is that the brightness / darkness component estimation subnetwork includes two parts: an encoder and a decoder.

[0072] For the encoder part of the light and dark component estimation subnetwork:

[0073] Starting from the input end of the encoder, the encoder sequentially includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, and a fifth convolution module;

[0074] The first convolutional module contains the following layers in sequence: a 5×5 convolutional layer with a stride of 2, a BN layer, a linear rectified function layer, a 1×1 convolutional layer with a stride of 1, a BN layer, and a linear rectified function layer.

[0075] The structures of the second, third, fourth, and fifth convolutional modules are all the same as those of the first convolutional module;

[0076] For the decoder part of the brightness and darkness component estimation subnetwork:

[0077] Starting from the input end of the decoder, the decoder section sequentially includes a first upsampling convolution module, a second upsampling convolution module, a third upsampling convolution module, a fourth upsampling convolution module, a fifth upsampling convolution module, and a sixth convolution module;

[0078] The first upsampling convolutional module includes a bilinear upsampling layer, a 1×1 convolutional layer with a stride of 1, a BN layer, and a linear rectified function layer;

[0079] The structures of the second, third, fourth, and fifth upsampling convolutional modules are all the same as those of the first upsampling convolutional module.

[0080] The sixth convolutional module contains a convolutional layer with a size of 1×1 and a stride of 1.

[0081] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0082] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the working process of the brightness and darkness component estimation sub-network is as follows:

[0083] The input data is used as the input to the first convolutional module of the encoder. The input data passes through the first convolutional module, the second convolutional module, the third convolutional module, the fourth convolutional module and the fifth convolutional module in sequence in the encoder. The output of the fifth convolutional module is used as the input to the first upsampling convolutional module of the decoder.

[0084] The output of the first upsampling convolutional module is concatenated with the output of the fourth convolutional module, and the concatenation result is used as the input of the second upsampling convolutional module.

[0085] The output of the second upsampling convolutional module is concatenated with the output of the third convolutional module, and the concatenation result is used as the input of the third upsampling convolutional module.

[0086] The output of the third upsampling convolutional module then passes through the fourth, fifth, and sixth upsampling convolutional modules in sequence. The output of the sixth convolutional module is then used as the output of the brightness and darkness component estimation subnetwork.

[0087] Bilinear upsampling layers can increase the size of the feature map while reducing the number of feature channels. Similar to the reflectance estimation subnetwork, skip connections are applied in the brightness component estimation subnetwork. Since deep networks often contain more brightness and darkness structure information, skip connections are only selected from the third and fourth layers of the encoder and decoder.

[0088] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0089] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the output loss term of the brightness and darkness component estimation sub-network is:

[0090]

[0091] Among them, L sha (S) is the output loss term of the light and dark component estimation subnetwork.

[0092] Considering that the brightness and darkness components in hyperspectral remote sensing images contain spatially varying illumination components, the output loss term for the brightness and darkness components includes two parts: one is to consider the continuously varying regions in the original hyperspectral image, and the corresponding brightness and darkness component variations should also be continuous; the other is to consider the regions with discontinuous variations under the same reflectance component, and the discontinuous variations in these regions in the hyperspectral image can only be caused by the brightness and darkness components.

[0093] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0094] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that the input data is:

[0095] D = [I, ρ]

[0096] Where D represents the input data.

[0097] In this invention, data with mixed physical knowledge is used as the input to the network to enhance the performance of unsupervised networks.

[0098] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0099] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that the reconstruction loss term is:

[0100] L rec (ρ,S)=||logρ+logS-logI||1

[0101] Among them, L rec (ρ,S) is the reconstruction loss term, ||·||1 is the 1 norm, and log is the natural logarithm, i.e., the base is e.

[0102] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0103] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One through Nine in that the loss function term is:

[0104]

[0105] Where λ1 and λ2 are respectively L ref (R) and L sha The weight of (S).

[0106] In this embodiment, λ1 is set to 0.9 and λ2 to 0.5. Adjusting the weights enhances network performance and improves the accuracy of the decomposed reflectance components. The Adam optimizer is used for network optimization, with a learning rate of 10. -3 .

[0107] Its steps and parameters are the same as those in one of the specific implementation methods one to nine.

[0108] Experimental section:

[0109] The data used in the experiment consisted of a set of hyperspectral remote sensing images taken by the ZY-1 02D satellite, with a spatial resolution of 30m and an image size of 192×192×156. Table 1 shows the intrinsic decomposition results using the method of this invention, the decomposition results based on the optimized model, and a comparison of intra-class consistency of reflectance in the original hyperspectral remote sensing images (the smaller the value, the better the intra-class consistency). The comparison results show that the reflectance components generated by the method proposed in this invention can effectively improve the intrinsic decomposition accuracy of hyperspectral remote sensing images.

[0110] Table 1

[0111]

[0112] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for hyperspectral endmember decomposition based on physical knowledge driving and unsupervised learning, characterized in that, The method specifically comprises the following steps: Step 1, decomposing a hyperspectral remote sensing image based on a physical model to obtain a reflectance component; The specific process of step 1 is as follows: The physical model for decomposing the hyperspectral remote sensing image into the light-dark component and the reflectance component is as follows: wherein, denotes a hyperspectral remote sensing image, denotes a reflectance component, denotes a shading component, , , , denotes the real domain, n denotes the total number of pixels in the hyperspectral remote sensing image, and d is the spectral number of the hyperspectral remote sensing image; The expression of the reflectance component is as follows: wherein, denotes a hyperspectral remote sensing image after spectral and spatial averaging, , denotes an identity matrix, denotes an identity matrix, denotes an all-one vector, denotes an all-one vector, the upper index T denotes transposition, and the upper index -1 denotes the inverse of a matrix, denotes a decomposition matrix; wherein denotes a similarity matrix; Step 2, constructing a reflectance component estimation subnetwork and an output loss term of the reflectance component estimation subnetwork; The output loss term of the reflectance component estimation subnetwork is as follows: wherein, is an output loss term of the reflectance component estimation subnetwork, denotes a normalization operation on the energy amplitude of a pixel in the hyperspectral remote sensing image, denotes a spectral angle between adjacent pixels in the energy amplitude normalized hyperspectral remote sensing image, e is the base of the natural logarithm, represents a 2-norm, represents an absolute value, represents a dot product operator, denotes a gradient operator vector, , denotes a horizontal gradient operator, denotes a vertical gradient operator; Step 3, constructing a light-dark component estimation subnetwork and an output loss term of the light-dark component estimation subnetwork; The output loss term of the light-dark component estimation subnetwork is as follows: wherein, is the output loss term of the light-dark component estimation subnetwork; Step 4, generating input data based on the decomposition result in step 1, combining the output loss terms of the reflectance component estimation subnetwork and the light-dark component estimation subnetwork to construct a reconstruction loss term, pre-training the reflectance component estimation subnetwork and the light-dark component estimation subnetwork according to the input data and the reconstruction loss term, stopping the pre-training when a set pre-training number M1 is reached, and taking the network parameters at the end of the pre-training as the initialization parameters for training the reflectance component estimation subnetwork and the light-dark component estimation subnetwork; The reconstruction loss term is as follows: wherein, is a reconstruction loss term, is a 1-norm, and log is the natural logarithm; Step 5, constructing a loss function term according to the reconstruction loss term, the output loss term of the reflectance component estimation subnetwork and the output loss term of the light-dark component estimation subnetwork, training the reflectance component estimation subnetwork and the light-dark component estimation subnetwork after parameter initialization by using the input data and the loss function term, stopping the training when a set training number M2 is reached, and taking the output of the reflectance component estimation subnetwork at the stop of the training as the final reflectance component; The loss function term is as follows: wherein and are the weights of and respectively.

2. The method of claim 1, wherein, In step 2, the constructed reflectance component estimation subnetwork comprises an encoder, a decoder, a first connection module, a second connection module, a third connection module, a fourth connection module and a fifth connection module; For the encoder part of the reflectance component estimation subnetwork: Starting from the input end of the encoder, the encoder part comprises a first downsampling module, a second downsampling module, a first residual module, a third downsampling module, a fourth downsampling module and a fifth downsampling module in sequence, wherein: The first downsampling module comprises a convolution layer with a size of 7x7 and a step of 2, a batch normalization layer and a linear rectifier function layer; The second downsampling module comprises a max-pooling layer with a size of 3x3 and a step of 2; The first residual module comprises 3 residual units A; The third downsampling module comprises 4 downsampling residual units A'; The fourth downsampling module comprises 6 downsampling residual units A'; The fifth downsampling module comprises 3 downsampling residual units A'; The residual unit A includes a convolution layer with a size of 1x1 and a step of 1, a convolution layer with a size of 3x3 and a step of 2, and a convolution layer with a size of 1 x1 and a step of 1; The downsampling residual unit A' is a convolution layer with a size of 1x1 and a step of 2 connected after the residual unit A; For the decoder part of the reflectance component estimation subnetwork: Starting from the input end of the decoder, the decoder part comprises a first upsampling module, a second upsampling module, a third upsampling module, a fourth upsampling module, a second residual module, a fifth upsampling module and a sigmoid activation function module in sequence, wherein: The first up-sampling module comprises six up-sampling residual units B'; The second up-sampling module comprises four up-sampling residual units B'; The third up-sampling module comprises three up-sampling residual units B'; The fourth up-sampling module comprises three up-sampling residual units B'; The second residual module comprises three residual units B; The fifth up-sampling module comprises a convolutional layer with a size of 2*2 and a stride of 0.5; The residual unit B comprises a convolutional layer with a size of 3*3 and a stride of 1, and a convolutional layer with a size of 3*3 and a stride of 0.5; the up-sampling residual unit B' is connected with a convolutional layer with a size of 2*2 and a stride of 0.5 after the residual unit B; The output of the first down-sampling module is input into the first connection module, and the output of the first connection module is spliced with the output of the fourth up-sampling module, and the spliced result is input into the second residual module; The output of the first residual module is input into the second connection module, and the output of the second connection module is spliced with the output of the third up-sampling module, and the spliced result is input into the fourth up-sampling module; The output of the third down-sampling module is input into the third connection module, and the output of the third connection module is spliced with the output of the second up-sampling module, and the spliced result is input into the third up-sampling module; The output of the fourth down-sampling module is input into the fourth connection module, and the output of the fourth connection module is spliced with the output of the first up-sampling module, and the spliced result is input into the second up-sampling module; The output of the fifth down-sampling module is input into the fifth connection module, and the output of the fifth connection module is spliced with the output of the fifth down-sampling module, and the spliced result is input into the first up-sampling module; The first connection module, the second connection module, the third connection module, the fourth connection module and the fifth connection module have the same structure, and each connection module comprises a convolutional layer with a size of 1*1 and a stride of 1, a batch normalization layer and a linear rectifier function layer.

3. The method of claim 2, wherein, The light-dark component estimation sub-network comprises an encoder and a decoder; For the encoder part of the light-dark component estimation sub-network: The encoder comprises a first convolutional module, a second convolutional module, a third convolutional module, a fourth convolutional module and a fifth convolutional module in sequence from the input end of the encoder; The layers in the first convolutional module are in sequence: a convolutional layer with a size of 5*5 and a stride of 2, a BN layer, a linear rectifier function layer, a convolutional layer with a size of 1*1 and a stride of 1, a BN layer and a linear rectifier function layer; The structures of the second convolutional module, the third convolutional module, the fourth convolutional module and the fifth convolutional module are the same as that of the first convolutional module; For the decoder part of the light-dark component estimation sub-network: The decoder part comprises a first up-sampling convolutional module, a second up-sampling convolutional module, a third up-sampling convolutional module, a fourth up-sampling convolutional module, a fifth up-sampling convolutional module and a sixth convolutional module in sequence from the input end of the decoder; The first up-sampling convolution module comprises a bilinear up-sampling layer, a convolution layer with a size of 1*1 and a step of 1, a BN layer, and a linear rectifier function layer; The second up-sampling convolution module, the third up-sampling convolution module, the fourth up-sampling convolution module and the fifth up-sampling convolution module all have the same structure as the first up-sampling convolution module; The sixth convolution module comprises a convolution layer with a size of 1*1 and a step of 1.

4. The method of claim 3, wherein, The working process of the light-dark component estimation sub-network is as follows: input data is input into the first convolution module of the encoder, and the input data sequentially passes through the first convolution module, the second convolution module, the third convolution module, the fourth convolution module and the fifth convolution module in the encoder, and the output of the fifth convolution module is input into the first up-sampling convolution module of the decoder; the output of the first up-sampling convolution module is spliced with the output of the fourth convolution module, and the spliced result is input into the second up-sampling convolution module; the output of the second up-sampling convolution module is spliced with the output of the third convolution module, and the spliced result is input into the third up-sampling convolution module; the output of the third up-sampling convolution module sequentially passes through the fourth up-sampling convolution module, the fifth up-sampling convolution module and the sixth convolution module, and the output of the sixth convolution module is output as the output of the light-dark component estimation sub-network.

5. The method of claim 4, wherein, The input data is as follows: wherein is the input data.

Images