High dynamic reconstruction method and device for low-illumination remote sensing images

Abstract

The embodiment of the present invention discloses a high dynamic reconstruction method and device for low illumination remote sensing images. The high dynamic reconstruction method for low illumination remote sensing images comprises: acquiring a low illumination remote sensing image; mapping the low illumination remote sensing image data to a deep learning feature space to obtain a deep feature F x ; According to the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l , short-term characteristics y s is the pixel-level dynamic feature determined by at least the convolution operation in the spatial domain, and the long-term feature y l Characterize the deep feature F x The dependencies between the representation features are determined after processing by the Transformer-based pre-training model; according to the short-term feature y s and the long-term characteristics y l , determine the brightness enhancement curve; according to the brightness enhancement curve, adjust the low-illuminance remote sensing image pixel by pixel. Through the present invention, the problem of low accuracy of high-dynamic reconstruction of low-illuminance remote sensing images in the prior art is solved, and the high-dynamic reconstruction accuracy of low-illuminance remote sensing images is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a high-dynamic reconstruction method and device for low-illuminance remote sensing images. Background Art

[0002] At present, remote sensing images have been widely used in geological exploration, urban planning, disaster monitoring and other fields. In practical applications, due to the limitation of imaging time or unexpected weather conditions, such as at night or on cloudy days with insufficient light, usually only low-light remote sensing images can be captured. This results in low contrast images that are difficult for machines to understand. How to effectively enhance the brightness when the original remote sensing image illumination is too low and complete the task of high-dynamic reconstruction of remote sensing images has gradually attracted widespread attention from scholars at home and abroad.

[0003] In the prior art, there is a problem of low accuracy when adjusting low-illuminance remote sensing images, and the effects after adjustment vary greatly for different images, resulting in poor display effects of the adjusted images.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet.

[0005] The above information disclosed in the background technology section is only used to enhance the understanding of the background technology of the technology described in this article. Therefore, the background technology may contain some information that does not form the known prior art for those skilled in the art. Summary of the invention

[0006] The embodiments of the present invention provide a method and device for high dynamic reconstruction of low-illuminance remote sensing images, so as to at least solve the problem of low accuracy in high dynamic reconstruction of low-illuminance remote sensing images in the prior art.

[0007] According to a first aspect of an embodiment of the present invention, a high dynamic reconstruction method for low-light remote sensing images is provided, which comprises: acquiring a low-light remote sensing image; mapping the low-light remote sensing image data to a deep learning feature space to obtain a deep feature F x ; According to the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l , short-term characteristics y s is the pixel-level dynamic feature determined by at least the convolution operation in the spatial domain, and the long-term feature y l Characterize the deep feature F x The dependencies between the representation features are determined after processing by the Transformer-based pre-training model; according to the short-term feature y s and the long-term characteristics y l, determine the brightness enhancement curve; according to the brightness enhancement curve, adjust the low-illumination remote sensing image pixel by pixel.

[0008] Optionally, based on the short-term feature y s and the long-term characteristics y l , determine the brightness enhancement curve, including: converting the short-term characteristic y s and the long-term characteristics y l Input the pre-trained brightness enhancement model to obtain a brightness enhancement curve.

[0009] Optionally, the brightness enhancement model is used to: i (I(x); α i )=LE i-1 (x)+α i LE i-1 (x)(1-LE i-1 (x)) calculates the brightness enhancement curve; where α i =τ(tanh(FC([y s ,y l ]))), α i is the pixel scale factor, FC([y s ,y l ]) indicates that long-distance features are converted by using a fully connected layer, LE i represents the result of brightness enhancement, LE0(x)=x, τ is the interpolation function, and i is the number of iterations.

[0010] Optionally, the high dynamic reconstruction method for low-illuminance remote sensing images further includes: optimizing the brightness enhancement model through a back propagation algorithm according to a solution result of the loss function.

[0011] Optionally, according to the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l , including: transforming the deep feature F x The weight generation network and multiple base networks are input respectively, and activated with a linear rectification layer, wherein the multiple base networks have multiple convolution windows of different sizes correspondingly; according to the output results of the weight generation network and the output results of the multiple base networks, the short-term feature y is determined s .

[0012] Optionally, the short-term feature y is determined based on the output of the weight generation network and the output of multiple base networks. s , including: inputting the output results of the weight generation network and the output results of multiple base networks into the linear fusion model In the above example, we get the short-term feature y s ; Among them, g i (Fx ,θ i ) is given by θ i The output result of the parameterized i-th base network, n is the total number of base networks, Reason The parameterized weights generate the output of the network.

[0013] Optionally, according to the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l , including: transforming the deep feature F x Flattened into a series of vectors The vector F t Input the Transformer-based pre-trained model to obtain the long-term feature y l ; Where L is the vector length and Ct is the number of channels after mapping.

[0014] Optionally, a Transformer-based pre-trained model is used to: add a learnable positional encoding to each labeled vector feature; use a multi-head self-attention model to determine the dependencies between vectors in the deep feature space; and use a feedforward neural network with skip connections to process the output of the multi-head self-attention model to obtain the long-term feature y l .

[0015] Optionally, acquiring the low-illumination remote sensing image includes: based on an input initial remote sensing image, simulating and generating low-illumination remote sensing image data corresponding to the initial remote sensing image.

[0016] According to a second aspect of an embodiment of the present invention, a high dynamic reconstruction device for low-light remote sensing images is also provided, which includes: an acquisition unit, used to acquire a low-light remote sensing image; a mapping unit, used to map the low-light remote sensing image data to a deep learning feature space to obtain a deep feature F x ; A first determination unit, configured to determine the depth of the feature F x , determine the short-term characteristics y s and the long-term characteristics y l , short-term characteristics y s is the pixel-level dynamic feature determined by at least the convolution operation in the spatial domain, and the long-term feature y l Characterize the deep feature F x The dependency relationship between the representation features is determined after being processed by the Transformer-based pre-training model; the second determination unit is used to determine the dependency relationship between the representation features according to the short-term feature y s and the long-term characteristics y l , determine the brightness enhancement curve; and the adjustment unit is used to adjust the low-illuminance remote sensing image pixel by pixel according to the brightness enhancement curve.

[0017] The high dynamic reconstruction method of low illumination remote sensing images of the embodiment of the present invention comprises: acquiring low illumination remote sensing images; mapping the low illumination remote sensing image data to a deep learning feature space to obtain a deep feature F x ; According to the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l , short-term characteristics y s is the pixel-level dynamic feature determined by at least the convolution operation in the spatial domain, and the long-term feature y l Characterize the deep feature F x The dependencies between the representation features are determined after processing by the Transformer-based pre-training model; according to the short-term feature y s and the long-term characteristics y l , determine the brightness enhancement curve; according to the brightness enhancement curve, adjust the low-light remote sensing image pixel by pixel. The high dynamic reconstruction method of low-light remote sensing images using the above implementation method utilizes the long-term and short-term characteristics of low-light remote sensing images in the process of high-dynamic reconstruction of low-exposure remote sensing images, combines pixel-level dynamic features and feature component dependencies, determines the brightness enhancement curve, and then adjusts the low-light remote sensing images pixel by pixel according to the brightness enhancement curve. For different low-light remote sensing images, specific adjustments will be made based on their specific long-term and short-term features, so that the basis for high-dynamic reconstruction of low-light remote sensing images is more comprehensive and accurate, and can adaptively fit specific images, effectively improving the high-dynamic reconstruction accuracy of low-light remote sensing images, and solving the problem of low accuracy of high-dynamic reconstruction of low-light remote sensing images in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 A schematic diagram of a process for high dynamic reconstruction of low-illumination remote sensing images provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of a high dynamic reconstruction device for low-illumination remote sensing images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second" and the like in the specification, claims and drawings of the present invention are used to distinguish different objects rather than to limit a specific order.

[0023] The embodiment of the present invention provides a high dynamic reconstruction method for low-illumination remote sensing images. Figure 1 FIG. 1 is a flow chart of a high dynamic reconstruction method for low-light remote sensing images according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0024] Step S102, acquiring a low-light remote sensing image;

[0025] Step S104, mapping the low-light remote sensing image data to the deep learning feature space to obtain the deep feature F x ;

[0026] Step S106: Based on the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l , short-term characteristics y s is the pixel-level dynamic feature determined by at least the convolution operation in the spatial domain, and the long-term feature y l Characterize the deep feature F x The dependencies between representation features determined after processing by the Transformer-based pre-trained model;

[0027] Step S108, based on the short-term feature y s and the long-term characteristics y l , determine the brightness enhancement curve;

[0028] Step S110: adjusting the low-illuminance remote sensing image pixel by pixel according to the brightness enhancement curve.

[0029] The high dynamic reconstruction method of low illumination remote sensing images according to an embodiment of the present invention comprises: acquiring a low illumination remote sensing image; mapping the low illumination remote sensing image data to a deep learning feature space, and obtaining a deep feature F x ; According to the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l, short-term characteristics y s is the pixel-level dynamic feature determined by at least the convolution operation in the spatial domain, and the long-term feature y l Characterize the deep feature F x The dependencies between the representation features are determined after processing by the Transformer-based pre-training model; according to the short-term feature y s and the long-term characteristics y l , determine the brightness enhancement curve; according to the brightness enhancement curve, adjust the low-light remote sensing image pixel by pixel. The high dynamic reconstruction method of low-light remote sensing images using the above implementation method utilizes the long-term and short-term characteristics of low-light remote sensing images in the process of high-dynamic reconstruction of low-exposure remote sensing images, combines pixel-level dynamic features and feature component dependencies, determines the brightness enhancement curve, and then adjusts the low-light remote sensing images pixel by pixel according to the brightness enhancement curve. For different low-light remote sensing images, specific adjustments will be made based on their specific long-term and short-term features, so that the basis for high-dynamic reconstruction of low-light remote sensing images is more comprehensive and accurate, and can adaptively fit specific images, effectively improving the high-dynamic reconstruction accuracy of low-light remote sensing images, and solving the problem of low accuracy of high-dynamic reconstruction of low-light remote sensing images in the prior art.

[0030] It should be pointed out that the low-light remote sensing image mentioned in this application does not mean that the exposure of the remote sensing image is lower than a certain value. According to the actual brightness requirements, as long as the brightness of the remote sensing image does not meet the expectations, it can be considered as a low-light remote sensing image. It can also be understood that after the high-dynamic reconstruction of the remote sensing image, as long as at least part of its brightness is improved, the remote sensing image before reconstruction is a low-light remote sensing image relative to after reconstruction. The so-called image reconstruction generates a new image. This new image can be an image recreated independently of the original image, or a new image formed by directly modifying and overwriting the original image.

[0031] For low-light remote sensing image data Its high dynamic image data is Where H represents the image length, W represents the image height, and C represents the number of image channels.

[0032] Specifically, according to the short-term characteristics y s and the long-term characteristics y l , determine the brightness enhancement curve, including: converting the short-term characteristic y s and the long-term characteristics y l Input the pre-trained brightness enhancement model to obtain a brightness enhancement curve.

[0033] Map the low-light remote sensing image data to the deep learning feature space to obtain the deep feature F x, including: mapping low-light remote sensing image data to deep feature space through convolution layer; obtaining deep feature F through adaptive global average pooling layer x In this way, the amount of calculation in the process of processing remote sensing images can be effectively reduced, which is conducive to improving the processing speed. For example, in a specific embodiment, the convolution window size of the convolution layer is 7×7, the step size is 4, and the output channel is 16. The adaptive global average pooling layer reduces the feature map to one eighth of the original size.

[0034] In a specific embodiment, the brightness enhancement model is used to: i (I(x); α i )=LE i-1 (x)+α i LE i-1 (x)(1-LE i-1 (x)) calculates the brightness enhancement curve; where α i =τ(tanh(FC([y s ,y l ]))), α i is the pixel scale factor, FC([y s ,y l ]) indicates that long-distance features are converted by using a fully connected layer, LE i represents the result of brightness enhancement, LE0(x)=x, τ is the interpolation function, and i is the number of iterations. Using this function to calculate the brightness enhancement curve can obtain an enhancement curve that is more in line with the brightness characteristics of low-illuminance remote sensing images, thereby improving the display effect of the reconstructed remote sensing image.

[0035] Specifically, the high dynamic reconstruction method of low-illuminance remote sensing images also includes: optimizing the brightness enhancement model through a back propagation algorithm according to the solution result of the loss function. This is conducive to improving the accuracy of the brightness enhancement model, thereby ensuring that more accurate high dynamic images can be created subsequently.

[0036] In a preferred embodiment, the loss function / error function is The learning rate of the brightness enhancement model is le-3, and the total number of training times is 400.

[0037] Specifically, according to the depth feature F x , determine the short-term characteristics y s and the long-term characteristics y l , including: transforming the deep feature F x The weight generation network and multiple base networks are input respectively, and activated with a linear rectification layer, wherein the multiple base networks have multiple convolution windows of different sizes correspondingly; according to the output results of the weight generation network and the output results of the multiple base networks, the short-term feature y is determined s .

[0038] Specifically, the short-term feature y is determined based on the output of the weight generation network and the output of multiple base networks. s , including: inputting the output results of the weight generation network and the output results of multiple base networks into the linear fusion model In the above example, we get the short-term feature y s ; Among them, g i (F x ,θ i ) is given by θ i The output result of the parameterized i-th base network, n is the total number of base networks, Reason The parameterized weights generate the output of the network.

[0039] In this embodiment, n is 3, and Fx is input into three parallel base networks and a weight generation network. The three base networks are composed of convolution blocks with convolution windows of 3, 7, and 11, and two convolution blocks with convolution windows of size 3, and activated by a linear rectifier layer (Rectified Linear Unit, ReLU). The weight generation network consists of two stacked convolution blocks with a convolution window of 3 and activated by a linear rectifier layer. Then the linear fusion module is used to integrate the results of all branches with the weights output by the weight generation network.

[0040] According to the deep feature F x , determine the short-term characteristics y s and the long-term characteristics y l , including: transforming the deep feature F x Flattened into a series of vectors The vector F t Input the Transformer-based pre-trained model to obtain the long-term feature y l ; Where L is the vector length and Ct is the number of channels after mapping.

[0041] Specifically, the Transformer-based pre-trained model is used to: add a learnable positional encoding to each labeled vector feature; use a multi-head self-attention model to determine the dependencies between vectors in the deep feature space; use a feedforward neural network with skip connections to process the output of the multi-head self-attention model to obtain the long-term feature y l .

[0042] In this embodiment, the Transformer-based pre-training model is specifically used to add a learnable position encoding to each labeled vector feature, that is, y0 = F t+p; a multi-head self-attention model is used to determine the dependencies between vectors in the deep feature space, that is, The output result of the multi-head self-attention model is processed using a feedforward neural network with skip connections to obtain the long-term feature y l ,Right now Among them, the above p is a learnable position code, MSA is a multi-head self-attention model, FFN is a feedforward neural network, and LN represents layer normalization.

[0043] In a specific embodiment, a new dual-branch network architecture is proposed. The dynamic long-term and short-term feature extraction network includes two branches, one of which is a pixel-level dynamic feature extraction branch, which includes three parallel base network branches and a weight generation network branch, and the second branch is a long-term feature extraction branch, which makes the model more integrated and convenient to manage and use.

[0044] Specifically, obtaining a low-illumination remote sensing image includes: based on an input initial remote sensing image, simulating and generating low-illumination remote sensing image data corresponding to the initial remote sensing image.

[0045] A low-illumination remote sensing image is simulated based on the input initial remote sensing image, and then the simulated low-illumination remote sensing image is reconstructed with high dynamic range using long-term and short-term features in subsequent steps. The difference between the initial remote sensing image and the reconstructed image can be compared more intuitively, so as to more clearly grasp the effect of remote sensing image reconstruction, facilitate the adjustment of method steps, model parameters, etc., and thus help to improve the quality of high-dynamic reconstruction of remote sensing images.

[0046] Specifically, based on the input initial remote sensing image, the low-light remote sensing image data corresponding to the initial remote sensing image can be generated by a variety of technical means, as long as the camera can simulate the low-light remote sensing image captured on a cloudy day or at night. For example, in a specific embodiment, the formula Z can be used. i,j =f(E i Δt j ) is used to simulate low-light images, where f represents the camera response function and Δtj represents the exposure time. Ei represents the pixel value of the original remote sensing image at i. Zi,j represents the pixel value of pixel i at the continuous exposure time index j, i.e., the obtained low-light remote sensing image data. In order to improve the simulation quality, in this embodiment, the most representative curves can be selected from multiple preset response curves. For the selection of exposure time, let Then, EiΔtj is normalized so that the average pixel value of EiΔtT / 2+1 is 0.5. Substitute the exposure time into the above low-light image simulation formula to obtain the corresponding low-light remote sensing image data. Preferably, T=8, τ=√2, j=1,2,...,T+1.

[0047] Secondly, if Figure 2 As shown, an embodiment of the present invention further provides a high dynamic reconstruction device for low illumination remote sensing images, which includes: an acquisition unit, used to acquire low illumination remote sensing images; a mapping unit, used to map the low illumination remote sensing image data to a deep learning feature space to obtain a deep feature F x ; A first determination unit, configured to determine the depth of the feature F x , determine the short-term characteristics y s and the long-term characteristics y l , short-term characteristics y s is the pixel-level dynamic feature determined by at least the convolution operation in the spatial domain, and the long-term feature y l Characterize the deep feature F x The dependency relationship between the representation features is determined after being processed by the Transformer-based pre-training model; the second determination unit is used to determine the dependency relationship between the representation features according to the short-term feature y s and the long-term characteristics y l , determine the brightness enhancement curve; the adjustment unit is used to adjust the low-illuminance remote sensing image pixel by pixel according to the brightness enhancement curve. When the high-dynamic reconstruction device of low-illuminance remote sensing images using the above implementation method is used, it simultaneously utilizes the long-term and short-term characteristics of the low-illuminance remote sensing images, combines the pixel-level dynamic characteristics and the feature component dependency, determines the brightness enhancement curve, and then adjusts the low-illuminance remote sensing images pixel by pixel according to the brightness enhancement curve. In addition, for different low-illuminance remote sensing images, specific adjustments will be made based on their specific long-term and short-term characteristics, so that the basis for the high-dynamic reconstruction of low-illuminance remote sensing images is more comprehensive and accurate, and can adaptively fit specific images, effectively improving the high-dynamic reconstruction accuracy of low-illuminance remote sensing images, and solving the problem of low accuracy of high-dynamic reconstruction of low-illuminance remote sensing images in the prior art.

[0048] It should be pointed out that the low-light remote sensing image mentioned in this application does not mean that the exposure of the remote sensing image is lower than a certain value. According to the actual brightness requirements, as long as the brightness of the remote sensing image does not meet the expectations, it can be considered as a low-light remote sensing image. It can also be understood that after the high-dynamic reconstruction of the remote sensing image, as long as at least part of its brightness is improved, the remote sensing image before reconstruction is a low-light remote sensing image relative to after reconstruction. The so-called image reconstruction generates a new image. This new image can be an image recreated independently of the original image, or a new image formed by directly modifying and overwriting the original image.

[0049] For low-light remote sensing image data Its high dynamic image data is Where H represents the image length, W represents the image height, and C represents the number of image channels.

[0050] Specifically, the second determining unit is used to: s and the long-term characteristics y l Input the pre-trained brightness enhancement model to obtain a brightness enhancement curve.

[0051] The mapping unit is used to: map the low-light remote sensing image data to the deep feature space through the convolution layer; obtain the deep feature F through the adaptive global average pooling layer x In this way, the amount of calculation in the process of processing remote sensing images can be effectively reduced, which is conducive to improving the processing speed. For example, in a specific embodiment, the convolution window size of the convolution layer is 7×7, the step size is 4, and the output channel is 16. The adaptive global average pooling layer reduces the feature map to one eighth of the original size.

[0052] Specifically, the brightness enhancement model is used to: According to the function LE i (I(x); α i )=LE i-1 (x)+α i LE i-1 (x)(1-LE i-1 (x)) calculates the brightness enhancement curve; where α i =τ(tanh(FC([y s ,y l ]))), α i is the pixel scale factor, FC([y s ,y l ]) indicates that long-distance features are converted by using a fully connected layer, LE i represents the result of brightness enhancement, LE0(x)=x, τ is the interpolation function, and i is the number of iterations. Using this function to calculate the brightness enhancement curve can obtain an enhancement curve that is more in line with the brightness characteristics of low-illuminance remote sensing images, thereby improving the display effect of the reconstructed remote sensing image.

[0053] Specifically, the high dynamic reconstruction device for low-illuminance remote sensing images also includes an optimization unit, which is used to optimize the brightness enhancement model through a back propagation algorithm according to the solution result of the loss function. This is conducive to improving the accuracy of the brightness enhancement model, thereby ensuring that more accurate high dynamic images can be created later.

[0054] In a preferred embodiment, the loss function / error function is The learning rate of the brightness enhancement model is le-3, and the total number of training times is 400.

[0055] Specifically, the first determination unit includes a first input module and a determination module: the first input module is used to convert the depth feature F xThe weight generation network and the multiple base networks are input respectively, and activated by a linear rectification layer, wherein the multiple base networks have multiple convolution windows of different sizes correspondingly; the determination module is used to determine the short-term feature y according to the output results of the weight generation network and the output results of the multiple base networks. s .

[0056] Specifically, the determination module is used to: input the output results of the weight generation network and the output results of multiple base networks into the linear fusion model In the above example, we get the short-term feature y s ; Among them, g i (F x ,θ i ) is given by θ i The output result of the parameterized i-th base network, n is the total number of base networks, Reason The parameterized weights generate the output of the network.

[0057] In this embodiment, n is 3, and Fx is input into three parallel base networks and a weight generation network. The three base networks are composed of convolution blocks with convolution windows of 3, 7, and 11, and two convolution blocks with convolution windows of size 3, and activated by a linear rectifier layer (Rectified Linear Unit, ReLU). The weight generation network consists of two stacked convolution blocks with a convolution window of 3 and activated by a linear rectifier layer. Then the linear fusion module is used to integrate the results of all branches with the weights output by the weight generation network.

[0058] Specifically, the first determination unit further includes a flattening module and a second input module: the flattening module is used to transform the depth feature F x Flattened into a series of vectors The second input module is used to transform the vector F t Input the Transformer-based pre-trained model to obtain the long-term feature y l ; Where L is the vector length and Ct is the number of channels after mapping.

[0059] Specifically, the Transformer-based pre-trained model is used to: add a learnable positional encoding to each labeled vector feature; use a multi-head self-attention model to determine the dependencies between vectors in the deep feature space; use a feedforward neural network with skip connections to process the output of the multi-head self-attention model to obtain the long-term feature y l .

[0060] In this embodiment, the Transformer-based pre-training model is specifically used to add a learnable position encoding to each labeled vector feature, that is, y0 = F t+p; a multi-head self-attention model is used to determine the dependencies between vectors in the deep feature space, that is, The output result of the multi-head self-attention model is processed using a feedforward neural network with skip connections to obtain the long-term feature y l ,Right now Among them, the above p is a learnable position code, MSA is a multi-head self-attention model, FFN is a feedforward neural network, and LN represents layer normalization.

[0061] In a specific embodiment, a new dual-branch network architecture is proposed. The dynamic long-term and short-term feature extraction network includes two branches, one of which is a pixel-level dynamic feature extraction branch, which includes three parallel base network branches and a weight generation network branch, and the second branch is a long-term feature extraction branch, which makes the model more integrated and convenient to manage and use.

[0062] Specifically, the acquisition unit includes a simulation module, and the simulation module is used to simulate and generate low-illuminance remote sensing image data corresponding to the initial remote sensing image according to the input initial remote sensing image.

[0063] A low-illumination remote sensing image is simulated based on the input initial remote sensing image, and then the simulated low-illumination remote sensing image is reconstructed with high dynamic range using long-term and short-term features in subsequent steps. The difference between the initial remote sensing image and the reconstructed image can be compared more intuitively, so as to more clearly grasp the effect of remote sensing image reconstruction, facilitate the adjustment of method steps, model parameters, etc., and thus help to improve the quality of high-dynamic reconstruction of remote sensing images.

[0064] Specifically, based on the input initial remote sensing image, the low-light remote sensing image data corresponding to the initial remote sensing image can be generated by a variety of technical means, as long as the camera can simulate the low-light remote sensing image captured on a cloudy day or at night. For example, in a specific embodiment, the formula Z can be used. i,j =f(E i Δt j ) is used to simulate low-light images, where f represents the camera response function and Δtj represents the exposure time. Ei represents the pixel value of the original remote sensing image at i. Zi,j represents the pixel value of pixel i at the continuous exposure time index j, i.e., the obtained low-light remote sensing image data. In order to improve the simulation quality, in this embodiment, the most representative curves can be selected from multiple preset response curves. For the selection of exposure time, let Then, EiΔtj is normalized so that the average pixel value of EiΔtT / 2+1 is 0.5. Substitute the exposure time into the above low-light image simulation formula to obtain the corresponding low-light remote sensing image data. Preferably, T=8, τ=√2, j=1,2,...,T+1.

[0065] In addition, an embodiment of the present invention further provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned high dynamic reconstruction method of low-illumination remote sensing images.

[0066] Thirdly, an embodiment of the present invention further provides a processor, and the processor is used to run a program, wherein the program executes the above-mentioned high dynamic reconstruction method of low-illumination remote sensing images when running.

[0067] Finally, an embodiment of the present invention also provides a high dynamic reconstruction device for low-illumination remote sensing images, including a display, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-mentioned high dynamic reconstruction method for low-illumination remote sensing images.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A high dynamic reconstruction method for low-illumination remote sensing images, characterized in that: include: Acquire low-light remote sensing images; Map the low-light remote sensing image data to the deep learning feature space to obtain the deep feature ; According to the depth feature , determine the short-term characteristics and long-term characteristics , the short-term characteristics The long-term feature is a pixel-level dynamic feature determined at least based on a convolution operation in the spatial domain. Characterizing the deep features The dependencies between representation features determined after processing by the Transformer-based pre-trained model; According to the short-term characteristics and the long-term characteristics , determine the brightness enhancement curve; According to the brightness enhancement curve, adjusting the low-illuminance remote sensing image pixel by pixel; According to the short-term characteristics and the long-term characteristics , determine the brightness enhancement curve, including: The short-term characteristics and the long-term characteristics Inputting the pre-trained brightness enhancement model to obtain the brightness enhancement curve; The brightness enhancement model is used to: According to the function Calculating the brightness enhancement curve; in, , is the pixel scale factor, It means that by using the fully connected layer to transform long-distance features, Indicates the result of brightness enhancement, , τ is the interpolation function, i is the number of iterations; According to the depth feature , determine the short-term characteristics and long-term characteristics ,include: The depth feature The weight generation network and the multiple base networks are input respectively, and activated by a linear rectification layer, wherein the multiple base networks correspondingly have multiple convolution windows of different sizes; Determine the short-term feature according to the output result of the weight generation network and the output results of the plurality of base networks .

2. The high dynamic reconstruction method of low illumination remote sensing images according to claim 1, characterized in that: The high dynamic reconstruction method of the low illumination remote sensing image also includes: According to the solution result of the loss function, the brightness enhancement model is optimized through a back propagation algorithm.

3. The high dynamic reconstruction method of low illumination remote sensing images according to claim 1, characterized in that: Determine the short-term feature according to the output result of the weight generation network and the output results of the plurality of base networks ,include: The output result of the weight generation network and the output results of the multiple base networks are input into the linear fusion model The short-term characteristics are obtained ; in, Reason The parameterized output result of the i-th base network, n is the total number of base networks, Reason The parameterized weights generate the output of the network.

4. The high dynamic reconstruction method of low illumination remote sensing images according to claim 1, characterized in that: According to the depth feature , determine the short-term characteristics and long-term characteristics ,include: The depth feature Flattened into a series of vectors ; The vector Input the Transformer-based pre-trained model to obtain the long-term features ; Among them, L is the vector length and Ct is the number of channels after mapping.

5. The high dynamic reconstruction method of low illumination remote sensing images according to claim 1, characterized in that: The Transformer-based pre-trained model is used to: Add a learnable positional encoding to each labeled vector feature; A multi-head self-attention model is used to determine the dependencies between vectors in the deep feature space; The output result of the multi-head self-attention model is processed by a feedforward neural network with skip connections to obtain the long-term feature .

6. The high dynamic reconstruction method of low illumination remote sensing images according to any one of claims 1 to 5, characterized in that: Acquire low-light remote sensing images, including: According to the input initial remote sensing image, low-illumination remote sensing image data corresponding to the initial remote sensing image is simulated and generated.

7. A high dynamic reconstruction device for low-light remote sensing images, characterized in that: include: An acquisition unit, used for acquiring low-light remote sensing images; A mapping unit is used to map the low-light remote sensing image data to a deep learning feature space to obtain a deep feature ; A first determining unit is configured to determine, based on the depth feature , determine the short-term characteristics and long-term characteristics , the short-term characteristics The long-term feature is a pixel-level dynamic feature determined at least based on a convolution operation in the spatial domain. Characterizing the deep features The dependencies between representation features determined after processing by the Transformer-based pre-trained model; The second determining unit is used to determine the short-term characteristics according to the short-term characteristics. and the long-term characteristics , determine the brightness enhancement curve; An adjustment unit, configured to adjust the low-illuminance remote sensing image pixel by pixel according to the brightness enhancement curve; According to the short-term characteristics and the long-term characteristics , determine the brightness enhancement curve, including: The short-term characteristics and the long-term characteristics Inputting the pre-trained brightness enhancement model to obtain the brightness enhancement curve; The brightness enhancement model is used to: According to the function Calculating the brightness enhancement curve; in, , is the pixel scale factor, It means that by using the fully connected layer to transform long-distance features, Indicates the result of brightness enhancement, , τ is the interpolation function, i is the number of iterations; According to the depth feature , determine the short-term characteristics and long-term characteristics ,include: The depth feature The weight generation network and the multiple base networks are input respectively, and activated by a linear rectification layer, wherein the multiple base networks correspondingly have multiple convolution windows of different sizes; Determine the short-term feature according to the output result of the weight generation network and the output results of the plurality of base networks .

Images