A method for enhancing low-light images suitable for haze weather

Abstract

The present application relates to a kind of micro-light image enhancement method suitable for haze climate, belong to digital image processing field.First, the method can effectively learn the denoising information in image pair in the process of learning through GCA (Group Channel Attention) group channel attention structure, can solve the interference caused by external noise;Second, the transformer block of the method makes the network have better window adaptive ability through self-attention mechanism, can solve local overexposure phenomenon with better effect;Third, the method can be fine-tuned by a small amount of field samples, greatly improve the adaptability to the required environment, and can learn new tasks on the task of low-light enhancement, solve the influence of non-illumination factors on image quality.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of digital image processing, and particularly relates to a low-light image enhancement method suitable for foggy weather. BACKGROUND

[0002] Low-light enhancement is a technique for acquiring, converting, enhancing, recording, and displaying target image information in low-light conditions. However, the environment itself is affected by other non-low-light factors, such as fog on the sea at night, smog in the city at night, and rainy environment, etc. Therefore, when performing low-light enhancement in a foggy environment, both low-light enhancement and dehazing need to be considered.

[0003] At present, in the field of low-light enhancement, there have been many major breakthroughs in deep learning through different data sets and learning methods to deal with external light exposure, noise, color deviation, etc. However, from the actual scene, there are still many factors not considered, and the same is true for dehazing.

[0004] Low-light enhancement and dehazing are key subfields of image restoration, which are widely used in the field of computer vision. They can effectively capture the problem of missing image information caused by dark light and fog, and restore it based on the corresponding method.

[0005] Common traditional methods for low-light enhancement include image equalization, gamma correction, Retinex algorithm, etc. Common traditional methods for dehazing include dark channel dehazing, etc. In addition, many deep learning methods have also been gradually applied to low-light enhancement and dehazing tasks. Recent studies have shown that deep learning-based methods can significantly improve the visual quality of images under low-light or foggy conditions. For example, some methods can effectively reduce noise and enhance details, making the image visually clearer and more natural. With the further development of deep learning technology and the improvement of computing hardware, low-light image enhancement is expected to be applied in a wider range of application scenarios, including night shooting, security monitoring, and unmanned driving. At the same time, improving the efficiency and performance of the algorithm, and further exploring new deep learning architectures and training methods, will be the focus of future research.

[0006] In existing deep learning methods for low-light image enhancement, there are three major problems:

[0007] 1) Real low-light images are more susceptible to external additive noise due to their dark nature, which can be enhanced as useful information during the image enhancement process.

[0008] 2) In real low-light images, light sources often appear. Due to the limited tolerance of the camera sensor, it cannot record both the dark environment and the light source, resulting in a local overexposure problem in night shooting.

[0009] 3) The environment itself is affected by other non-micro-light factors, such as fog on the sea at night, haze in the city at night, and rainy environment, etc. Affected by these non-micro-light factors, only using a general micro-light enhancement network often cannot constrain this influence. Training two networks to complete the micro-light image enhancement and dehazing task and then splicing will bring the problem of doubling the model parameters. SUMMARY

[0010] The present application aims to overcome the problems of the prior art, and provides a micro-light image enhancement method suitable for haze climate. First, the method can effectively learn the de-noising information in the image pair in the learning process through the GCA (Group Channel Attention) group channel attention structure, and can solve the interference caused by external noise. Second, the transformer block of the method has good window adaptive ability through the self-attention mechanism, and can solve the local overexposure phenomenon with good effect. Third, the method can be fine-tuned by a small amount of field samples, greatly improving the adaptability to the required environment, and can learn new tasks on the micro-light enhancement task, solving the influence of non-illumination factors on image quality.

[0011] To achieve the above purpose, the technical scheme of the present application is: a micro-light image enhancement method suitable for haze climate, which uses a self-attention mechanism for self-distribution of image weights, and introduces an Unet structure to obtain multi-scale features to realize self-adaptive adjustment of the window in the micro-light image enhancement and dehazing task process, and uses a continuous learning training method to cooperate with the micro-light image enhancement and dehazing task.

[0012] In an embodiment of the present application, a micro-light image enhancement and dehazing network is constructed, which consists of two parts: the first part is an improved Transformer structure and a self-encoder-decoder composed of the improved Transformer structure, and the second part is a plug-and-play module composed of TUnet+GCA; finally, a continuous learning training method is used to make the micro-light image enhancement and dehazing network suitable for haze climate.

[0013] In an embodiment of the present application, the micro-light image enhancement and dehazing network structure is represented as:

[0014]

[0015] Wherein, a1, a2, b1, b2, c1, c2, res are all intermediate variables, T1 and T2 represent single-layer TransformerBlock and double-layer TransformerBlock respectively, F represents a feature fusion module, GCA is a group channel attention structure, Conv represents a convolution operation, the subscripts 1*1 and 3*3 represent 1*1 convolution kernel and 3*3 convolution respectively, and input represents input.

[0016] In an embodiment of the present application, the improved Transformer structure and the self-encoder-decoder composed of the improved Transformer structure are as follows:

[0017] The improved Transformer structure adopts a ternary gated feedforward network.

[0018] The improved Transformer structure is used to construct the front and rear self-encoder-decoders, that is, three Transformer blocks and a feature fusion module and a convolution layer are used to construct the self-encoder-decoder.

[0019] In an embodiment of the present application, the feature fusion module is composed of three convolution layers.

[0020] In an embodiment of the present application, the plug-and-play module composed of TUnet+GCA is as follows:

[0021] The improved Transformer structure is used to replace the original convolution layer in Unet to form the TUnet structure, that is, the improved Transformer structure with a predetermined number of layers is embedded in the four times of down-sampling and one time of up-sampling of the front structure of TUnet, the jump connection of the input and output ends, and the output result of the front structure of TUnet is connected to the rear structure of TUnet through a jump connection with parameters, and the last matrix summation operation is used to sum the three branches with parameters and send them to the GCA structure.

[0022] The GCA structure contains multiple jump links and concat splicing operations.

[0023] In an embodiment of the present application, the training method of continuous learning is as follows:

[0024] Step 1, adjust the micro-light image enhancement and dehazing network model structure according to the GPU size to ensure that no CUDA out of memory (insufficient memory) occurs.

[0025] Step 2, perform data preprocessing operations including random cropping and shuffling on the training data set.

[0026] Step 3: Introduce a paired dataset of low-light enhancement into the low-light image enhancement and dehazing network models, train the main task, i.e., the low-light image enhancement task, and obtain pre-trained weights, here named Params1;

[0027] Step 4. Import nighttime pictures or high-definition pictures of the required application scenario;

[0028] Step 5: If there is no HD image, perform post-stage degradation and pre-stage enhancement on the night image. If there is no night image and the HD image has reached the optimal state and does not need enhancement, only perform post-stage degradation on the HD image to obtain a new dataset, which is named Data1.

[0029] Step 6: Train the sub-task, i.e., the defogging task. The dataset uses the defogging paired dataset and the dataset processed by the previous and next stages in Step 4. A regularization constraint is introduced into the original loss in the sub-task training. The Euclidean distance between the pre-trained weights Params2 and Params1 in the sub-task training is constrained by regularization.

[0030] Step 7. If you need to continue training a new image restoration sub-task, repeat Step 4-Step 6; otherwise, proceed to Step 8.

[0031] Step 8: The final output result is obtained through post-processing including lighting correction and white balance.

[0032] In one embodiment of the present invention, the post-stage degradation model uses a fogging algorithm, dark processing and APSF blur kernel for light blurring; the pre-stage enhancement model uses the main task model for enhancement, and the obtained result is subjected to filtering and denoising.

[0033] In one embodiment of the present invention, the overall loss function of the method is as follows:

[0034] Loss=α×L1+β×L2+γ×(1-s)+L3

[0035]

[0036] Among them, α, β, γ, and weight are hyperparameters, result and label are predicted values ​​and true values, SSIM is structural similarity, params1 and params2 are the model weights of the main task and sub-task;

[0037] The loss term L1 is the edge loss term of the Sobel operator. It is obtained by using two types of Sobel operators, the horizontal axis and the vertical axis, to extract the edge information of the output result and the high-definition image, and using both as the input of the SmoothL1 Loss function.

[0038] The loss term L2 directly takes the network output result and the high-definition picture as the input of the SmoothL1 Loss to obtain;

[0039] The loss term 1-s is the SSIM loss, which is obtained by subtracting the SSIM index of the output result and the high-definition picture from 1;

[0040] The loss term L3 is a continuous learning regularization loss, which is used to constrain the distance between the learned parameters in the learning of the subtask and the learned parameters in the old task, and plays a role in reducing catastrophic forgetting.

[0041] The application also provides a computer readable storage medium, which stores computer program instructions capable of being run by a processor, and when the processor runs the computer program instructions, the method steps described above can be realized.

[0042] Compared with the prior art, the application has the following beneficial effects:

[0043] ① Compared with the existing deep learning method, the application does not increase the calculation amount of the model test and inference stage in the process of cooperating with the low-light enhancement and the defogging task, and reduces the training and inference burden of the GPU.

[0044] ② Compared with the existing end-to-end deep learning method, the image quality of the output result of the application exceeds the GT image of the low-light enhancement single task and the defogging single task, and breaks the method of relying on the stacking of picture quality of high-quality data sets.

[0045] ③ Compared with the existing traditional method, the innovation of the GCA structure and the ternary gating feedforward network of the Transformer makes the learning process be able to effectively utilize the intermediate features, and the effect on solving the external noise and the local exposure problem is remarkable.

[0046] ④ Compared with the existing training method, the application can fine-tune the model by using a small amount of application scene samples to adapt to different scenes. DETAILED DESCRIPTION

[0047] Figure 1 It is the overall network framework of the application.

[0048] Figure 2 It is a schematic diagram of the improved Transformer structure and the self-encoder-decoder structure composed of it.

[0049] Figure 3 It is a schematic diagram of the GCA structure.

[0050] Figure 4 It is a whole flow chart of the continuous learning training strategy.

[0051] Figure 5 It is a schematic diagram of the pre-stage enhancement and the post-stage degradation. DETAILED DESCRIPTION

[0052] The technical solutions of the present application will be described in detail below with reference to the drawings.

[0053] The present application provides a low-light image enhancement method suitable for haze climate, which adopts self-attention mechanism for self-distribution of image weight, and introduces Unet structure to obtain multi-scale features to realize self-adaptive adjustment of window in the process of low-light image enhancement and defogging task, and adopts continuous learning training mode to cooperate with low-light image enhancement and defogging task.

[0054] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise indicated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application pertains.

[0055] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.

[0056] As shown in Figure 1 The present embodiment provides a low-light image enhancement method suitable for haze climate, and the overall network framework is as shown in Figure 1 The self-attention mechanism is adopted for self-distribution of image weight, and the Unet structure is introduced to obtain multi-scale features to realize self-adaptive adjustment of window in the process of low-light image enhancement and defogging task, and the continuous learning training mode is adopted to cooperate with low-light image enhancement and defogging task. The entire network is composed of two parts: the first part is the self-encoder-decoder of the network and the improved Transformer structure therein, and the second part is the plug-and-play module composed of TUnet+GCA. Finally, the continuous learning training mode is adopted to make the low-light enhancement method suitable for haze climate.

[0057] In order to make the network obtain more knowledge of correlation between front and back levels in the learning process and speed up the convergence of the network, a large number of RL (Resnet Learning) operations are introduced in the front and back levels of the network structure, so that the network can capture the common points between different tasks faster in continuous learning training. The network structure of the present application is represented as:

[0058]

[0059] Wherein, a1, a2, b1, b2, c1, c2, res are all intermediate variables, T1 and T2 represent single-layer TransformerBlock and double-layer TransformerBlock respectively, F represents a feature fusion module, GCA is a group channel attention structure, Conv represents a convolution operation, and the subscripts 1*1 and 3*3 represent 1*1 convolution kernel and 3*3 convolution respectively, and input represents input.

[0060] Detail design

[0061] 1, Model structure

[0062] 1) Improved Transformer structure and self-encoder-decoder composed of it (as shown in Figure 2

[0063] A new type of Transformer structure is used on the network model, compared with the binary gated feedforward network used in the past, the ternary gated feedforward network is easy to learn more intermediate features, which raises the function that can only learn twice correlation to the highest exponential number of three, and three adaptive parameters are introduced, so as to realize the ternary gated feedforward network.

[0064] The Transformer structure is used to construct the front and rear self-encoder-decoders of the application, three Transformer blocks and a feature fusion module, a convolution layer are used to construct the self-encoder-decoder of the application. In addition, the feature fusion module of the self-encoder-decoder is composed of three convolution layers.

[0065] 2) TUnet+GCA

[0066] The TUnet structure of the application replaces the original convolution layer in Unet with the new Transformer structure designed by the application, and embeds a certain number of Transformer structures in the four times of down-sampling and one time of up-sampling of the front structure of TUnet, and the jump connection of the input and output ends, and connects the output results of the front stage with the rear stage through the jump connection with parameters, and finally the last matrix summation operation sums the three branches with parameters, and sends them to the GCA structure.

[0067] The GCA structure used in the application (as shown in Figure 3 ) contains multiple jump links and concat splicing operations, which is beneficial to the network to learn to retain the rich low-frequency information in the image. Since external additive noise usually exists in the image with a higher frequency. Therefore, the GCA structure can effectively learn the denoising information in the image pair in the learning process, and can better solve the interference caused by external noise in problem 1).

[0068] 2, Continuous learning training strategy​

[0069] Step1: Adjust the network model structure according to the size of the GPU itself to ensure that CUDA out of memory does not occur.

[0070] Step2: Randomly crop, shuffle and other data preprocessing operations are performed on the training required data set.

[0071] Step3: Introduce the paired data set (LOL, MIT, etc.) of the micro light enhancement on the network model, train the main task, and obtain the pre-training weight, which is named as Params1 here.

[0072] Step4: Introduce the night picture or high-definition picture of the required application scene, which can not require a paired data set. The data set used by us is a night picture.

[0073] Step5: If there is no high-definition picture, the night picture is subjected to post-level degradation and pre-level enhancement. If there is no night picture and the high-definition picture has reached the optimal and does not need to be enhanced, only the high-definition picture is subjected to post-level degradation. Thus, a new data set is obtained, which is named as Data1 here. (The post-level degradation model adopts the traditional fogging algorithm, dark processing and APSF blur kernel for light blur; the pre-level enhancement model adopts the main task model for enhancement, and the obtained result is appropriately subjected to filter denoising processing

determined by the shooting condition, if the noise has no influence, the denoising part can be ignored to avoid losing useful high-frequency information

[0074] Step6: Train the subtask, that is, the dehazing task, the data set adopts the dehazing paired data set (Dehaze_out, NH-HAZE, etc.) and the data set processed through the pre-level and post-level in Step4, and a regularization constraint term is introduced in the original loss of this training to constrain the Euclidean distance of the pre-training weight Params2 and Params1 in this training.

[0075] Step7: If it is required to continue training a new image repair subtask (such as deblurring, derain, super-resolution, etc.), Steps 4-6 are repeated.

[0076] Step8: The final output result is obtained through the post-processing mode such as light correction and white balance. (The overall flow chart is referred to in Figure 4 , and the pre-level enhancement and post-level degradation are referred to in Figure 5 ).

[0077] Since the network of the application can be fine-tuned through a small amount of on-site samples, the adaptability to the required environment is greatly improved, and new tasks can be learned on the micro light enhancement task. It has great advantages in solving problem 3) the influence of non-illumination factors on image quality, and the effect is remarkable.

[0078] 3, Loss function

[0079] The training mode ensures that the image can not only reduce the error at the pixel level during the recovery process, but also has a good recovery effect on the structure and illumination, so that the experimental results have high PSNR and SSIM indicators, and also have good visual effects. Therefore, the following expression is used as the loss function of the experiment, wherein alpha, beta, gamma and weight are hyperparameters, result and label are predicted values and true values, SSIM is a structural similarity, and params1 and params2 are model weights of the main task and the auxiliary task:

[0080] Loss = alpha * L1 + beta * L2 + gamma * (1-s) + L3

[0081] The loss function includes four terms, which can be represented as:

[0082]

[0083] The loss term L1 is the sobel operator edge loss term, which extracts the edge information of the output result and the high-definition image by using two types of sobel operators in the horizontal and vertical directions, and takes the two as the input of the SmoothL1 Loss (SmoothL1 Loss function) to obtain. The implicit edge prior knowledge is introduced into the loss term, so that the network can learn the edge information of the image in the training process.

[0084] The loss term L2 directly takes the network output result and the high-definition image as the input of the SmoothL1 Loss to obtain the loss of the image at the pixel level.

[0085] The loss term 1-s is the SSIM loss, which is obtained by subtracting the SSIM index of the output result and the high-definition image from 1. The loss term can make the network result have good structural similarity.

[0086] The loss term L3 is a continuous learning regularization loss, which is used to constrain the distance between the parameters learned by the new model in learning the auxiliary task and the parameters learned by the old task, so as to reduce the effect of catastrophic forgetting.

[0087] The application also provides a computer readable storage medium having computer program instructions capable of being run by a processor stored thereon, when the processor runs the computer program instructions, the method steps as described above can be realized.

[0088] Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In one

[0089] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

[0090] one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

[0091] one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

[0092] one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

Claims

1. A low-light image enhancement method suitable for haze climate, characterized in that: A self-attention mechanism is used to automatically assign image weights, and a Unet structure is introduced to obtain multi-scale features to achieve adaptive adjustment of windows during low-light image enhancement and defogging tasks. A continuous learning training method is also used to coordinate low-light image enhancement and defogging tasks. A low-light image enhancement and dehazing network was constructed, consisting of two parts: the first part is an improved Transformer structure and its own codec, and the second part is a plug-and-play module composed of TUnet and GCA. Finally, a continuous learning training method was used to make the low-light image enhancement and dehazing network suitable for foggy climates. The low-light image enhancement and dehazing network structure is expressed as: Among them, a1, a2, b1, b2, c1, c2, and res are all intermediate variables, T1 and T2 represent single-layer Transformer Block and double-layer Transformer Block respectively, F represents the feature fusion module, GCA is the group channel attention structure, Conv represents the convolution operation, the subscripts 1*1 and 3*3 represent 1*1 convolution kernel and 3*3 convolution respectively, and input represents the input; The improved Transformer structure and the self-encoder-decoder composed of the improved Transformer structure are as follows: Improved Transformer structure, using a ternary gated feedforward network; The improved Transformer structure is used to construct the front-end and back-end self-encoder-decoder, that is, three Transformer blocks, a feature fusion module, and a convolutional layer are used to construct the self-encoder-decoder respectively; The feature fusion module consists of three convolutional layers; The plug-and-play module composed of TUnet+GCA is as follows: The improved Transformer structure replaces the original convolutional layer in Unet to form the TUnet structure. That is, the improved Transformer structure with a predetermined number of layers is embedded in the four downsampling and one upsampling of the TUnet front-end structure and the jump connection at the input and output ends. The output result of the TUnet front-end structure is connected to the TUnet back-end structure through the jump connection with parameters. The last matrix summation operation sums the three parameterized branches and sends them to the GCA structure. The GCA structure contains multiple skip links and concat operations.

2. The low-light-level image enhancement method applicable to haze climate according to claim 1, characterized in that: The training methods for continuous learning are as follows: Step 1: Adjust the low-light image enhancement and dehazing network model structure according to the GPU size to ensure that there is no shortage of video memory; Step 2: Perform data preprocessing operations on the data set required for training, including random cropping and shuffling; Step 3: Introduce a paired dataset of low-light enhancement into the low-light image enhancement and dehazing network models, train the main task, i.e., the low-light image enhancement task, and obtain pre-trained weights, here named Params1; Step 4. Import nighttime pictures or high-definition pictures of the required application scenario; Step 5: If there is no HD image, perform post-stage degradation and pre-stage enhancement on the night image. If there is no night image and the HD image has reached the optimal state and does not need enhancement, only perform post-stage degradation on the HD image to obtain a new dataset, which is named Data1. Step 6: Train the sub-task, i.e., the defogging task. The dataset uses the defogging paired dataset and the dataset processed by the previous and next stages in Step 4. A regularization constraint is introduced into the original loss in the sub-task training. The Euclidean distance between the pre-trained weights Params2 and Params1 in the sub-task training is constrained by regularization. Step 7. If you need to continue training new image restoration tasks, repeat Step 4-Step 6. Otherwise, execute Step 8; Step 8: The final output result is obtained through post-processing including lighting correction and white balance.

3. The low-light-level image enhancement method applicable to haze climate according to claim 2, characterized in that: The post-stage degradation model uses atomization algorithm, dark processing and APSF blur kernel for light blurring; the pre-stage enhancement model uses the main task model for enhancement, and the obtained result is subjected to filtering and denoising.

4. The low-light-level image enhancement method applicable to haze climate according to claim 1, characterized in that: The overall loss function of this method is as follows: Loss=α×L1+β×L2+γ×(1-s)+L3 Among them, α, β, γ, and weight are hyperparameters, result and label are predicted values ​​and true values, SSIM is structural similarity, params1 and params2 are the model weights of the main task and sub-task; The loss term L1 is the edge loss term of the Sobel operator. It is obtained by using two types of Sobel operators, the horizontal axis and the vertical axis, to extract the edge information of the output result and the high-definition image, and using both as the input of the smooth L1 loss function SmoothL1 Loss. The loss term L2 directly uses the network output and high-definition image as the input of SmoothL1 Loss; The loss term 1-s is the SSIM loss, which is obtained by subtracting the SSIM index of the output result and the high-definition image from 1; The loss term L3 is the continuous learning regularization loss, which is used to constrain the distance between the parameters learned when learning the sub-task and the parameters learned in the old task, thereby reducing catastrophic forgetting.

5. A computer-readable storage medium storing computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, the method according to any one of claims 1 to 4 can be implemented.

Images