A method and apparatus for image sharpening in foggy weather

Abstract

This invention provides a method and apparatus for de-hazing images, belonging to the field of image processing. The method includes: designing a haze removal module, a haze synthesis module, a color and texture restoration module, and a discriminator; constructing haze removal paths, de-hazing result restoration paths, haze synthesis paths, and synthesis result restoration paths through the haze removal module, haze synthesis module, and color and texture restoration module; and training the haze removal path and de-hazing result restoration path based on the original haze image to obtain a haze-removed image, a color and texture-processed original image, a haze-removed synthesized image, and a de-hazing result restored image. This invention effectively solves the problems of difficulty in estimating based on prior knowledge and difficulty in obtaining real paired training datasets based on supervised learning. While effectively removing haze in various scenes, it also preserves good color and texture details, resulting in de-hazed images with richer image content and higher generation quality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and apparatus for improving image clarity in foggy weather. Background Technology

[0002] Image vision is one of the primary sources of human perception of the external world. However, extreme weather conditions such as fog and smog severely damage the quality of acquired images, causing information loss, blurred texture details, and color distortion. This affects human visual perception and limits the development of top-level computer vision fields (object detection and tracking, autonomous driving, satellite remote sensing, etc.). Therefore, image dehazing has become one of the important fundamental tasks in the fields of digital image processing and computer vision. However, due to the lack of real-world paired training data samples, existing dehazing methods in this field struggle to remove fog from real-world scenes.

[0003] In recent years, the field of single-image dehazing has attracted numerous researchers, resulting in a multitude of methods. Specifically, image dehazing algorithms can be broadly categorized into prior-based methods and learning-based methods. Prior-based methods rely on estimating prior knowledge, and their poor adaptability means they only achieve good dehazing results for specific scenes. For example, Kaiming He's dark channel prior dehazing algorithm won the 2009 CVPR Best Paper Award, providing a novel dehazing approach for later researchers. However, this method exhibits large areas of color spots and artifacts when processing outdoor sky areas or white buildings. Learning-based methods are mainly divided into supervised learning methods and unsupervised learning methods. However, the former suffers from difficulties in obtaining fog-related training datasets and poor performance in processing real foggy images; the latter suffers from inaccurate fog-related feature extraction and low image generation quality. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and apparatus for improving image clarity in foggy weather, addressing the shortcomings of the prior art.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A method for clarifying images in foggy weather, comprising the following steps:

[0006] S1: Design a fog removal module, a fog synthesis module, a color and texture restoration module, and a discriminator, and construct a fog removal path, a defogging result restoration path, a fog synthesis path, and a synthesis result restoration path through the fog removal module, the fog synthesis module, and the color and texture restoration module;

[0007] S2: Import multiple original foggy images, and train the fog removal path and the defogging result restoration path according to each original foggy image to obtain the fog-removed image, the original image after color and texture processing, the fog-removed composite image, and the defogging result restoration image of each original foggy image.

[0008] S3: Import the original foggy clear image corresponding to each of the original foggy images, and train the fog synthesis path and the synthesis result recovery path according to each of the original foggy clear images to obtain the fog clear synthesized image, fog-removed clear image and synthesis result recovery image of each of the original foggy clear images;

[0009] S4: The discriminator analyzes the loss values ​​of all the original foggy images, all the original clear foggy images, all the fog-removed images, all the original images after color and texture processing, all the fog-removed composite images, all the defogging result restored images, all the foggy clear composite images, all the fog-removed clear images, and all the composite result restored images to obtain the total loss value;

[0010] S5: Update the parameters of the fog removal module according to the total loss value to obtain the updated fog removal module, and use the updated fog removal module as the fog removal module for the next iteration, and return to execute S2 until the preset number of iterations is reached, and use the updated fog removal module obtained after the preset number of iterations as the target fog removal module.

[0011] S6: Import the foggy image to be clarified, and perform image clarification processing on the foggy image to be clarified through the target fog removal module to obtain the foggy image clarification result.

[0012] Based on the above-mentioned method for clarifying foggy images, the present invention also provides a device for clarifying foggy images.

[0013] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: A device for improving image clarity in foggy weather, comprising:

[0014] The path construction module is used to design the fog removal module, fog synthesis module, color and texture restoration module, and discriminator, and to construct the fog removal path, the fog result restoration path, the fog synthesis path, and the synthesis result restoration path through the fog removal module, the fog synthesis module, and the color and texture restoration module.

[0015] The original foggy image training module is used to import multiple original foggy images, and train the fog removal path and the defogging result restoration path according to each original foggy image to obtain the fog-removed image, the original image after color and texture processing, the fog-removed composite image, and the defogging result restoration image of each original foggy image.

[0016] The fog clear image training module is used to import the original fog clear images corresponding to each of the original fog images, and to train the fog synthesis path and the synthesis result recovery path according to each of the original fog clear images, so as to obtain the fog clear synthesized image, fog-removed clear image and synthesis result recovery image of each of the original fog clear images;

[0017] The loss value analysis module is used to analyze the loss values ​​of all the original foggy images, all the original clear foggy images, all the fog-removed images, all the original images after color and texture processing, all the fog-removed composite images, all the defogging result restored images, all the foggy clear composite images, all the fog-removed clear images, and all the composite result restored images through the discriminator, and obtain the total loss value;

[0018] The parameter update module is used to update the parameters of the fog removal module according to the total loss value, obtain the updated fog removal module, use the updated fog removal module as the fog removal module for the next iteration, and return to the original foggy image training module until a preset number of iterations is reached, and use the updated fog removal module obtained after the preset number of iterations as the target fog removal module.

[0019] The image sharpening result acquisition module is used to import the foggy image to be sharpened, and to perform image sharpening processing on the foggy image to be sharpened through the target fog removal module to obtain the foggy image sharpening result.

[0020] Based on the above-mentioned method for clarifying foggy images, the present invention also provides a foggy image clarification system.

[0021] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a foggy image sharpening system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the foggy image sharpening method as described above.

[0022] Based on the above-described method for enhancing images in foggy weather, this invention also provides a computer-readable storage medium.

[0023] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the foggy image sharpening method as described above.

[0024] The beneficial effects of this invention are as follows: A fog removal path, a defogging result restoration path, a fog synthesis path, and a synthesis result restoration path are constructed through a fog removal module, a fog synthesis module, and a color and texture restoration module. Based on the original foggy image, the fog removal path and the defogging result restoration path are trained to obtain a fog-removed image, a color and texture processed original image, a fog-removed synthesized image, and a defogging result restored image. Based on the original clear foggy image, the fog synthesis path and the synthesis result restoration path are trained to obtain a clear foggy synthesized image, a fog-removed clear image, and a synthesis result restored image. A discriminator then analyzes the original foggy image, the original clear foggy image, the fog-removed image, the color and texture processed original image, and the fog-removed synthesized image. The total loss value is obtained by analyzing the loss values ​​of the restored image after dehazing, the clear composite image after fog removal, the clear image after fog removal, and the restored image after composite. The parameters of the fog removal module are updated according to the total loss value to obtain the updated fog removal module. The updated fog removal module obtained after a preset number of iterations is used as the target fog removal module. The fog image is then sharpened by the target fog removal module to obtain the fog image sharpening result. This invention effectively solves the problems of difficulty in estimating based on prior knowledge and difficulty in obtaining real paired training datasets based on supervised learning. While effectively removing fog in various scenes, it also preserves good color and texture details and obtains dehazed images with richer image content and higher generation quality. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a method for sharpening foggy images according to an embodiment of the present invention;

[0026] Figure 2 This is a block diagram of a foggy image sharpening device provided in an embodiment of the present invention. Detailed Implementation

[0027] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0028] Figure 1 This is a flowchart illustrating a method for improving image clarity in foggy weather, as provided in an embodiment of the present invention.

[0029] like Figure 1 As shown, a method for sharpening images in foggy weather includes the following steps:

[0030] S1: Design a fog removal module, a fog synthesis module, a color and texture restoration module, and a discriminator, and construct a fog removal path, a defogging result restoration path, a fog synthesis path, and a synthesis result restoration path through the fog removal module, the fog synthesis module, and the color and texture restoration module;

[0031] S2: Import multiple original foggy images, and train the fog removal path and the defogging result restoration path according to each original foggy image to obtain the fog-removed image, the original image after color and texture processing, the fog-removed composite image, and the defogging result restoration image of each original foggy image.

[0032] S3: Import the original foggy clear image corresponding to each of the original foggy images, and train the fog synthesis path and the synthesis result recovery path according to each of the original foggy clear images to obtain the fog clear synthesized image, fog-removed clear image and synthesis result recovery image of each of the original foggy clear images;

[0033] S4: The discriminator analyzes the loss values ​​of all the original foggy images, all the original clear foggy images, all the fog-removed images, all the original images after color and texture processing, all the fog-removed composite images, all the defogging result restored images, all the foggy clear composite images, all the fog-removed clear images, and all the composite result restored images to obtain the total loss value;

[0034] S5: Update the parameters of the fog removal module according to the total loss value to obtain the updated fog removal module, and use the updated fog removal module as the fog removal module for the next iteration, and return to execute S2 until the preset number of iterations is reached, and use the updated fog removal module obtained after the preset number of iterations as the target fog removal module.

[0035] S6: Import the foggy image to be clarified, and perform image clarification processing on the foggy image to be clarified through the target fog removal module to obtain the foggy image clarification result.

[0036] Preferably, the preset number of iterations can be 12.

[0037] It should be understood that obtaining a large number of synthetic and real foggy images (i.e., the original foggy images) and clear images (i.e., the original foggy clear images) from online platforms and real environments as training samples for the model can improve the generalization ability of the model by acquiring rich data types. The acquisition of data does not need to consider the pairing of content, style, size and quantity between foggy images and clear images, thus reducing the difficulty of data collection.

[0038] Specifically, three modules are designed, including: the fog removal module. The fog synthesis module and color-texture restoration module (i.e., the color-texture restoration module) and the discriminator. The haze removal module, as the final module of the embedded device, is used to remove haze from foggy images; the haze synthesis module is used to synthesize realistic haze images from clear images; the color-texture restoration module (i.e., the color-texture restoration module) enhances and restores the haze removal result image. The discriminator is used to determine the authenticity of the results generated by the haze removal module and the haze synthesis module.

[0039] It should be understood that the multi-constraint criteria in the analysis invention utilize the output results of four paths (i.e., the haze-removed composite image, the dehaze-restored image, the haze-removed clear image, and the composite-restored image) to construct adversarial loss and cycle consistency loss.

[0040] It should be understood that in order to obtain the optimal fog removal module (i.e., the target haze removal module) must satisfy the following condition: after reaching a preset number of iterations (12 times), the last model is used as the latest model (i.e., the target haze removal module).

[0041] In the above embodiments, a fog removal path, a dehazing result restoration path, a fog synthesis path, and a synthesis result restoration path are constructed through a fog removal module, a fog synthesis module, and a color and texture restoration module. The fog removal path and the dehazing result restoration path are trained based on the original foggy image to obtain a fog-removed image, a color and texture processed original image, a fog-removed synthesized image, and a dehazing result restored image. Similarly, the fog synthesis path and the synthesis result restoration path are trained based on the original clear foggy image to obtain a clear foggy synthesized image, a clear fog-removed image, and a synthesis result restored image. A discriminator then analyzes the original foggy image, the original clear foggy image, the fog-removed image, the color and texture processed original image, the fog-removed synthesized image, and the dehazing result restored image. The total loss value is obtained by analyzing the loss values ​​of the restored fog image, the clear composite image, the clear image with fog removed, and the restored composite image. The parameters of the fog removal module are updated based on the total loss value to obtain the updated fog removal module. The updated fog removal module obtained after a preset number of iterations is used as the target fog removal module. The fog image is then sharpened by the target fog removal module to obtain the fog image sharpening result. This invention effectively solves the problems of difficulty in estimating based on prior knowledge and difficulty in obtaining real paired training datasets based on supervised learning. While effectively removing fog in various scenes, it also preserves good color and texture details, and obtains dehazed images with richer image content and higher generation quality.

[0042] Optionally, as an embodiment of the present invention, the fog removal path includes the fog removal module and the fog synthesis module. In step S2, the specific steps for training the fog removal path are as follows:

[0043] The fog removal module performs fog removal processing on each of the original foggy images to obtain fog-removed images of each of the original foggy images;

[0044] The fog synthesis module performs image synthesis processing on each of the fog-removed images to obtain fog-removed synthesized images of each of the original foggy images;

[0045] The dehazing result recovery path includes the haze removal module, the color and texture recovery module, and the haze synthesis module. In step S2, the specific steps for training the dehazing result recovery path are as follows:

[0046] The fog removal module performs fog removal processing on each of the original foggy images to obtain fog-removed images of each of the original foggy images;

[0047] Each of the haze removal images is processed to obtain a haze image derived from each of the original haze images.

[0048] The color and texture restoration module performs color and texture processing on the haze-removed image and the haze-derived image of each original foggy image to obtain the original image after color and texture processing of each original foggy image.

[0049] The fog synthesis module performs image synthesis processing on each of the original images after color and texture processing to obtain the defogging result restored image of each of the original foggy images;

[0050] The fog map derived image from the original foggy image includes a white balance image, a contrast-enhanced image, and a gamma-corrected image; specifically:

[0051] Based on the first formula, color and texture calculations are performed on the haze-removed image and the haze-derived image of each of the original foggy images to obtain the original image after color and texture processing of each of the original foggy images. The first formula is:

[0052] I fusion =W attention ·[I df ,I wb ,I ec ,I gm ],

[0053] Among them, I fusion The original image after color and texture processing, W attention I represents the weight values ​​for all original foggy images. df To remove haze from the image, I wb For white balance images, I ec For contrast-enhanced images, I gm Image for gamma correction.

[0054] It should be understood that the white balance image, the contrast enhancement image, and the gamma correction image are introduced as derived images of the foggy image, and an attention mechanism is used to fuse the dehazing result image (i.e., the fog removal image) and the foggy image derived image (i.e., the fog image derived image).

[0055] It should be understood that the haze synthesis module acts as a "bridge" connecting the haze removal module and the haze synthesis module. It takes the dehazing result image (i.e., the haze-removed image) output by the haze removal module and the derived images of the haze image (i.e., the white balance image, the contrast enhancement image, and the gamma correction image) as input. It assigns weights through an attention mechanism and then fuses all input images, paying more attention to the restoration effect of image color and texture details, and then feeds back to the haze removal module and the haze synthesis module.

[0056] It should be understood that the present invention uses the white balance image (I wb The contrast-enhanced image (I) ec ) and the gamma-corrected image (I gm (This is a derivative image of a fog image.)

[0057] Therefore, the output of the color-texture restoration module (i.e., the original image after color-texture processing) is expressed as follows:

[0058] I fusion =W attention ·[I df ,I wb ,I ec ,I gm (12)

[0059] In the formula, I df W represents the output result of the haze removal module (i.e., the haze-removed image). attention This represents the weight value acquired by the attention mechanism.

[0060] In the above embodiments, the fog removal path and the defogging result restoration path are trained using the original foggy image to obtain the fog-removed image, the original image after color and texture processing, the fog-removed composite image, and the defogging result restored image. This effectively solves the problems of difficulty in estimating based on prior knowledge and difficulty in obtaining real paired training datasets based on supervised learning. While effectively removing fog in various scenarios, it also preserves good color and texture details and obtains defogging images with richer image content and higher generation quality.

[0061] Optionally, as an embodiment of the present invention, the haze removal module includes an encoding block, a residual block, and a decoding block. The process of performing haze removal processing on each of the original haze images through the haze removal module to obtain haze-removed images of each of the original haze images includes:

[0062] The original foggy images are encoded using the coding blocks to obtain the encoded foggy features of each original foggy image. Specifically, the encoding of each original foggy image by the coding blocks is achieved through a second formula, which is:

[0063] h = F(x),

[0064] Where h is the encoded foggy feature, x is the original foggy image, and F(·) is the encoding function;

[0065] The residual features of each original foggy image are obtained by calculating the residual features of each encoded foggy feature using the residual blocks. Specifically, the calculation of the residual features of each encoded foggy feature by the residual blocks is achieved through a third equation, which is:

[0066] h′=R(h),

[0067] Where h′ is the residual feature, h is the foggy feature after encoding, and R(·) is the residual function;

[0068] The decoding block decodes each of the residual features to obtain the haze-removed images of each of the original foggy images; specifically, the decoding block decodes each of the residual features through a fourth equation, which is:

[0069] I df =G(h′),

[0070] Where h′ is the residual feature, G(·) is the decoding function, and I df Remove haze from the image.

[0071] It should be understood that the encoding block is used to encode and compress information from the original input image (i.e., the original foggy image); the residual block is used to prevent gradient vanishing while further refining the features of the output result of the encoding block; the decoding block is used to reconstruct the input from the encoding feature space to obtain a clear image after removing the fog (i.e., the fog-removed image); dense connections fully utilize contextual semantic features; skip connections fully utilize shallow encoding features.

[0072] It should be understood that the encoding stage (i.e., the encoding block) uses 3×3 convolution, ReLU activation function and max pooling as one feature extraction, and is performed twice; the feature refinement stage (i.e., the residual block) uses 6 residual blocks; the decoding stage (i.e., the decoding block) uses 3×3 transposed convolution and ReLU activation function as one feature reconstruction, and is performed twice.

[0073] Specifically, the design of the haze removal module includes the encoding block, the residual block, and the decoding block. The encoding block encodes and compresses the original input image (i.e., the original hazy image); the residual block prevents gradient vanishing caused by increasing network depth and refines the encoded features output by the encoding block; the decoding block reconstructs the input from the encoded feature space to obtain a clear image after haze removal (i.e., the haze-removed image). The formulas for the above three blocks are as follows:

[0074] h=F(x) (1)

[0075] h′=R(h) (2)

[0076]

[0077] In the formula, x represents the original input (i.e., the original foggy image), h, h′, and The encoding features (i.e., the encoded fog features), the residual features, and the decryption output (i.e., the fog-removed image) are respectively; and F(·), R(·), and G(·) are the encoding function, the residual function, and the decoding function, respectively.

[0078] It should be understood that, as the final module embedded in the dehazing device, it needs to maintain good color, contour, and texture details while removing haze from the image. Therefore, in designing this module (i.e., the haze removal module), in order to fully utilize the contextual semantic information in feature encoding and the shallow features rich in texture details and color information, this invention uses dense connections and skip connections.

[0079] In the above embodiments, the haze removal module processes the haze in the original foggy image to obtain a haze-removed image. This process can remove the haze from the image while maintaining good color, contour, and texture details. It can also make full use of the contextual semantic information in the feature encoding and the shallow features rich in texture details and color information.

[0080] Optionally, as an embodiment of the present invention, the haze removal image includes multiple haze removal pixels, and the process of performing image compositing processing on each of the haze removal images by the haze compositing module to obtain haze removal composite images of each of the original haze images includes:

[0081] The transmittance of each haze-removed image is obtained by analyzing the transmittance of all haze-removed pixels in each haze-removed image.

[0082] Using MATLAB, each of the haze-removed images was converted into a haze-sky grayscale image, and the grayscale value of each haze-removed pixel in each of the haze-sky grayscale images was obtained.

[0083] Based on the fifth formula, pixel segmentation is calculated for each fog-removed pixel in each fog-removed image according to the grayscale value of each fog-removed pixel in each fog-sky grayscale image, to obtain the segmented sky pixels of each fog-removed pixel in each fog-removed image. The fifth formula is:

[0084]

[0085] Where I(x) i )sky For the i-th haze pixel, after segmenting the sky, I(x) is the number of pixels after removing the haze pixel. i ) represents the pixel to be removed from the i-th fog, I(x) i ) gray Remove the grayscale value of the i-th fog pixel, where threshold is the binarization segmentation threshold;

[0086] The segmented sky images of each original foggy day image are obtained based on the segmented sky pixels corresponding to the fog removal pixels in each fog removal image.

[0087] Based on the sixth equation, global atmospheric light estimates are calculated for each of the segmented sky images to obtain the global atmospheric light estimates for each of the original foggy sky images. The sixth equation is:

[0088] A = mean(I(x)) sky ),

[0089] Where A is the global atmospheric light estimate, mean(·) is the average filter, and I(x) is the mean value. sky Image after sky segmentation;

[0090] Based on the seventh equation, haze-removed composite images are calculated according to the haze-removed images, global atmospheric light estimates, and transmittance of each of the original haze images, to obtain the haze-removed composite images of each of the original haze images. The seventh equation is:

[0091] J(x) = I(x)T + A(1-T),

[0092] Where J(x) is the haze-removed composite image, I(x) is the haze-removed image, T is the transmittance, and A is the global atmospheric light estimate.

[0093] It should be understood that the present invention introduces an atmospheric scattering model as a basis, and in order to synthesize more realistic fog images, the transmittance (T) and global atmospheric light value (A) (i.e., the global atmospheric light value estimate) are effectively estimated.

[0094] It should be understood that in order to obtain more specific global atmospheric light values ​​(i.e., the global atmospheric light estimate), it is necessary to segment the sky region and the non-sky region.

[0095] It should be understood that the global atmospheric light value (i.e., the global atmospheric light estimate) refers to the light source entering the imaging device at infinity, typically the sky region is considered to be at infinity.

[0096] It should be understood that, in order to synthesize realistic fog images, the fog synthesis module uses a method based on an atmospheric degradation model. The processes for obtaining the transmittance and global atmospheric light value (i.e., the global atmospheric light estimate) are as follows: Transmittance: The transmittance is obtained by stitching together the clear image and the minimum channel corresponding to each pixel, using multi-layer convolution and the Sigmoid activation function. Global atmospheric light value (i.e., the global atmospheric light estimate): The sky region and non-sky region are segmented using the Otsu algorithm, and the average intensity value of the sky region in the original image is used as the global atmospheric light estimate (i.e., the global atmospheric light estimate).

[0097] Specifically, this invention utilizes the Otsu binarization algorithm to cluster the input image and adaptively determines the binarization segmentation threshold. The image after sky segmentation (i.e., the pixels after sky segmentation) is represented as follows:

[0098]

[0099] In the formula, I(x) i I(x) represents the value of the i-th pixel in the input image. i ) gray I(x) represents i The grayscale value of the segmented sky region is used as the estimate of the global atmospheric light (A) (i.e., the global atmospheric light estimate), expressed as:

[0100] A = mean(I(x)) sky (5)

[0101] Where mean(·) represents the averaging filter. Therefore, the synthesis of the haze image can be represented as:

[0102] J(x)=I(x)T+A(1-T) (6)

[0103] In the formula, I(x) represents the input clear image (i.e. the haze-removed image), and J(x) represents the synthesized haze image (i.e. the haze-removed synthesized image).

[0104] In the above embodiments, by using the haze synthesis module to perform image synthesis processing on the haze-removed image to obtain a haze-removed synthesized image, more specific global atmospheric light values ​​can be obtained, thereby synthesizing a more realistic haze image.

[0105] Optionally, as an embodiment of the present invention, each of the haze removal pixels is equipped with multiple haze removal pixel values, and the process of analyzing the transmittance of each of the haze removal pixels to obtain the transmittance of each haze removal image includes:

[0106] The minimum pixel value is selected from the multiple fog removal pixel values ​​in each of the fog removal pixels to obtain the minimum pixel value of each fog removal pixel.

[0107] A minimum pixel value matrix is ​​constructed by removing the minimum pixel value of multiple fog-covered pixels from each of the original foggy images;

[0108] The minimum pixel value matrices and the multiple fog-removed pixels of the original fog images are stitched together to obtain the stitched fog image of the original fog images.

[0109] Based on multiple convolutional layers, feature extraction is performed on each of the stitched foggy images to obtain the stitched foggy features of each of the original foggy images.

[0110] The transmittance of each of the stitched fog features is obtained by normalizing each of the sigmoid activation function layers to obtain the fog-removed image.

[0111] Preferably, the number of pixel values ​​removed by the fog can be 3.

[0112] It should be understood that the clear, haze-free image (i.e., the haze-removed image) and the minimum channel corresponding to each pixel are stitched together, and the transmittance T is obtained by adaptive learning using multiple convolutional layers (i.e., the convolutional layers) and the Sigmoid activation function (i.e., the Sigmoid activation function layer).

[0113] In the above embodiments, by analyzing the transmittance of the removed pixels in the fog, a more specific global atmospheric light value can be obtained, thereby synthesizing a more realistic fog image.

[0114] Optionally, as an embodiment of the present invention, the haze removal image includes a plurality of haze removal pixels, each of the haze removal pixels being equipped with red channel pixel values, green channel pixel values, and blue channel pixel values. The process of performing derivation processing on each of the haze removal images to obtain haze image derivative images corresponding to each of the original haze images includes:

[0115] Each of the haze-removed images is subjected to contrast enhancement processing to obtain contrast-enhanced images of each of the original hazy images;

[0116] Each of the haze-removed images is subjected to gamma correction processing to obtain gamma-corrected images of each of the original haze images;

[0117] Import the image height and image width of each of the aforementioned haze removal images;

[0118] Based on Formula 8, white balance image processing is performed on each of the haze-removed images according to the image height, image width, multiple red channel pixel values, multiple green channel pixel values, and multiple blue channel pixel values ​​to obtain the corresponding white balance images of each of the original haze images. Formula 8 is:

[0119] in,

[0120] in,

[0121] in,

[0122] Specifically, I wb This is a white balance image, where I(R′) is the red channel matrix, I(G′) is the green channel matrix, I(B′) is the blue channel matrix, and k... r k is the gain coefficient for the red channel. g k is the gain coefficient for the green channel. b This represents the gain coefficient for the blue channel. The average of multiple red channel pixel values. It is the average of the pixel values ​​of multiple green channels. The value is the average of the pixel values ​​across multiple blue channels, where H is the image height and W is the image width. Let i' be the pixel value of the red channel. Let i' be the pixel value of the green channel. Let i be the pixel value of the i′-th blue channel.

[0123] Specifically, this invention introduces a white balance image (I) that restores latent colors in a scene, enabling the image to accurately reflect the color conditions of the real scene. wb ), and specifically contrast-enhanced images (I) that improve visibility in hazy areas. ec ) and gamma-corrected images (I gm (These are) derived images from fog images. They are defined as follows:

[0124]

[0125]

[0126] I wb =[I(R′),I(G′),I(B′)] (9)

[0127] in, and These represent the average values ​​of the three RGB channels; k r k g and k bThese represent the gain coefficients for the RGB three channels respectively; [·] represents the splicing function. I(R) is the red channel matrix.

[0128]

[0129] in, This represents the average brightness value of fog map I; μ is the linear gain factor, defined in this invention as...

[0130] I gm =αI γ (11)

[0131] In this invention, α = 1 and γ = 2.5 are set.

[0132] Specifically,

[0133] In the formula, H and W represent the height and width of the foggy image (i.e., the fog-removed image), respectively; and These represent the i-th pixel value of the R, G, and B channels respectively (i.e., the pixel value of the red channel, the pixel value of the green channel, or the pixel value of the blue channel).

[0134] In the above embodiments, by performing derivative processing on the haze removal image to obtain the haze image derivative image, the image accurately reflects the color conditions in the real scene, and pays more attention to the restoration effect of image color and texture details.

[0135] Optionally, as an embodiment of the present invention, the fog synthesis path includes the fog synthesis module and the fog removal module. In step S3, the specific steps for training the fog synthesis path are as follows:

[0136] The fog synthesis module performs image synthesis processing on each of the original foggy clear images to obtain foggy clear synthesized images of each of the original foggy images;

[0137] The fog removal module performs fog removal processing on each of the fog-clear composite images to obtain fog-removed clear images of each of the fog-clear composite images.

[0138] The synthesis result restoration path includes the haze synthesis module, the haze removal module, and the color texture restoration module. In step S3, the specific steps for training the synthesis result restoration path are as follows:

[0139] The fog synthesis module performs image synthesis processing on each of the original foggy clear images to obtain foggy clear synthesized images of each of the original foggy images;

[0140] The fog removal module performs fog removal processing on each of the original foggy day clear composite images to obtain fog-removed clear images of each original foggy day image.

[0141] Each of the aforementioned clear composite images of foggy weather is subjected to derivation processing to obtain fog map derived images corresponding to each of the aforementioned clear composite images of foggy weather;

[0142] The color and texture restoration module performs color and texture processing on the haze-removed images of each of the original foggy images and the haze-derived images of each of the clear foggy composite images to obtain the composite result restored images of each of the original foggy images.

[0143] In the above embodiments, fog-clear composite images, fog-removed clear images, and composite result restored images are obtained by training the fog synthesis path and the synthesis result restoration path on the original fog-clear image. This effectively solves the problems of difficulty in estimating based on prior knowledge and difficulty in obtaining real paired training datasets based on supervised learning. While effectively removing fog in various scenarios, it also preserves good color and texture details, and obtains dehazed images with richer image content and higher generation quality.

[0144] Optionally, as an embodiment of the present invention, the discriminator includes a first discriminator and a second discriminator, and the process of step S4 includes:

[0145] Based on the ninth equation and the second discriminator, the first adversarial loss value is calculated based on all original clear fog images and all fog-removed images. The ninth equation is as follows:

[0146]

[0147] in, y is the first adversarial loss value. i″ For the i″-th original foggy image, the original clear foggy image is... For the second discriminator, The image to remove the fog from the i″-th original foggy image, where N is the number of original foggy images;

[0148] Based on the tenth formula and the second discriminator, the second adversarial loss value is calculated according to all the original clear foggy images and all the original images after color and texture processing. The tenth formula is:

[0149]

[0150] in, For the second adversarial loss value, yi″ For the i″-th original foggy image, the original clear foggy image is... For the second discriminator, The original image after color and texture processing for the i″-th original foggy image, where N is the number of original foggy images;

[0151] Based on the eleventh equation and the first discriminator, the third adversarial loss value is calculated based on all original foggy images and all clear composite foggy images. The eleventh equation is as follows:

[0152]

[0153] in, x is the third adversarial loss value. i″ For the i″-th original foggy image, As the first discriminator, The clear foggy composite image is the i″th original foggy image, where N is the number of original foggy images;

[0154] Based on Equation Twelve, the first cyclic loss value is calculated from all original foggy images, all fog-removed composite images, and all defogging result restored images. Equation Twelve is as follows:

[0155]

[0156] in, x is the loss value for the first cycle. i″ For the i″-th original foggy image, The composite image is obtained by removing the haze from the i″-th original foggy image. The image is restored from the dehazing result of the i″-th original foggy image, where N is the number of original foggy images;

[0157] Based on Equation 13, the second-cycle loss value is calculated using all the original clear fogged images, all clear fogged images, and all synthesized restored images. Equation 13 is as follows:

[0158]

[0159] in, The second-cycle loss value, y i″ This is the original clear image of the i″-th original foggy image. Remove the clear haze from the i″-th original foggy image. To restore the image from the composite result of the i″-th original foggy image, where N is the number of original foggy images;

[0160] The total loss value is calculated based on the first adversarial loss value, the second adversarial loss value, the third adversarial loss value, the first cyclic loss value, and the second cyclic loss value.

[0161] It should be understood that the discriminator is key to achieving the conversion between different domains, and has similarity in distinguishing between real images and generating image feature representations.

[0162] It should be understood that in this invention, the hazy image discriminator (i.e., the first discriminator) and the clear image discriminator (i.e., the second discriminator) have the same structure, mainly consisting of 6 convolutional layers and a sigmoid activation function to obtain the evaluation value. Specifically, the number of convolutional filters in each layer from beginning to end are 64, 128, 256, 512, 512, and 1, respectively, with a stride of 2 for the first 4 layers and a stride of 1 for the last 2 layers. All convolutional kernels are 4×4 in size.

[0163] Specifically, the adversarial loss formula is expressed as:

[0164]

[0165]

[0166]

[0167] In the formula, and These represent the fog removal modules. Fog synthesis module and color-texture restoration module The adversarial losses (i.e., the first adversarial loss value, the second adversarial loss value, and the third adversarial loss value); and These represent the clear image discriminator (i.e., the second discriminator) and the hazy image discriminator (i.e., the first discriminator), respectively; N is the number of training samples.

[0168] It should be understood that the cyclic uniformity loss of the present invention can be expressed as:

[0169]

[0170]

[0171] In the above embodiments, the total loss value is obtained by analyzing the loss values ​​of the original foggy image, the original clear foggy image, the fog-removed image, the original image after color and texture processing, the fog-removed composite image, the defogging result restored image, the fog-clear composite image, the fog-removed clear image, and the composite result restored image through a discriminator. While effectively removing fog in various scenes, it also preserves better color and texture details, and obtains a defogging image with richer image content and higher generation quality.

[0172] Optionally, as an embodiment of the present invention, the process of calculating the total loss value based on the first adversarial loss value, the second adversarial loss value, the third adversarial loss value, the first cyclic loss value, and the second cyclic loss value to obtain the total loss value includes:

[0173] Based on the fourteenth formula, the total loss value is calculated according to the first adversarial loss value, the second adversarial loss value, the third adversarial loss value, the first cyclic loss value, and the second cyclic loss value. The fourteenth formula is as follows:

[0174]

[0175] in, This is the total loss value. The first adversarial loss value, This is the second adversarial loss value. This is the third adversarial loss value. This is the loss value for the first cycle. The second cycle loss value is represented by λ1, λ2, λ3, λ4, and λ5, which are all loss balancing weights.

[0176] Preferably, λ1 = λ2 = λ3 = 10 and λ4 = λ5 = 5.

[0177] It should be understood that the total loss of this invention is defined as:

[0178]

[0179] In the formula, λ1, λ2, λ3, λ4 and λ5 are used as loss balancing weights, and are set to λ1=λ2=λ3=10 and λ4=λ5=5 respectively based on experimental experience.

[0180] In the above embodiments, the image sharpening model is obtained by analyzing the total loss value of the adversarial loss value and the cyclic loss value. This model removes the fog in the real scene while retaining good color and texture effects, effectively solving the problems of difficulty in estimating based on prior knowledge and difficulty in obtaining real paired training datasets based on supervised learning.

[0181] Optionally, as another embodiment of the present invention, four closed-loop paths are constructed by designing three modules and two discriminators. A multi-constraint criterion strategy is used to constrain three models within these four paths, achieving multiple transformations between the real haze image domain and the clear image domain, resulting in the final dehazing result of the real haze image. This invention effectively solves the problems of difficulty in estimating based on prior knowledge and difficulty in obtaining real paired training datasets based on supervised learning. It effectively removes haze in various scenes while preserving good color and texture details, thus improving the visual effect of the image.

[0182] Optionally, as another embodiment of the present invention, four closed-loop paths are established based on three modules, including: a haze removal path. Color and texture restoration path of dehazing results Fog synthesis path and the color and texture recovery path of the synthesized fog result

[0183] The path for haze removal is represented as follows:

[0184]

[0185] The color and texture restoration path of the dehazing result is represented as follows:

[0186]

[0187] The fog synthesis path is represented as follows:

[0188]

[0189] The color and texture recovery path of the synthesized fog result is represented as follows:

[0190]

[0191] In the four equations above, x represents the input fog image in the fog image domain (i.e., the original foggy image), I x y represents the derived image of the input fog image x (including the white balance image, contrast-enhanced image, and gamma-corrected image); y represents the input fog-free image in the clear image domain (i.e., the original foggy clear image), I s_y This represents a derived image of the synthesized fog map.

[0192] By utilizing the established structure of four closed-loop paths, multiple constraint criteria are designed to enable the three modules to learn from each other through game theory, achieving a mapping between the hazy image domain and the clear image domain. Finally, the trained haze removal module is embedded into a hazy image sharpening device to remove haze from real-world scenes. The constraint process is as follows:

[0193] 1) As shown in formula (20), the input fog map x is processed by the fog removal module. A haze-removed image is generated, and this generated image is evaluated using a clear image discriminator before being processed by the haze synthesis module. A synthetic fog map is obtained, and constraints are constructed between the synthetic fog map and the input fog map. This forms a closed loop of fog map → defogging map → synthetic fog map.

[0194] 2) As shown in formula (21), the generated haze-removed image and haze-derived image are subjected to color-texture restoration. An enhanced dehazed image is generated, and this generated image is also evaluated using a clear image discriminator. The haze synthesis module is then used again to further refine the image. A synthetic fog map is obtained, and constraints are constructed between the synthetic fog map and the input fog map. This forms a closed loop of fog map → defogging map → enhanced defogging map → synthetic fog map.

[0195] 3) As shown in formula (22), the input fog-free image y passes through the module A composite haze image is generated, which is then analyzed using a haze image discriminator, and subsequently processed by a module. Generate a dehazed image, and construct constraints between the generated dehazed image and the input haze-free image. This forms a closed loop of haze-free image → synthesized haze image → dehazed image.

[0196] As shown in formula (23), the derived images of the generated dehazed image and the synthesized hazy image are processed by the module. An enhanced dehazed image is generated, and this enhanced dehazed image is used to construct constraints with the input hazy image. This forms a closed loop: hazy image → synthesized hazy image → dehazed image → enhanced dehazed image.

[0197] Optionally, as another embodiment of the present invention, the fog image discriminator (i.e., the first discriminator) and the clear image discriminator (i.e., the second discriminator) have the same structure and can also be composed of 6 convolutional layers and a series of BN normalization and LeakyReLU activation functions.

[0198] Optionally, as another embodiment of the present invention, the unsupervised dehazing method based on domain transformation constructs four closed-loop paths through three modules. Under a multi-constraint criterion strategy, unsupervised learning of the model is achieved, enabling multiple transformations between the hazy image domain and the clear image domain. Ultimately, this removes haze from the real scene while preserving good color and texture effects. Furthermore, the dehazing device only needs to retain one haze removal module to achieve clear hazy images, greatly reducing storage and computational costs and improving the system's processing speed.

[0199] Optionally, as another embodiment of the present invention, the present invention provides an unsupervised foggy image sharpening device based on domain transformation, comprising: a model training unit for training the fog removal module, fog synthesis module, and color-texture restoration module designed in the present invention; a model storage unit for storing only the pre-trained weights and biases in the optimal fog removal module; and an image display unit for displaying the fog image to be processed and the restored image after processing the fog image using the fog removal module in the storage unit.

[0200] Figure 2 This is a block diagram of a foggy image sharpening device provided in an embodiment of the present invention.

[0201] Alternatively, as another embodiment of the present invention, such as Figure 2 As shown, a device for improving image clarity in foggy weather includes:

[0202] The path construction module is used to design the fog removal module, fog synthesis module, color and texture restoration module, and discriminator, and to construct the fog removal path, the fog result restoration path, the fog synthesis path, and the synthesis result restoration path through the fog removal module, the fog synthesis module, and the color and texture restoration module.

[0203] The original foggy image training module is used to import multiple original foggy images, and train the fog removal path and the defogging result restoration path according to each original foggy image to obtain the fog-removed image, the original image after color and texture processing, the fog-removed composite image, and the defogging result restoration image of each original foggy image.

[0204] The fog clear image training module is used to import the original fog clear images corresponding to each of the original fog images, and to train the fog synthesis path and the synthesis result recovery path according to each of the original fog clear images, so as to obtain the fog clear synthesized image, fog-removed clear image and synthesis result recovery image of each of the original fog clear images;

[0205] The loss value analysis module is used to analyze the loss values ​​of all the original foggy images, all the original clear foggy images, all the fog-removed images, all the original images after color and texture processing, all the fog-removed composite images, all the defogging result restored images, all the foggy clear composite images, all the fog-removed clear images, and all the composite result restored images through the discriminator, and obtain the total loss value;

[0206] The parameter update module is used to update the parameters of the fog removal module according to the total loss value, obtain the updated fog removal module, use the updated fog removal module as the fog removal module for the next iteration, and return to the original foggy image training module until a preset number of iterations is reached, and use the updated fog removal module obtained after the preset number of iterations as the target fog removal module.

[0207] The image sharpening result acquisition module is used to import the foggy image to be sharpened, and to perform image sharpening processing on the foggy image to be sharpened through the target fog removal module to obtain the foggy image sharpening result.

[0208] Optionally, another embodiment of the present invention provides a foggy image sharpening system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the foggy image sharpening method described above. This system can be a computer or similar system.

[0209] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the foggy image sharpening method described above.

[0210] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0211] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0212] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0213] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0214] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0215] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0216] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for defogging an image, characterized by, Comprising the following steps: S1: design a haze removal module, a haze synthesis module, a color texture recovery module and a discriminator, and build a haze removal path, a haze result recovery path, a haze synthesis path and a synthesis result recovery path through the haze removal module, the haze synthesis module and the color texture recovery module; S2: import a plurality of original foggy images, train the haze removal path and the haze result recovery path according to each of the original foggy images respectively, obtain the haze removal image, the color texture processed original image, the haze removal synthesis image and the haze result recovery image of each of the original foggy images; S3: import the original clear foggy image corresponding to each of the original foggy images, train the haze synthesis path and the synthesis result recovery path according to each of the original clear foggy images respectively, obtain the clear foggy synthesis image, the haze removal clear image and the synthesis result recovery image of each of the original clear foggy images; S4: analyze the loss value of all the original foggy images, all the original clear foggy images, all the haze removal images, all the color texture processed original images, all the haze removal synthesis images, all the haze result recovery images, all the clear foggy synthesis images, all the haze removal clear images and all the synthesis result recovery images through the discriminator, and obtain the total loss value; S5: update the parameters of the haze removal module according to the total loss value, obtain the updated haze removal module, and use the updated haze removal module as the haze removal module for the next iteration, and return to execute S2 until the preset iteration number is reached, and use the updated haze removal module obtained after the preset iteration number as the target haze removal module; S6: import the to-be-clear foggy image, perform image clearification processing on the to-be-clear foggy image through the target haze removal module, and obtain the foggy image clearification result; The haze removal path comprises the haze removal module and the haze synthesis module, and in S2, the specific steps of training the haze removal path are: The haze removal module is used to remove the haze of each of the original foggy images to obtain the haze removal image of each of the original foggy images; The haze synthesis module is used to perform image synthesis processing on each of the haze removal images to obtain the haze removal synthesis image of each of the original foggy images; The haze result recovery path comprises the haze removal module, the color texture recovery module and the haze synthesis module, and in S2, the specific steps of training the haze result recovery path are: The haze removal module is used to remove the haze of each of the original foggy images to obtain the haze removal image of each of the original foggy images; Each of the haze removal images is derived to obtain the fog image derived image of each of the original foggy images; The color texture recovery module is configured to perform color texture processing on the haze-removed image and the haze image derived image of each original foggy image respectively, to obtain a color texture processed original image of each original foggy image; The haze synthesis module is configured to perform image synthesis processing on each color texture processed original image respectively, to obtain a haze-removed result recovered image of each original foggy image; The haze image derived image of the original foggy image includes a white balance image, a contrast enhanced image, and a gamma correction image; specifically, Based on the first formula, the color texture of the haze-removed image and the haze image derived image of each original foggy image is calculated, to obtain a color texture processed original image of each original foggy image, the first formula is: ， wherein, is the original image after color texture processing, is the weight value of all original foggy images, is the haze removal image, is the white balance image, is the contrast enhancement image, is the gamma correction image.

2. The method according to claim 1, wherein The haze removal module includes an encoding block, a residual block, and a decoding block, and the process of removing the haze from each original foggy image by the haze removal module to obtain a haze-removed image of each original foggy image includes: The encoding block encodes each original foggy image respectively to obtain an encoded foggy feature of each original foggy image; specifically, the encoding of each original foggy image by the encoding block is implemented by a second formula, the second formula is: ， wherein, is an encoded foggy feature, is an original foggy image, is an encoding function; The residual block calculates the residual feature of each encoded foggy feature respectively to obtain a residual feature of each original foggy image; specifically, the calculation of the residual feature of each encoded foggy feature by the residual block is implemented by a third formula, the third formula is: ， wherein, is a residual feature, is an encoded haze feature, is a residual function; The decoding block decodes each residual feature respectively to obtain a haze-removed image of each original foggy image; specifically, the decoding of each residual feature by the decoding block is implemented by a fourth formula, the fourth formula is: ， wherein, is a residual feature, is a decoding function, is a fog removed image.

3. The method of claim 1, wherein The haze-removed image includes a plurality of haze-removed pixel points, and the process of performing image synthesis processing on each haze-removed image by the haze synthesis module to obtain a haze-removed synthesis image of each original foggy image includes: The transmittance of all haze-removed pixel points in each haze-removed image is analyzed respectively to obtain the transmittance of each haze-removed image; Each haze-removed image is converted into a foggy gray image by using a MATLAB tool, and the gray value of each haze-removed pixel point in each foggy gray image is obtained; Based on a fifth formula, the calculation of pixel point segmentation of each haze-removed pixel point in each haze-removed image is performed according to the gray value of each haze-removed pixel point in each foggy gray image, to obtain a segmented sky after pixel point of each haze-removed pixel point in each haze-removed image, the fifth formula is: ， wherein, is the number of the sky pixel point removed by the fog removal pixel point, is the number of the sky pixel point removed by the fog removal pixel point, is the number of the sky pixel point removed by the fog removal pixel point, is the number of the sky pixel point removed by the fog removal pixel point, is the number of the sky pixel point removed by the fog removal pixel point, is the number of the sky pixel point removed by the fog removal pixel point, is the binary segmentation threshold value; The segmented sky after image of each original foggy image is obtained according to the segmented sky after pixel point corresponding to each haze-removed pixel point in each haze-removed image; Based on the sixth formula, the global atmospheric light estimation of each of the original foggy images is obtained by calculating the global atmospheric light estimation according to each of the segmented skyless images, and the sixth formula is: ， wherein, is a global atmospheric light estimate, is an average filter, is a segmented sky image; Based on the seventh formula, the haze removal composite image of each of the original foggy images is obtained by calculating the haze removal composite image according to the haze removal image, the global atmospheric light estimation and the transmittance of each of the original foggy images, and the seventh formula is: ， wherein, is a haze-removed composite image, is a haze-removed image, is a transmittance, is a global airlight estimate.

4. The method of claim 3, wherein, Each of the haze removal pixel points is provided with a plurality of haze removal pixel values, and the process of analyzing the transmittance of each of the haze removal pixel points includes: The minimum pixel value of each of the haze removal pixel points is obtained by screening the plurality of haze removal pixel values in each of the haze removal pixel points; A minimum pixel value matrix is constructed by the minimum pixel values of the plurality of haze removal pixel points of each of the original foggy images; Each of the minimum pixel value matrix and the plurality of haze removal pixel points of each of the original foggy images is spliced to obtain a spliced foggy image of each of the original foggy images; Based on a plurality of convolution layers, the feature extraction of each of the spliced foggy images is performed to obtain a spliced foggy feature of each of the original foggy images; Based on a Sigmoid activation function layer, the normalization processing of each of the spliced foggy features is performed to obtain the transmittance of each of the haze removal images.

5. The method of claim 1, wherein The haze removal image includes a plurality of haze removal pixel points, each of which is provided with a red channel pixel value, a green channel pixel value and a blue channel pixel value, and the process of deriving each of the haze removal images includes: The contrast enhancement of each of the haze removal images is performed to obtain a contrast enhancement image of each of the original foggy images; The gamma correction of each of the haze removal images is performed to obtain a gamma correction image of each of the original foggy images; The image height and the image width of each of the haze removal images are imported; Based on the eighth formula, the white balance image of each of the original foggy images is obtained by performing white balance image processing on the image height, the image width, a plurality of red channel pixel values, a plurality of green channel pixel values and a plurality of blue channel pixel values of each of the haze removal images, and the eighth formula is: ， wherein , wherein , wherein , In particular, is a white balanced image, is a red channel matrix, is a green channel matrix, is a blue channel matrix, is a red channel gain coefficient, is a green channel gain coefficient, is a blue channel gain coefficient, is an average of a plurality of red channel pixel values, is an average of a plurality of green channel pixel values, is an average of a plurality of blue channel pixel values, is an image height, is an image width, is an i th red channel pixel value, is an i th green channel pixel value, is an i th blue channel pixel value.

6. The method of claim 1, wherein, The haze synthesis path includes the haze synthesis module and the haze removal module, and the specific steps of training the haze synthesis path in the S3 are: The haze synthesis module is used to perform image synthesis processing on each of the original clear foggy images to obtain a clear foggy composite image of each of the original foggy images; The haze removal module is used to perform haze removal processing on each of the clear foggy composite images to obtain a haze removal clear image of each of the clear foggy composite images; The synthesis result recovery path comprises the haze synthesis module, the haze removal module and the color texture recovery module, and the specific steps for training the synthesis result recovery path in the S3 are: The haze synthesis module is used for performing image synthesis processing on each original clear image of the foggy day to obtain a clear synthesis image of each original foggy day image; The haze removal module is used for performing haze removal processing on each clear synthesis image of the foggy day to obtain a haze-removed clear image of each original foggy day image; Derivation processing is performed on each clear synthesis image of the foggy day to obtain a haze map derived image corresponding to each clear synthesis image of the foggy day; The color texture recovery module is used for performing color texture processing on each haze-removed image of the original foggy day and each haze map derived image of the clear synthesis image of the foggy day to obtain a synthesis result recovery image of each original foggy day image.

7. The method of claim 1, wherein The discriminator comprises a first discriminator and a second discriminator, and the process of the step S4 comprises: Based on the ninth formula and the second discriminator, a first adversarial loss value is calculated according to all original clear images of the foggy day and all haze-removed images, to obtain the first adversarial loss value, the ninth formula is: ， in, The first adversarial loss value, For the first The original foggy image with clear image, For the second discriminator, For the first Image with haze removed from an original foggy image. The number of original foggy images; Based on the tenth formula and the second discriminator, a second adversarial loss value is calculated according to all original clear images of the foggy day and all color texture-processed original images, to obtain the second adversarial loss value, the tenth formula is: ， wherein, is a second adversarial loss value, is a first original clear image of the i-th original foggy image, is a second discriminator, is a first color-texture processed original image of the i-th original foggy image, is a number of original foggy images; Based on the eleventh formula and the first discriminator, a third adversarial loss value is calculated according to all original foggy day images and all clear synthesis images of the foggy day, to obtain the third adversarial loss value, the eleventh formula is: ， wherein, is a third adversarial loss value, is a first original foggy image, is a first discriminator, is a second foggy clear synthetic image of the is a number of original foggy images; Based on the twelfth formula, a first cycle loss value is calculated according to all original foggy day images, all haze-removed synthesis images and all haze-removed result recovery images, to obtain the first cycle loss value, the twelfth formula is: ， wherein, is a first cycle loss value, is a first original foggy image, is a first fog-removed composite image of the original foggy image, is a first dehazing result recovery image of the original foggy image, is a number of original foggy images; Based on the thirteenth formula, a second cycle loss value is calculated according to all original clear images of the foggy day, all haze-removed clear images and all synthesis result recovery images, to obtain the second cycle loss value, the thirteenth formula is: ， in, This is the loss value for the second cycle. For the first The original foggy image corresponds to the original clear foggy image. For the first The haze was removed from the original foggy image to create a clearer image. For the first The synthesized image is a restored image from the original foggy image. The number of original foggy images; A total loss value is calculated according to the first adversarial loss value, the second adversarial loss value, the third adversarial loss value, the first cycle loss value and the second cycle loss value, to obtain the total loss value.

8. The method of claim 7, wherein, The process of calculating the total loss value according to the first adversarial loss value, the second adversarial loss value, the third adversarial loss value, the first cycle loss value and the second cycle loss value to obtain the total loss value comprises: Based on the fourteenth formula, the total loss value is calculated according to the first adversarial loss value, the second adversarial loss value, the third adversarial loss value, the first cycle loss value and the second cycle loss value, to obtain the total loss value, the fourteenth formula is: ， wherein, is a total loss value, is a first adversarial loss value, is a second adversarial loss value, is a third adversarial loss value, is a first cycle loss value, is a second cycle loss value, , , , and are loss balancing weights.

9. A haze image sharpening apparatus, characterized by comprising: It comprises: a path construction module configured to design a haze removal module, a haze synthesis module, a color texture recovery module, and a discriminator, and to construct a haze removal path, a haze result recovery path, a haze synthesis path, and a synthesis result recovery path through the haze removal module, the haze synthesis module, and the color texture recovery module; a raw haze image training module configured to import a plurality of raw haze images, train the haze removal path and the haze result recovery path according to each of the raw haze images, and obtain a haze removal image, a color texture processed raw image, a haze removal synthesis image, and a haze result recovery image of each of the raw haze images; a haze clear image training module configured to import a raw haze clear image corresponding to each of the raw haze images, train the haze synthesis path and the synthesis result recovery path according to each of the raw haze clear images, and obtain a haze clear synthesis image, a haze removal clear image, and a synthesis result recovery image of each of the raw haze clear images; a loss value analysis module configured to analyze loss values of all of the raw haze images, all of the raw haze clear images, all of the haze removal images, all of the color texture processed raw images, all of the haze removal synthesis images, all of the haze result recovery images, all of the haze clear synthesis images, all of the haze removal clear images, and all of the synthesis result recovery images through the discriminator, and obtain a total loss value; a parameter update module configured to update parameters of the haze removal module according to the total loss value, obtain an updated haze removal module, use the updated haze removal module as the haze removal module for next iteration, and return to the raw haze image training module until a preset number of iterations is reached, and use the updated haze removal module obtained after the preset number of iterations as a target haze removal module; a clear result obtaining module configured to import a haze image to be clarified, perform image clarification processing on the haze image to be clarified through the target haze removal module, and obtain a haze image clarification result. The haze removal path includes the haze removal module and the haze synthesis module. In the raw haze image training module, the specific steps of training the haze removal path are as follows: perform haze removal processing on each of the raw haze images through the haze removal module to obtain a haze removal image of each of the raw haze images; perform image synthesis processing on each of the haze removal images through the haze synthesis module to obtain a haze removal synthesis image of each of the raw haze images; The haze result recovery path includes the haze removal module, the color texture recovery module, and the haze synthesis module. In the raw haze image training module, the specific steps of training the haze result recovery path are as follows: The haze removal module removes the haze from each of the original foggy images to obtain a haze-removed image of each of the original foggy images; The derivative processing module performs derivative processing on each of the haze-removed images to obtain a haze map derivative image of each of the original foggy images; The color texture recovery module performs color texture processing on each of the haze-removed images and the haze map derivative image of each of the original foggy images to obtain a color-texture-processed original image of each of the original foggy images; The haze synthesis module performs image synthesis processing on each of the color-texture-processed original images to obtain a haze-removed result recovery image of each of the original foggy images; The haze map derivative image of each of the original foggy images includes a white balance image, a contrast enhancement image, and a gamma correction image; specifically, Based on a first formula, the color texture of each of the haze-removed images and the haze map derivative image of each of the original foggy images is calculated to obtain a color-texture-processed original image of each of the original foggy images, and the first formula is: ， wherein, is a color texture processed original image, is a weight value of all original foggy images, is a fog removal image, is a white balance image, is a contrast enhancement image, is a gamma correction image.

Images