Method and system for defogging foggy images by filling in information

Abstract

The present invention relates to a method and system for defogging information of foggy images. The method comprises: obtaining a foggy image for image preprocessing; performing a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data; performing grayscale gamma transformation on the intermediate image data, adjusting the gamma value of the intermediate image data, and performing contrast adjustment and a second image enhancement to obtain image data to be filled; using a coloring model to fill missing information on the image data to be filled, and outputting a defogged image. By performing two image enhancements on the collected foggy image, and the second image enhancement is combined with a coloring model to fill missing information, the missing information of the original foggy image is largely restored, which is convenient for computer analysis and subjective observation of the human eye, and can effectively solve the defogging operation of the foggy image, so that the image information is more realistically displayed, and the defogging effect of the image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for defogging information of a foggy image. Background Art

[0002] With the development of society, people's living standards are constantly improving, the ecological environment is also gradually changing, and haze and other weather conditions often appear. Images taken in haze and other weather conditions are foggy images and are not clear enough. So far, defogging has been widely used in various fields. The traditional defogging method is often through the video defogging framework MAP-Net. The memory-based physical prior guidance module of this framework is designed to enhance brightness recovery. This module encodes fog prior related features as remote memory. There is also a multi-distance scene cycle brightness restoration module based on spatiotemporal deformable attention and multi-distance guided aggregation. This module can effectively capture long-distance temporal fog and scene clues from adjacent frames, and a multi-distance scene cycle brightness restoration module based on spatiotemporal deformable attention and multi-distance guided aggregation is proposed. This module can effectively capture long-distance temporal fog and scene clues from adjacent frames. With the help of this network, foggy images can be restored to a certain extent. Someone proposed a real image dehazing network based on high-quality codebook prior (RIDCP). VQGAN was pre-trained on a large-scale high-quality dataset to obtain a discrete codebook and encapsulate high-quality priors. After the decoder uses HQPs to replace the negative effects of haze, it is equipped with a new normalized feature alignment module, which can effectively utilize high-quality features and produce clean results. Someone proposed an end-to-end system DehazeNet for medium transmission estimation. The system takes hazy images as input and outputs their medium projection map. Then, the haze-free image is restored through the atmospheric scattering model (ASM). At the same time, the innovation of this system is that the removal of haze in the image is not to identify the whole, but to train blocks according to the uneven distribution of fog in the image. The transmission function in the output image block ASM is different to restore the target image. Someone else proposed an end-to-end feature fusion attention network (FFA-Net). Considering that different communication features contain completely different weighted information and the uneven distribution of haze on different images, a new feature attention (FA) module is proposed to merge channel attention with pixel attention mechanism. Different features and pixels are treated unequally, which provides additional flexibility for processing different types of information and expands the capabilities of convolutional neural networks (CNNs). The experimental conclusion is that this method surpasses the most advanced single image dehazing methods in quantitative and qualitative aspects.

[0003] However, the above-mentioned image dehazing methods are almost all about IET and IRT for hazy images, which have the problems of severe image distortion and poor dehazing effect. Summary of the invention

[0004] Based on this, in order to solve the above technical problems, a method and system for defogging a foggy image by filling in information is provided, which can improve the defogging effect of the image.

[0005] A method for defogging a foggy image by filling in information, the method comprising:

[0006] Acquire a foggy image, and perform image preprocessing on the foggy image to obtain processed image data;

[0007] Performing a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data;

[0008] Performing grayscale gamma transformation on the intermediate image data, adjusting the gamma value of the intermediate image data, and performing contrast adjustment and second image enhancement to obtain image data to be filled;

[0009] The shading model is used to fill in missing information of the image data to be filled, and a defogging image is output.

[0010] In one embodiment, performing a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data includes:

[0011] Performing color space adjustment on the processed image data based on the ACE algorithm to obtain adjusted image data;

[0012] Dynamic tone scaling is performed on the adjusted image data to obtain intermediate image data.

[0013] In one embodiment, the color space adjustment of the processed image data based on the ACE algorithm to obtain the adjusted image data includes:

[0014] Calculating output pixel values ​​for each channel in the processed image data based on the ACE algorithm;

[0015] Color comparison is performed according to the output pixel values, and local or global balancing processing is performed, and an image subset is selected to obtain adjusted image data.

[0016] In one embodiment, the step of performing dynamic tone scaling on the adjusted image data to obtain intermediate image data includes:

[0017] Using the adjusted image data as an intermediate pixel matrix;

[0018] The intermediate pixel matrix is ​​scaled using a linear scaling method to obtain intermediate image data.

[0019] In one embodiment, performing grayscale gamma transformation on the intermediate image data to adjust the gamma value of the intermediate image data includes:

[0020] Determining an optimal gamma value according to the grayscale value of the intermediate image data;

[0021] The gamma value in the intermediate image data is adjusted to the optimal gamma value through grayscale gamma transformation.

[0022] In one embodiment, the step of using a shading model to fill missing information in the image data to be filled includes:

[0023] Extract image features in the image data to be filled using a backbone network in the coloring model;

[0024] Inputting the image features into a pixel decoder to restore the image spatial structure;

[0025] Performing a color query on the visual features in the image data to be filled by a color decoder, and outputting a color representation;

[0026] The image space structure and the color representation are integrated to generate a color channel output coloring result and fill in the missing information.

[0027] In one embodiment, the method further comprises:

[0028] Inputting the image features and color query instructions into a color decoding block in the shading model;

[0029] The correlation between semantic and color representations is established through the cross-attention mechanism, self-attention mechanism and feed-forward operation adopted in the colorization model.

[0030] In one embodiment, the method further comprises:

[0031] The coloring model colors the image data to be filled in an end-to-end manner.

[0032] A system for filling and defogging foggy images, the system comprising:

[0033] An image acquisition module is used to acquire a foggy image and perform image preprocessing on the foggy image to obtain processed image data;

[0034] A first image enhancement module, used for performing a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data;

[0035] A second image enhancement module, used for performing grayscale gamma transformation on the intermediate image data, adjusting the gamma value of the intermediate image data, and performing contrast adjustment and second image enhancement to obtain image data to be filled;

[0036] The information filling module is used to fill in the missing information of the image data to be filled using a shading model and output a defogging image.

[0037] The above-mentioned information filling and defogging method and system for foggy images perform two image enhancements on the collected foggy images, and the second image enhancement is combined with a coloring model to fill in the missing information, so that the missing information of the original foggy image is largely restored, which is convenient for computer analysis and subjective observation by the human eye. It can effectively solve the defogging operation of foggy images, make the image information more realistic, and improve the defogging effect of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A diagram of an application environment of a method for defogging a foggy image using information filling in an embodiment;

[0039] Figure 2 A schematic diagram of a flow chart of a method for defogging a foggy image by filling in information in an embodiment;

[0040] Figure 3 A schematic diagram of a process of performing image processing using the ACE algorithm in one embodiment;

[0041] Figure 4 is a schematic diagram of function curves of different gamma values ​​in one embodiment;

[0042] Figure 5 A schematic diagram of different effects of gamma values ​​in one embodiment;

[0043] Figure 6 A schematic diagram of a shading model principle diagram in one embodiment;

[0044] Figure 7 A comparison diagram before and after defogging using a defogging method for a foggy image using information filling in an embodiment;

[0045] Figure 8 A structural block diagram of an information filling and defogging system for a foggy image in one embodiment;

[0046] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The information filling and defogging method for foggy images provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown. Figure 1 As shown, the application environment includes a computer device 110. The computer device 110 can obtain a foggy image and perform image preprocessing on the foggy image to obtain processed image data; the computer device 110 can perform a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data; the computer device 110 can perform a grayscale gamma transformation on the intermediate image data, adjust the gamma value of the intermediate image data, and perform contrast adjustment and a second image enhancement to obtain image data to be filled; the computer device 110 can use a shading model to fill missing information in the image data to be filled, and output a defogging image. The computer device 110 can be, but is not limited to, various personal computers, laptops, smart phones, robots, unmanned aerial vehicles, tablet computers and other devices.

[0049] In one embodiment, Figure 2 As shown, a method for defogging a foggy image by filling in information is provided, comprising the following steps:

[0050] Step 202: Acquire a foggy image and perform image preprocessing on the foggy image to obtain processed image data.

[0051] The computer device can obtain a foggy image through an image acquisition device, wherein the foggy image can be a foggy image at sea. Then, the computer device can perform image preprocessing operations on the foggy image. Specifically, the preprocessing operation can be image processing such as denoising and edge detection. The computer device can perform denoising on the foggy image to eliminate noise in the image; secondly, it is also necessary to perform edge detection on the foggy image to obtain edge information of the target in the foggy image; finally, it is also necessary to perform scale normalization on the foggy image so that subsequent processing can be better performed.

[0052] Step 204: Perform a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data.

[0053] Among them, the ACE algorithm is an unsupervised global and local color correction algorithm. The ACE algorithm performs the first image enhancement processing on the processed image data to obtain color intermediate image data.

[0054] Step 206 , performing grayscale gamma transformation on the intermediate image data, adjusting the gamma value of the intermediate image data, and performing contrast adjustment and second image enhancement to obtain image data to be filled.

[0055] The sensitivity of the human eye to external light sources is not linearly related to the input light intensity, but exponentially related. Under low illumination, the human eye can more easily distinguish changes in brightness. As the illumination increases, the human eye cannot easily distinguish changes in brightness, while the camera sensitivity is linearly related to the input light intensity. In order to more effectively save image brightness information, in this embodiment, the computer device can perform a grayscale gamma transformation on the image after the first image enhancement process. Among them, gamma correction is used in 8-bit RGB images, which can be used to solve the problem of saving as much color content that humans are sensitive to as possible in a limited storage space. The computer device can obtain the image data to be filled by adjusting the gamma value of the intermediate image data, performing contrast adjustment, and performing a second image enhancement.

[0056] Step 208: Use the shading model to fill in missing information of the image data to be filled, and output a dehazed image.

[0057] The shading model may be a DDColor shading model having a dual decoder for image shading.

[0058] In this embodiment, the collected foggy image is enhanced twice, and the second image enhancement is combined with a coloring model to fill in the missing information, so that the missing information of the original foggy image is largely restored, which is convenient for computer analysis and subjective observation by the human eye. The defogging operation of the foggy image can be effectively solved, so that the image information is displayed more realistically, and the defogging effect of the image is improved.

[0059] In one embodiment, a method for dehazing a foggy image by filling in information may also include a first image enhancement process, the specific process including: performing color space adjustment on the processed image data based on the ACE algorithm to obtain adjusted image data; performing dynamic tone scaling on the adjusted image data to obtain intermediate image data.

[0060] Specifically, in one embodiment, the computer device can calculate the output pixel value for each channel in the processed image data based on the ACE algorithm; perform color comparison according to the output pixel value, and perform local or global balancing processing, select an image subset, and obtain adjusted image data.

[0061] The computer device can mainly include two stages in the first image enhancement process of the processed image data based on the ACE algorithm: the first stage is responsible for color space adjustment, responsible for color constancy and contrast adjustment, while the second stage configures the output range to achieve accurate tone mapping, performs a lateral suppression mechanism, weighted by pixel distance, and the result is local-global filtering. The second stage is to dynamically scale the image and normalize white only in the entire range. This two-stage structure is a feature of most human visual system computational models in the field of digital images.

[0062] like Figure 3 As shown, in this embodiment, when the computer device performs the first stage, it generates an output image R by adjusting the color space, wherein each pixel is recalculated according to the image content. Each pixel P of the output image R is calculated for each channel c, and the formula is: Among them, I c (p)-I c (j) represents lateral suppression, d(·) is a distance function that weights local or global quantities, and r(·) represents the relative brightness appearance of the pixel. In the process of calculating the output image pixels, the compensation of the distance from the pixel to the edge is not calculated. Therefore, the calculation formula is modified using the normalization coefficient, and the resulting formula is: Among them, r max is the maximum value of r(·), simulating the lateral inhibition mechanism by computing the difference between each pixel value and all other pixel values ​​in the selected image subset.

[0063] In one embodiment, the computer device may use the adjusted image data as an intermediate pixel matrix; and use a linear scaling method to perform scaling processing on the intermediate pixel matrix to obtain intermediate image data.

[0064] When the computer device implements the second stage, i.e., dynamically scaling the image, it can map the intermediate pixel matrix R to the final output image O, not only by performing a simple dynamic maximization (linear scaling), but also by selecting different reference values ​​in the intermediate matrix to map the relative brightness appearance of each channel to the grayscale, and according to the selected reference point, an additional global balance can be added between the gray world and the white patches. Two linear scaling methods are proposed to obtain a standard 24-bit output image from the signed floating-point matrix R, and alternative scaling methods can be used to take into account the nonlinearity of human brightness adaptation without changing the ACE two-stage structure.

[0065] Specifically, it can be expressed by formula O c (p) = round[127.5 + s c R c (p)] Linear scaling R c The value in Mc As a white reference point, M c The zero value in is used as an estimate of the mid-gray reference point to calculate the slope S. c As a result, the available dynamics cannot be fully utilized and tones around very dark values ​​may be lost, or, c Some values ​​in may lead to negative results. In this case, values ​​less than zero are set to zero. A global gray world adjustment is added to the final scaling, so the dynamics of the final image are always centered around mid-gray.

[0066] In one embodiment, a provided method for defogging a foggy image by filling in information may further include a gamma transform process, the specific process including: determining an optimal gamma value based on the grayscale value of the intermediate image data; and adjusting the gamma value in the intermediate image data to the optimal gamma value through a grayscale gamma transform.

[0067] The basic formula for the gamma transform is: Among them, A and γ are constants, V in is the input gray level, V out is the output gray level; as a function of γ, V out The curves plotted as various values ​​of γ are as follows Figure 4 shown.

[0068] like Figure 4 As shown in the figure, the γ value in the curve maps the input narrowband dark value to the broadband output value, which is also true when the input is high; the curves produced when γ>1 and γ<1 have opposite results. Since the image after the first enhancement is overall dim and dark, grayscale expansion is necessary, which can be accomplished by gamma transformation. The optimal gamma value is 0.4545, where the selection of gamma value is subjectively judged by the human eye, making the secondary enhancement value more suitable for human eye observation. Different gamma values ​​perform gamma transformation on the original grayscale image as shown in the figure below. Figure 5 As shown, when the gamma value is 0.4545, the image is brighter than before, and the details in the image are more obvious, which is convenient for computer analysis and human eye observation.

[0069] In one embodiment, a method for information filling and defogging of a foggy image provided may also include a process of performing image processing using a shading model, and the specific process includes: using a backbone network in the shading model to extract image features in the image data to be filled; inputting the image features into a pixel decoder to restore the image spatial structure; performing color query on the visual features in the image data to be filled through a color decoder and outputting a color representation; fusing the image spatial structure and color representation to generate a color channel output shading result to fill in missing information.

[0070] After the computer device processes the image through the ACE algorithm and gamma transformation, the image is no longer dim but becomes bright, and the local features of the image are obvious, which is conducive to the coloring of the coloring model. In this embodiment, the DDColor coloring model can be used, using an end-to-end method with a dual decoder for image coloring. In one embodiment, the coloring model colors the image data to be filled in an end-to-end manner. The principle of the coloring model is as follows Figure 6 As shown, the model DDColor processes the grayscale image x in an end-to-end manner. l Coloring.

[0071] like Figure 6 As shown in the figure, the backbone network (a) is first used to extract features, which are then input into the pixel decoder to restore the spatial structure of the image. At the same time, the color decoder performs color queries on visual features of different scales to learn semantically aware color representations; the fusion module combines the outputs of the two decoders to produce a color channel output. Finally, along the channel dimension and x l Connect them together to get the final coloring result

[0072] In one embodiment, a computer device may input image features and color query instructions into a color decoding block in a coloring model; and establish a correlation between semantic and color representations through a cross-attention mechanism, a self-attention mechanism, and a feedforward operation adopted in the coloring model.

[0073] like Figure 6 As shown, (b) is the color decoding block structure. The color decoding block takes image features and color queries as input, and establishes the correlation between semantic and color representations through cross-attention mechanism, self-attention mechanism and feedforward operation.

[0074] In one embodiment, a provided information filling and defogging method for a foggy image can be compared with a traditional enhancement algorithm using the SSIM (Structure Similarity Index Measure) structural measurement index. The structural similarity index can measure the degree of distortion of an image, and can also measure the degree of similarity between two images. Unlike MSE and PSNR, which measure absolute errors, SSIM is a perceptual model, which is more in line with the intuitive perception of the human eye. The value range of SSIM is [-1, 1], and it has the properties of symmetry, boundary, and unique maximum (SSIM = 1 when and only when x = y), and is a distance formula.

[0075] SSIM mainly considers three key features of the image: luminance, contrast, and structure. SSIM is an indicator to measure the similarity between two images. Its value range is [0, 1]. The larger the SSIM value, the smaller the image distortion, indicating better image quality. The following table compares the evaluation parameters of 4 images processed by some traditional algorithms. The indicators of most images are better than those of traditional algorithms:

[0076] evaluate Figure 1 Figure 2 Figure 3 Figure 4 Histogram Equalization 0.7783 0.7904 0.6590 0.6838 retinex 0.9025 0.9225 0.8981 0.9741 Dark channel (MSRCR) 0.6203 0.7068 0.6832 0.6606 Information filling dehazing method for foggy images 0.9047 0.9742 0.9190 0.8416

[0077] As shown in the table above, retinex is used in processing Figure 4 The time parameter is better than the information filling defogging method for foggy images provided in this embodiment, but from the subjective observation of the human eye, the defogged image in this embodiment is obviously better than the retinex defogged image, which also shows that this indicator cannot fully evaluate whether the defogged image is convenient for computer and human eye observation, and there are certain shortcomings.

[0078] The image comparison before and after the image processing by the information filling and defogging method for foggy images provided in this embodiment is as follows: Figure 7 As shown, after the defogging method in this embodiment is used for defogging, the image is clearer and the defogging effect is good.

[0079] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0080] In one embodiment, Figure 8 As shown, a system for defogging an image containing fog is provided, comprising: an image acquisition module 810, a first image enhancement module 820, a second image enhancement module 830 and an information filling module 840, wherein:

[0081] The image acquisition module 810 is used to acquire a foggy image and perform image preprocessing on the foggy image to obtain processed image data;

[0082] A first image enhancement module 820 is used to perform a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data;

[0083] The second image enhancement module 830 is used to perform grayscale gamma transformation on the intermediate image data, adjust the gamma value of the intermediate image data, and perform contrast adjustment and second image enhancement to obtain image data to be filled;

[0084] The information filling module 840 is used to fill in missing information of the image data to be filled using a shading model, and output a defogging image.

[0085] In one embodiment, the first image enhancement module 820 is further used to perform color space adjustment on the processed image data based on the ACE algorithm to obtain adjusted image data; and perform dynamic tone scaling processing on the adjusted image data to obtain intermediate image data.

[0086] In one embodiment, the first image enhancement module 820 is also used to calculate the output pixel value for each channel in the processed image data based on the ACE algorithm; perform color comparison according to the output pixel value, and perform local or global balancing processing, select an image subset, and obtain adjusted image data.

[0087] In one embodiment, the first image enhancement module 820 is further configured to use the adjusted image data as an intermediate pixel matrix; and use a linear scaling method to perform scaling processing on the intermediate pixel matrix to obtain intermediate image data.

[0088] In one embodiment, the second image enhancement module 830 is further used to determine the optimal gamma value according to the grayscale value of the intermediate image data; and adjust the gamma value in the intermediate image data to the optimal gamma value through grayscale gamma transformation.

[0089] In one embodiment, the second image enhancement module 830 is also used to use the backbone network in the shading model to extract image features in the image data to be filled; input the image features into the pixel decoder to restore the image space structure; perform color query on the visual features in the image data to be filled through the color decoder, and output the color representation; fuse the image space structure and color representation, generate a color channel output shading result, and fill in the missing information.

[0090] In one embodiment, the second image enhancement module 830 is also used to input image features and color query instructions into the color decoding block in the coloring model; through the cross-attention mechanism, self-attention mechanism and feedforward operation adopted in the coloring model, the correlation between semantic and color representation is established.

[0091] In one embodiment, the second image enhancement module 830 is also used for the coloring model to colorize the image data to be filled in an end-to-end manner.

[0092] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for defogging an image containing fog is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0093] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0094] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, steps of a method for defogging a foggy image by filling in information are implemented.

[0095] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for defogging a foggy image by filling information are implemented.

[0096] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0097] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for defogging a foggy image by filling in information, characterized in that: The method comprises: Acquire a foggy image, and perform image preprocessing on the foggy image to obtain processed image data; Performing a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data, including: performing color space adjustment on the processed image data based on the ACE algorithm to obtain adjusted image data, including: calculating output pixel values ​​for each channel in the processed image data based on the ACE algorithm; performing color comparison according to the output pixel values, and performing local or global balancing processing, selecting an image subset, and obtaining adjusted image data; performing dynamic tone scaling processing on the adjusted image data to obtain intermediate image data, including: using the adjusted image data as an intermediate pixel matrix; and scaling the intermediate pixel matrix using a linear scaling method to obtain intermediate image data; Among them, the first image enhancement includes two stages: the first stage is responsible for color space adjustment, color constancy and contrast adjustment, the second stage configures the output range to achieve tone mapping, performs a lateral suppression mechanism, weighted by pixel distance, and the result is local-global filtering; the second stage is to dynamically scale the image and normalize white only in the entire range; when implementing the first stage, an output image R is generated by color space adjustment, and each pixel is recalculated according to the image content; each pixel P of the output image R is calculated for each channel c separately, and the formula is: Among them, I c (p)-I c (j) represents lateral suppression, d(·) is a distance function that weights local or global quantities, r(·) represents the relative brightness appearance of the pixel, and j represents the pixel in the output image. In the process of calculating the output image pixel, the compensation of the distance from the pixel to the edge is not calculated, and the calculation formula is modified using the normalization coefficient, and the resulting formula is: Among them, r max is the maximum value of r(·), simulating the lateral inhibition mechanism by calculating the difference between each pixel value and all other pixel values ​​in the selected image subset; Performing grayscale gamma transformation on the intermediate image data to adjust the gamma value of the intermediate image data, including: determining an optimal gamma value according to the grayscale value of the intermediate image data; adjusting the gamma value in the intermediate image data to the optimal gamma value through grayscale gamma transformation; and performing contrast adjustment and a second image enhancement to obtain image data to be filled; The method uses a coloring model to fill missing information in the image data to be filled, including: using a backbone network in the coloring model to extract image features in the image data to be filled; inputting the image features and color query instructions into a color decoding block in the coloring model; establishing a correlation between semantics and color representation through a cross-attention mechanism, a self-attention mechanism and a feedforward operation adopted in the coloring model; inputting the image features into a pixel decoder to restore the image space structure; performing a color query on the visual features in the image data to be filled through a color decoder to output a color representation; fusing the image space structure and the color representation to generate a color channel output coloring result to fill missing information; and outputting a dehazed image.

2. The method for defogging foggy images by filling in information according to claim 1, characterized in that: The method further comprises: The coloring model colors the image data to be filled in an end-to-end manner.

3. A foggy image information filling and defogging system, characterized in that: The system comprises: An image acquisition module is used to acquire a foggy image and perform image preprocessing on the foggy image to obtain processed image data; The first image enhancement module is used to perform a first image enhancement on the processed image data based on the ACE algorithm to obtain intermediate image data, including: performing color space adjustment on the processed image data based on the ACE algorithm to obtain adjusted image data, including: calculating output pixel values ​​for each channel in the processed image data based on the ACE algorithm; performing color comparison according to the output pixel values, and performing local or global balancing processing, selecting an image subset, and obtaining adjusted image data; performing dynamic tone scaling processing on the adjusted image data to obtain intermediate image data, including: using the adjusted image data as an intermediate pixel matrix; and scaling the intermediate pixel matrix using a linear scaling method to obtain intermediate image data; Among them, the first image enhancement includes two stages: the first stage is responsible for color space adjustment, color constancy and contrast adjustment, the second stage configures the output range to achieve tone mapping, performs a lateral suppression mechanism, weighted by pixel distance, and the result is local-global filtering; the second stage is to dynamically scale the image and normalize white only in the entire range; when implementing the first stage, an output image R is generated by color space adjustment, and each pixel is recalculated according to the image content; each pixel P of the output image R is calculated for each channel c separately, and the formula is: Among them, I c (p)-I c (j) represents lateral suppression, d(·) is a distance function that weights local or global quantities, and r(·) represents the relative brightness appearance of pixels. In the process of calculating the output image pixels, the compensation of the distance from the pixel to the edge is not calculated. The calculation formula is modified using the normalization coefficient, and the resulting formula is: Among them, r max is the maximum value of r(·), simulating the lateral inhibition mechanism by calculating the difference between each pixel value and all other pixel values ​​in the selected image subset; a second image enhancement module, configured to perform grayscale gamma transformation on the intermediate image data and adjust the gamma value of the intermediate image data, including: determining an optimal gamma value according to the grayscale value of the intermediate image data; adjusting the gamma value in the intermediate image data to the optimal gamma value through grayscale gamma transformation; performing contrast adjustment and second image enhancement to obtain image data to be filled; The information filling module is used to use the coloring model to fill in the missing information of the image data to be filled, including: using the backbone network in the coloring model to extract the image features in the image data to be filled; inputting the image features and color query instructions into the color decoding block in the coloring model; establishing the correlation between semantics and color representation through the cross-attention mechanism, self-attention mechanism and feedforward operation adopted in the coloring model; inputting the image features into the pixel decoder to restore the image space structure; performing color query on the visual features in the image data to be filled through the color decoder and outputting the color representation; fusing the image space structure and the color representation to generate a color channel output coloring result to fill in the missing information; and outputting a dehazed image.

Images