Single Image Dehazing Method Based on Retinex Theory and Adversarial Neural Networks

Abstract

This application discloses a single-image dehazing method, apparatus, device, and storage medium based on Retinex theory and adversarial neural networks. The method includes: determining the hazy image I. H , where I S This represents an image of illumination including haze particles, and represents I. H After G h2f The obtained dehazing result; the hazy image I H After processing with an adversarial network, I is obtained. H And substitute into the solution I S ; and I S A sample is formed and sent to G f2h In the network, a cyclically reconstructed graph is obtained, where G f2h This refers to a haze image generator. The solution provided in this application has good dehazing effect, fast processing time, and portability. It adopts a lightweight network with a model size of 11.38M. On an Nvidia RTX A5000 device, it only takes 0.005 seconds to process a 512*512 color image. Due to the lightweight model and fast calculation speed, the model can be easily ported to existing mobile devices for real-time processing.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method, apparatus, device, and storage medium for single-image dehazing based on Retinex theory and adversarial neural networks. Background Technology

[0002] Haze, a common atmospheric phenomenon, typically reduces scene visibility, thus impacting the performance of many advanced computer vision tasks, such as object detection, segmentation, and remote sensing. Therefore, image dehazing is crucial for outdoor vision systems.

[0003] The classical atmospheric scattering physics model is widely expressed as: I(x) = J(x)t(x) + A(1-t(x)), where I(x) is the observed haze image, J(x) is the dehazed image to be recovered, t(x) is the medium transmission map, x is the pixel coordinate, and A is the global atmospheric light. If the global atmosphere is homogeneous, then the transmission map t(x) can be represented by: t(x) = e βd(x) Let be the descriptor. Here, d(x) represents the scene depth, and β is the atmospheric scattering coefficient. The goal of single-image dehazing is to derive the fog-free scene J(x) from the foggy input t(x) without any other known parameters, which makes the solution of the task highly ill-posed.

[0004] In recent years, significant progress has been made in the study of this problem. Many handcrafted methods rely on priors or assumptions to estimate the medium transmission map t(x) and atmospheric light A. However, in some cases, these methods fail to recover a sharp image when the assumptions and priors are not valid, and may even lead to color distortion and artifacts. The rapid development of deep neural networks in recent years has also facilitated learning-based dehazing methods. This data-driven approach significantly reduces the reliance on handcrafted priors, resulting in fewer artifacts and sharper restorations.

[0005] Due to the lack of real-world training samples, most learning-based methods are trained on synthetic datasets such as RESIDE, Make3D, and the NYU dataset. Because these synthetic, blurred datasets contain limited image categories and depths, while existing learning-based methods can be trained well on synthetic datasets, their generalization to real-world foggy images remains limited. Summary of the Invention

[0006] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a single image dehazing method, apparatus, device and storage medium based on Retinex theory and adversarial neural networks.

[0007] In a first aspect, embodiments of this application provide a single-image dehazing method based on Retinex theory and adversarial neural networks, the method comprising: determining a hazy image I H ,in I S This represents a light image that includes smog particles. Indicate I H After G h2f The obtained dehazing result; the hazy image I H After processing by adversarial networks, it is obtained Will I H and Substitute into Solve I S ;Will and I S A sample is formed and sent to G f2h In the network, a cyclically reconstructed graph is obtained. Among them, G f2h G represents a haze image generator. h2f This indicates a fog-free image generator.

[0008] In one embodiment, the image I with fog H After processing by adversarial networks, it is obtained Includes: Fog Image I H Through adversarial networks D and D dc In the real, clear image I F Dark Channel Diagram Learning to obtain defogging results

[0009] In one embodiment, the clear image I F Send to G h2f In the network, the reconstructed clear image itself is obtained. Among them, G h2f This indicates a fog-free image generator.

[0010] In one embodiment, there will be a foggy image I H Send to G f2h A smog reconstruction map was obtained from the internet.

[0011] Secondly, embodiments of this application also provide a single-image dehazing device based on a Retinex-based adversarial neural network. The device includes: a determining unit, configured to determine a hazy image I. H ,in I S This represents a light image that includes smog particles. Indicate I H After G h2fThe obtained dehazing result; processing unit, used to process the hazy image I H After processing by adversarial networks, it is obtained Solver unit, used to convert I H and Substitute into Solve I S ; Reconstruction unit, used to... and I S A sample is formed and sent to G f2h In the network, a cyclically reconstructed graph is obtained. Among them, G f2h G represents a haze image generator. h2f This indicates a fog-free image generator.

[0012] In one embodiment, the image I with fog H After processing by adversarial networks, it is obtained Includes: Fog Image I H Through adversarial networks D and D dc In the real, clear image I F Dark Channel Diagram Learning to obtain defogging results

[0013] In one embodiment, the clear image I F Send to G h2f In the network, the reconstructed clear image itself is obtained. Among them, G h2f This indicates a fog-free image generator.

[0014] In one embodiment, there will be a foggy image I H Send to G f2h A smog reconstruction map was obtained from the internet.

[0015] Thirdly, embodiments of this application also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the methods described in the embodiments of this application.

[0016] Fourthly, embodiments of this application also provide a computer device and a computer-readable storage medium having a computer program stored thereon, the computer program being used to: implement any of the methods described in the embodiments of this application when the computer program is executed by a processor.

[0017] The beneficial effects of this invention are:

[0018] The single-image dehazing method based on Retinex theory and adversarial neural networks provided by this invention has good dehazing effect, fast processing time and portability. The overall network is relatively lightweight, with a model size of 11.38M. On an Nvidia RTX A5000 device, a 512*512 color image was tested in just 0.005 seconds. Due to the lightweight model and fast calculation speed, the model can be easily ported to existing mobile devices for real-time processing. Attached Figure Description

[0019] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0020] Figure 1 A flowchart illustrating the single-image dehazing method based on Retinex theory and adversarial neural networks provided in this application embodiment is shown.

[0021] Figure 2 An exemplary structural block diagram of a single-image dehazing device 200 based on a Retinex-based adversarial neural network according to an embodiment of this application is shown;

[0022] Figure 3 A schematic diagram of the structure of a computer system suitable for implementing the terminal device of the present application is shown;

[0023] Figure 4 A schematic diagram of the model provided in the embodiments of this application is shown;

[0024] Figure 5 The embodiment of G provided in this application is shown. h2f and G f2h Network structure diagram;

[0025] Figure 6 The embodiments of D and D provided in this application are shown. dc A schematic diagram of the network structure. Detailed Implementation

[0026] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0027] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0028] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0029] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0030] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0031] It should be noted that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.

[0032] Please refer to Figure 1 , Figure 1 The diagram shows a flowchart of a single-image dehazing method based on Retinex theory and adversarial neural networks provided in an embodiment of this application.

[0033] like Figure 1 As shown, the method includes:

[0034] Step 110, determine the foggy image I H ,in I S This represents a light image that includes smog particles. Indicate I H After G h2f The defogging results obtained;

[0035] Step 120, take the foggy image I H After processing by adversarial networks, it is obtained

[0036] Step 130, I H and Substitute into Solve I S ;

[0037] Step 140, will and I S A sample is formed and sent to G f2h In the network, a cyclically reconstructed graph is obtained. Among them, G f2h G represents a haze image generator. h2f This indicates a fog-free image generator.

[0038] The above technical solution offers excellent dehazing performance, fast processing time, and portability. It utilizes a lightweight network with a model size of 11.38M. On an Nvidia RTX A5000 device, a 512*512 color image was tested in just 0.005 seconds. Due to the lightweight model and fast computation speed, it is easily ported to existing mobile devices for real-time processing.

[0039] In some embodiments, reference Figure 6 As shown, the image I in this application will have fog. H After processing by adversarial networks, it is obtained Includes: Fog Image I H Through adversarial networks D and D dc In the real, clear image I F Dark Channel Diagram Learning to obtain defogging results

[0040] In some embodiments, reference Figure 4 and Figure 5 As shown, in this application, clear image I F Send to G h2f In the network, the reconstructed clear image itself is obtained. Among them, G h2f This indicates a fog-free image generator.

[0041] In some embodiments, reference Figure 4 and Figure 5 As shown, there will be a foggy image I. H Send to G f2h A smog reconstruction map was obtained from the internet.

[0042] This invention extends the image decomposition principle in Retinex to the image dehazing task. Specifically, it decomposes a foggy image I... H It is considered to be the result of the combined effect of the reflection map and the illumination map, that is... Where I S This image shows the lighting caused by the presence of haze particles.

[0043] like Figure 4 As shown in the model design in the image: (left) This application will have a fog image I H Enter G h2f Networks, utilizing adversarial networks D and D dc In the real, clear image I F and their secret passage diagram The defogging results were obtained from the middle school. Then, according to formula (1), the illumination diagram I is obtained. S .Will and I S A sample is formed and sent to G. f2h The network yields a graph that is reconstructed cyclically. (Right) Two self-supervised modules used to learn the relationship between haze and light intensity. For clear image I F After passing through a generator G for producing a sharp effect h2f The reconstructed image itself is obtained. For a hazy image, the illumination information caused by the haze is already included, so it and a mask with a value of 0 form an input, which is then processed by a haze generator G. f2h Obtain the smog reconstruction map The two modules work together to train the entire design, enabling it to function and optimize effectively, resulting in a satisfactory defogging effect.

[0044] Further, refer to Figure 2 , Figure 2 An exemplary structural block diagram of a single-image dehazing device 200 based on a Retinex-based adversarial neural network according to an embodiment of this application is shown.

[0045] like Figure 2 As shown, the device includes:

[0046] Determining unit 210 is used to determine the foggy image I. H ,in I S This represents a light image that includes smog particles. Indicate I H After G h2f The defogging results obtained;

[0047] Processing unit 220 is used to process the foggy image I H I is obtained after processing by an adversarial network. F ;

[0048] Solver 230 is used to solve I H and Substitute into Solve I S ;

[0049] Reconstruction unit 240, used to... and I S A sample is formed and sent to G f2h In the network, a cyclically reconstructed graph is obtained. Among them, G f2h G represents a haze image generator. h2f This indicates a fog-free image generator.

[0050] It should be understood that the units or modules described in device 200 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations and features described above for the method also apply to the device 200 and the units contained therein, and will not be repeated here. The device 200 can be pre-implemented in the browser or other security applications of an electronic device, or it can be loaded into the browser or its security applications of an electronic device through download or other means. The corresponding units in the device 200 can cooperate with the units in the electronic device to implement the solutions of the embodiments of this application.

[0051] The following is for reference. Figure 3 It shows a schematic diagram of the structure of a computer system 300 suitable for implementing terminal devices or servers in the embodiments of this application.

[0052] like Figure 3 As shown, the computer system 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the system 300. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0053] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0054] In particular, according to embodiments of this disclosure, the above references Figure 1 The described process can be implemented as a computer software program. For example, embodiments of this disclosure include a single-image dehazing method based on Retinex theory and adversarial neural networks, which includes a computer program tangibly contained on a machine-readable medium, the computer program containing instructions for performing... Figure 1 The program code for the method. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 309, and / or installed from the removable medium 311.

[0055] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0056] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor can be described as including a first sub-region generation unit, a second sub-region generation unit, and a display area generation unit. The names of these units or modules do not necessarily limit the specific unit or module itself; for example, the display area generation unit can also be described as "a unit for generating a display area of ​​text based on the first and second sub-regions."

[0057] In another aspect, this application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the aforementioned apparatus in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the text generation method described in this application for transparent window envelopes.

[0058] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A single-image dehazing method based on Retinex theory and adversarial neural networks, characterized in that, The method includes: A fog-free image generator is obtained by training both modules together. This is used for dehazing a single image, and the training steps of one of the modules include: Image with fog confirmed There will be fog images Through adversarial networks and In real, clear images Dark Channel Diagram After processing, the result is obtained , express go through The defogging results obtained; Will and Substitute into Solve , This represents a light image that includes smog particles; Will and A sample is sent to In the middle, the cyclic reconstruction graph is obtained. , This refers to a haze image generator; The training steps for another module include: Haze Image Generator Training will be performed on foggy images. A mask with a sum of 0 is fed into In the middle, a smog reconstruction map was obtained. ; Haze-free image generator Trained as a self-supervised module, using clear images Send to In the process, a clear reconstructed image is obtained. .

2. A single-image dehazing device based on Retinex theory and adversarial neural networks, characterized in that, The device comprises two modules configured to be trained together to obtain a fog-free image generator. Used for dehazing a single image; A module includes: The processing unit is configured to determine the presence of fog in the image. There will be fog images Through adversarial networks and In real, clear images Dark Channel Diagram After processing, the result is obtained , express go through The defogging results obtained; Solver element, configured to... and Substitute into Solve , This represents a light image that includes smog particles; Reconstruction unit, configured to... and A sample is sent to In the middle, the cyclic reconstruction graph is obtained. , This refers to a haze image generator; Another module includes: The first self-supervised module is configured to generate haze images. Training will be performed on foggy images. A mask with a sum of 0 is fed into In the middle, a smog reconstruction map was obtained. ; The second self-supervised module is configured to generate a fog-free image. Trained as a self-supervised module, using clear images Send to In the process, a clear reconstructed image is obtained. .

3. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, the computer program being configured to: implement the method of claim 1 when executed by a processor.

Images