Image defogging method based on feature aggregation and prior feature compensation and related device
By constructing pseudo-clear images and unpaired data using real-world scene data in the image dehazing algorithm, and combining convolutional neural networks and physically driven feature aggregation, the adaptability and interpretability issues of existing algorithms in real-world scenes are solved, achieving better dehazing effects and model transparency.
Patent Information
- Application Number
- CN202511550851.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-28
- Publication Date
- 2026-02-06
AI Technical Summary
Existing image dehazing algorithms perform poorly in real-world scenarios. The training data differs greatly from the real-world scene, and they lack physical and prior knowledge, resulting in insufficient processing capabilities for complex scenes and poor interpretability.
By collecting real-world foggy image data, constructing pseudo-clear image data and unpaired real-world foggy image data, convolutional neural networks are used for feature aggregation and physical-driven processing. Combined with multiple image prior processing methods, a fusion and splicing semi-supervised learning method is used for model training.
It significantly improves the adaptability and interpretability of image dehazing models to real and complex environments, reduces artifacts and overprocessing, and improves dehazing effect and model transparency.
Smart Images
Figure CN121481882A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of image data processing, and particularly relates to an image defogging method based on feature aggregation and prior feature compensation and a related device. BACKGROUND
[0002] In automatic driving and remote sensing monitoring systems, the image defogging processing effect of real scenes is crucial for the detection and decision of subsequent systems. However, the existing image defogging algorithms in industrial scenes generally have the following problems: (1) the difference between the training data and the real scene is too large: the training data is artificially synthesized to fit the foggy scene, which is quite different from the real foggy weather, affecting the performance of the algorithm in the real scene; (2) less physical knowledge: the image defogging algorithm based on neural network pays more attention to the network architecture design and ignores the physical modeling process, resulting in poor network interpretability; (3) weak prior knowledge learning: the existing statistical prior knowledge is not fully utilized, and the network processing effect is limited.
[0003] Due to the above problems, the existing image defogging algorithm can only process synthetic scene or specific scene foggy image data, and lacks the ability to process complex real scene foggy image data well and lacks interpretability. SUMMARY
[0004] The purpose of the present application is to solve the problems in the prior art, and to provide an image defogging method based on feature aggregation and prior feature compensation and a related device to improve the defogging processing effect and interpretability of complex real scene foggy images.
[0005] In order to achieve the above purpose, the present application has the following technical solutions: In a first aspect, an image defogging method based on feature aggregation and prior feature compensation is provided, comprising: Collecting real scene foggy image data, constructing pseudo-clear image data according to the real scene foggy image data, and using a data augmentation method based on a diffusion model to construct non-paired real scene foggy image data; Using a convolutional neural network (CNN) to extract features of the real scene foggy image data, and using a physically driven feature aggregation method to preliminarily defog the extracted features to obtain preliminary defogging image data; Using multiple image prior processing methods to obtain multiple groups of image features, and compensating the preliminary defogging image data; Using the defogging image data after feature compensation, the pseudo-clear image data, and the non-paired real scene foggy image data, using a fusion and splicing semi-supervised learning method to train an image defogging model, and obtaining a defogging image through the trained image defogging model.
[0006] As a preferred embodiment, the step of constructing pseudo-clear image data based on real-world foggy image data involves selecting Enhanced Pixel-to-Pixel Image Dehazing Algorithm (EPDN), Rule-Driven Synthesis-to-Realism Image Dehazing Algorithm (PSD), and Domain Adaptive Image Dehazing Algorithm (DAD). The selected image dehazing algorithms are used to perform preliminary processing on the real-world foggy image data, yielding three intermediate dehazing result data. These three intermediate dehazing result data are then synthesized using a mask to obtain synthesized dehazing image data. Finally, an image domain transfer network is used to perform image domain transfer on the synthesized dehazing image data, resulting in pseudo-clear image data that approximates the real-world image domain.
[0007] As a preferred embodiment, the step of constructing unpaired real-world foggy image data using a data augmentation method based on a diffusion model involves selecting multiple unpaired prompt words and generating multiple unpaired real-world foggy image data that are similar to the style, scene, and fog effect features of the real-world foggy image data through the unpaired prompt words.
[0008] As a preferred embodiment, the real scene foggy image data consists of 200 images, the constructed pseudo-clear image data consists of 200 images, and the constructed unpaired real scene foggy image data consists of 1000 images.
[0009] As a preferred embodiment, the steps of using a convolutional neural network (CNN) to extract features from real-world foggy image data, performing preliminary dehazing on the extracted features using a physically driven feature aggregation method to obtain preliminary dehazed image data, and using a U-shaped architecture convolutional neural network to perform preliminary dehazing on the extracted features, replacing the skip connections in the U-shaped architecture convolutional neural network with a physically driven feature aggregation method to obtain preliminary dehazed image data, are expressed as follows:
[0010]
[0011] in, For the U-shaped architecture convolutional neural network, the first i One upsampling module, These are the coefficients of the defogging physical model, derived from... Uniform feature separation was performed to obtain, For the U-shaped architecture convolutional neural network, the first i One downsampling module; This indicates splitting a vector into two vectors along the channel dimension; This represents the Hadamard product operation; The expression for the defogging physical model is as follows: In the formula, Represents a foggy image. representing a clear image, representing a transmittance map, representing a global atmospheric light intensity.
[0012] As a preferred scheme, the step of using multiple image prior processing methods to obtain multiple groups of image features and performing feature compensation on the preliminary defogging image data includes: using a dark channel prior processing method and a limited contrast adaptive histogram equalization prior processing method to obtain image features, respectively; splicing the obtained image features and the preliminary defogging image features to obtain spliced features and send them into an attention module to obtain adaptive weights; using the spliced features and the adaptive weights to obtain an intermediate defogging result, and the expression is as follows:
[0013] In the formula, is the preliminary defogging image data, is the image feature obtained by using the dark channel prior processing method, is the image feature obtained by using the limited contrast adaptive histogram equalization prior processing method, are adaptive weights, respectively; represents Hadamard product operation; using a feature extraction module, inputting the intermediate defogging result and the preliminary defogging image feature to obtain a defogging image data after feature compensation , and the expression is as follows:
[0014]
[0015] In the formula, is a channel attention matrix, is a spatial attention matrix, is the preliminary defogging image feature; represents a first attention mechanism based module; represents a second attention mechanism based module; represents a convolution layer.
[0016] As a preferred scheme, in the step of using the dehazed image data after feature compensation and the pseudo-clear image data, the unpaired real scene hazy image data for image dehazing model training, the cutting of the real scene hazy image data and the unpaired real scene hazy image data is divided into four equal image blocks of upper left, lower left, upper right and lower right; the upper left block and the lower right block of one image are spliced with the lower left block and the upper right block of another image; the lower left block and the upper right block of one image are spliced with the upper left block and the lower right block of another image; the image dehazing algorithm is used to process the two spliced images multiple times to obtain intermediate results 1, 2, …, N-1 and a final result N; for the intermediate results, the spliced parts are separated; for the image block results of the real scene hazy image data, the corresponding pseudo-clear image data image block is used for loss function calculation; for the image block results of the unpaired real scene hazy image data, the final result N corresponding image block is used for loss function calculation; The loss function calculation expression of the image dehazing model is as follows:
[0017]
[0018]
[0019] In the formula, is the loss calculation of the real scene hazy image data image block result part, is the loss calculation of the unpaired real scene hazy image data image block result part, is the real scene hazy image data image block intermediate result, is the unpaired real scene hazy image data image block intermediate result, is the pseudo-clear image data image block, is the loss calculation weight.
[0020] In a second aspect, an image dehazing system based on feature aggregation and prior feature compensation is provided, comprising: An image data construction module is configured to collect real scene hazy image data, construct pseudo-clear image data according to the real scene hazy image data, and construct unpaired real scene hazy image data using a data augmentation method based on a diffusion model; A feature extraction and aggregation module is configured to extract features of the real scene hazy image data using a convolutional neural network (CNN), and perform preliminary dehazing processing on the extracted features using a physically driven feature aggregation method to obtain preliminary dehazed image data; A prior feature compensation module is configured to obtain multiple groups of image features using multiple image prior processing methods, and perform feature compensation on the preliminary dehazed image data; The fusion splicing semi-supervised learning module is used for training an image defogging model by using a fusion splicing semi-supervised learning method, and obtaining a defogging image by using the trained image defogging model.
[0021] In a third aspect, an electronic device is provided, comprising: The memory stores at least one instruction, and the processor executes the instruction stored in the memory to realize the image defogging method based on feature aggregation and prior feature compensation as described in the first aspect.
[0022] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores at least one instruction, which is executed by a processor in an electronic device to realize the image defogging method based on feature aggregation and prior feature compensation as described in the first aspect.
[0023] Compared with the prior art, the present application has at least the following beneficial effects: The application significantly enhances the adaptability and representation ability of the image defogging model to real complex environments through a multi-level real scene data construction strategy. Specifically, by collecting real scene foggy image data, different fog concentration, lighting conditions and dynamic scenes can be covered, providing a data basis close to actual application. According to the real scene foggy image data, the corresponding pseudo-clear image is generated in reverse, avoiding the reliance on artificial synthetic data, ensuring that the clear image is highly consistent with the geometric structure and texture features of the real scene, and reducing data bias. The data augmentation method based on the diffusion model is used to construct unpaired real scene foggy image data, which can control the fog type and scene elements through the prompt (Prompt), expand the data size without pairing annotation, and improve the coverage ability of the image defogging model to rare scenes. The application solves the problems of artifacts and over-processing that are easily produced by traditional methods under complex fog conditions through a feature processing mechanism combining physical constraints and prior knowledge. Specifically, the convolutional neural network (CNN) is used to extract the features of the real scene foggy image data, and a physically driven feature aggregation method is used for preliminary defogging processing of the extracted features to obtain preliminary defogging image data, avoiding the non-physical distortion that may be introduced by data-driven methods. At the same time, multiple image prior processing methods are used to obtain multiple sets of image features, and the preliminary defogging image data is compensated to improve the quality of the defogging image data. In addition, the application realizes the dual improvement of efficient training and model stability through a semi-supervised learning framework and a feature fusion strategy. Specifically, the semi-supervised training strategy uses pseudo-clear image data (labeled) and unpaired real scene foggy image data (unlabeled) for joint training, reducing the dependence on manual annotation, greatly shortening the training time, and avoiding overfitting. In the model architecture, physically driven features, prior compensation features and diffusion model augmented features are explicitly fused, combining physical laws and data-driven advantages in the defogging process. The application uses a physically driven feature aggregation method for preliminary defogging processing, and improves the transparency of the model decision-making process through the integration of physically embedded features and feature visualization design. In the feature aggregation stage, physical modules such as transmittance estimation layer and atmospheric light correction layer are embedded, and their output (such as transmittance map and atmospheric light value) can directly reflect the physical parameters of the fog effect, which is convenient for analyzing the model's processing logic for fog. Through the visualization of preliminary defogging features, prior compensation weights and final output, the model's attention to different fog effect areas (such as high transmittance clear area and low transmittance heavy fog area) can be directly displayed, providing a basis for parameter adjustment in actual application. BRIEF DESCRIPTION OF DRAWINGS
[0024] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and for those skilled in the art, other related drawings can also be obtained without creative labor.
[0025] Figure 1 The embodiment of the application constructs a pseudo-clear image data and a non-paired real scene foggy image data flow chart; Figure 2 The embodiment of the application is based on a feature aggregation and prior feature compensation image defogging method flow chart; Figure 3 The embodiment of the application is a fusion splicing semi-supervised learning method principle diagram. DETAILED DESCRIPTION
[0026] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, other embodiments can be obtained by those skilled in the art without creative labor.
[0027] The embodiment of the application proposes an image defogging method based on feature aggregation and prior feature compensation. The image defogging model's representation ability for the real world is improved by training data of real scene defogging images. The processing effect and interpretability of the image defogging model are improved by using a physically driven feature aggregation method and introducing various image prior processing methods.
[0028] The embodiment of the application is based on an image defogging method based on feature aggregation and prior feature compensation, including the following steps: S1, collect real scene foggy image data, construct pseudo-clear image data according to the real scene foggy image data, and use a data augmentation method based on a diffusion model to construct non-paired real scene foggy image data; S2, use a convolutional neural network (CNN) to extract features of the real scene foggy image data, and use a physically driven feature aggregation method to preliminarily defog the extracted features to obtain preliminary defogging image data; S3, use various image prior processing methods to obtain multiple groups of image features, and compensate the preliminary defogging image data; S4, use the defogging image data after feature compensation, the pseudo-clear image data, and the non-paired real scene foggy image data to train an image defogging model using a fusion splicing semi-supervised learning method, and obtain a defogging image through the trained image defogging model.
[0029] Please refer to Figure 1In a possible implementation, when constructing the pseudo-clear image data from the real scene hazy image data in step S1, three image dehazing algorithms are selected, including an enhanced pixel-to-pixel image dehazing algorithm EPDN, a rule-driven synthetic-to-real image dehazing algorithm PSD, and a domain adaptation image dehazing algorithm DAD. The selected image dehazing algorithms are used to preliminarily process the real scene hazy image data to obtain three dehazing intermediate result data. The intermediate result data is used for artificial result synthesis, and the three intermediate result data are synthesized through a mask by a subjective evaluation method to obtain synthetic dehazing image data. The image domain migration network is used for image domain migration of the synthetic dehazing image data to obtain high-quality pseudo-clear image data closer to the real world image domain.
[0030] The enhanced pixel-to-pixel image dehazing algorithm EPDN, the rule-driven synthetic-to-real image dehazing algorithm PSD, and the domain adaptation image dehazing algorithm DAD all have unique advantages in the field of image dehazing and are suitable for different scene requirements.
[0031] The EPDN algorithm usually adopts an end-to-end training method and can directly learn the mapping relationship from a hazy image to a haze-free image. It uses a generative adversarial network (GAN) or a similar deep learning architecture to enhance the clarity and contrast of the image through pixel-level processing. The end-to-end training method of the EPDN algorithm simplifies the training process and improves the generalization ability of the model. With pixel-level processing performance, it can more accurately restore image details and reduce artifacts and noise. The EPDN algorithm is suitable for various scenes and can achieve good dehazing results whether for synthetic data or real data. However, the EPDN algorithm requires a large number of hazy-haze-free image pairs for training, which may be difficult to obtain in real scenarios. Due to the use of deep learning architecture, the computational complexity of the EPDN algorithm is usually high, which may not be suitable for real-time processing.
[0032] The PSD algorithm aims to solve the domain adaptation problem from synthetic data to real data. It uses physical priors or rules to guide the training of the model, so that the model trained on synthetic data can better adapt to real data. The PSD algorithm has strong domain adaptation ability and can reduce the domain gap between synthetic data and real data by introducing physical priors or rules. Unlike EPDN, the PSD algorithm does not require a large number of real hazy-haze-free image pairs for training. Due to the introduction of physical priors or rules, the PSD algorithm usually has strong interpretability. However, the PSD algorithm relies on the accuracy of the physical priors or rules: if the physical priors or rules are not accurate, the dehazing effect may be poor. Compared with deep learning-based algorithms, the PSD algorithm may lack adaptability to complex scenes.
[0033] The DAD algorithm also focuses on domain adaptation, but it may use different strategies or techniques to realize the transition from synthetic data to real data. The DAD algorithm may combine feature alignment, image translation, etc. to reduce the domain gap. The DAD algorithm can use multiple domain adaptation strategies such as feature alignment, image translation, etc. to adapt to different scene requirements. Due to the use of multiple techniques, the DAD algorithm may have better adaptability to complex scenes. Under the appropriate domain adaptation strategy, the DAD algorithm can usually show stable defogging effect. However, due to the combination of multiple techniques, the implementation complexity of the DAD algorithm may be high. The combination of multiple techniques may cause parameter adjustment difficulty, which requires more experiments and tuning.
[0034] Further, the non-paired real scene fog image data is constructed using a diffusion model-based data augmentation method, a plurality of non-paired prompt words are selected, and a plurality of non-paired real scene fog image data similar in style, scene, and fog effect characteristics to the real scene fog image data are generated through the non-paired prompt words, while avoiding directly copying the original data to ensure data diversity.
[0035] The real scene fog image data of the embodiment of the application is 200, the pseudo-clear image data constructed is 200, and the non-paired real scene fog image data constructed is 1000.
[0036] Please refer to Figure 2 In a possible implementation, the step S2 of the embodiment of the application adopts a U-shaped architecture convolutional neural network to preliminarily defog the extracted features, adopts a physically driven feature aggregation method to replace the skip connection process in the U-shaped architecture convolutional neural network, and obtains preliminary defogging image data. The processing process is expressed as follows:
[0037]
[0038] wherein, is the i-th up-sampling module of the U-shaped architecture convolutional neural network, and the upward arrow represents the features from the up-sampling layer of the U-shaped architecture convolutional neural network; i are defogging physical model coefficients, respectively, and are obtained by uniform feature separation by is the i-th down-sampling module of the U-shaped architecture convolutional neural network, and the downward arrow represents the features from the down-sampling layer of the U-shaped architecture convolutional neural network; i represents that a vector is split into two vectors in the channel dimension. denotes the Hadamard product, i.e., the element-wise multiplication operation of matrices or vectors.
[0039] The expression of the defogging physical model is , wherein denotes the fog image, denotes the clear image, denotes the transmittance map, denotes the global atmospheric light intensity.
[0040] The expression of the defogging physical model is converted to the feature level, i.e., the above expression is obtained, and is integrated into the network architecture.
[0041] In a possible implementation, the step S3 of the embodiment of the present application adopts a dark channel prior processing method and a limited contrast adaptive histogram equalization prior processing method to obtain image features respectively; the obtained image features and preliminary defogging image features are spliced to obtain spliced features and send the spliced features into an attention module to obtain adaptive weights; an intermediate defogging result is obtained by using the spliced features and the adaptive weights, and the expression is as follows:
[0042] , wherein is preliminary defogging image data, is image features obtained by using the dark channel prior processing method, is image features obtained by using the limited contrast adaptive histogram equalization prior processing method, are adaptive weights respectively; denotes the Hadamard product, i.e., the element-wise multiplication operation of matrices or vectors.
[0043] The feature extraction module is used, the intermediate defogging result and the preliminary defogging image features are input, and defogging image data after feature compensation is obtained , and the expression is as follows:
[0044]
[0045] , wherein is a channel attention matrix, is a spatial attention matrix, is preliminary defogging image features; denotes a first attention mechanism based module; denotes a second attention mechanism based module; denotes a convolution layer.
[0046] The embodiment of the present application adopts physical driving feature aggregation, designs a feature extraction module based on an atmospheric scattering model, guides a CNN network to focus on key features of fog effects (such as color attenuation and edge blur) through physical constraints such as transmittance estimation and atmospheric light correction, and avoids non-physical distortion that may be introduced by a pure data-driven method. Meanwhile, a multi-image prior compensation is adopted, and multi-element information such as a dark channel prior and a color constancy prior is introduced to compensate features of a preliminary defogging result. For example, the dark channel prior is used to optimize a transmittance map and eliminate a halo effect; and the color constancy prior is used to correct color cast and make the image color more natural.
[0047] Please refer to Figure 3 When the embodiment of the present application uses the fusion splicing semi-supervised learning method to train the image defogging model in step S4, the real scene foggy image data and the non-paired real scene foggy image data are cut into four equal image blocks, i.e., upper left, lower left, upper right and lower right blocks; the upper left block and the lower right block of one of the images are spliced with the lower left block and the upper right block of the other image; and the lower left block and the upper right block of one of the images are spliced with the upper left block and the lower right block of the other image. The image defogging algorithm is used to process the two spliced images multiple times to obtain intermediate results 1, 2,..., N-1 and a final result N. For the intermediate results, the spliced parts are separated; for the image block results of the real scene foggy image data, the corresponding pseudo-clear image data image blocks are used for loss function calculation; and for the image block results of the non-paired real scene foggy image data, the corresponding image blocks of the final result N are used for loss function calculation. The loss function calculation expression of the image defogging model is as follows:
[0048]
[0049]
[0050] In the formula, is the loss calculation of the image block results of the real scene foggy image data, is the loss calculation of the image block results of the non-paired real scene foggy image data, is the intermediate result of the image block of the real scene foggy image data, is the intermediate result of the image block of the non-paired real scene foggy image data, is the image block of the pseudo-clear image data, is the loss calculation weight.
[0051] Another embodiment of the present application further provides an image defogging system based on feature aggregation and prior feature compensation, comprising: The image data construction module is used for collecting real scene foggy image data, constructing pseudo-clear image data according to the real scene foggy image data, and constructing unpaired real scene foggy image data using a diffusion model-based data augmentation method. The feature extraction and aggregation module is used for extracting features of the real scene foggy image data using a convolutional neural network (CNN), performing preliminary de-fogging processing on the extracted features using a physically driven feature aggregation method, and obtaining preliminary de-fogging image data. The prior feature compensation module is used for obtaining multiple groups of image features using multiple image prior processing methods, and performing feature compensation on the preliminary de-fogging image data. The fusion and splicing semi-supervised learning module is used for training an image de-fogging model using the de-fogging image data after feature compensation, pseudo-clear image data, and unpaired real scene foggy image data, and obtaining a de-fogging image through the trained image de-fogging model.
[0052] Another embodiment of the present application also provides an electronic device, comprising: The memory stores at least one instruction, and the processor executes the instruction stored in the memory to realize the image de-fogging method based on feature aggregation and prior feature compensation.
[0053] Another embodiment of the present application also provides a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to realize the image de-fogging method based on feature aggregation and prior feature compensation.
[0054] For example, the instruction stored in the memory can be divided into one or more modules / units, which are stored in the computer readable storage medium and executed by the processor to complete the image de-fogging method based on feature aggregation and prior feature compensation. The one or more modules / units can be a series of computer readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the server.
[0055] The electronic device can be a smart phone, a notebook, a palm computer, a cloud server, and other computing devices. The electronic device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the electronic device can further include more or less components, or combine certain components, or different components, for example, the electronic device can further include an input / output device, a network access device, a bus, etc.
[0056] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.
[0057] The memory can be an internal storage unit of the server, such as a hard disk or a memory of the server. The memory can also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory can also include both the internal storage unit and the external storage device of the server. The memory is used to store the computer readable instructions and other programs and data required by the server. The memory can also be used to temporarily store data that has been output or will be output.
[0058] It should be noted that the information interaction and execution process between the above module units are based on the same concept as the method embodiments, and the specific functions and technical effects brought about can be referred to the method embodiment part, which will not be repeated here.
[0059] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit or module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit or module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit or module in the system can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.
[0060] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.
[0061] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments.
[0062] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. An image dehazing method based on feature aggregation and prior feature compensation, characterized in that, include: Collect real-world foggy image data, construct pseudo-sharp image data based on the real-world foggy image data, and use a data augmentation method based on a diffusion model to construct unpaired real-world foggy image data. A convolutional neural network (CNN) is used to extract features from real-world foggy image data. The extracted features are then processed using a physical-driven feature aggregation method to perform preliminary defogging, resulting in preliminary defogging image data. Multiple image prior processing methods are used to obtain multiple sets of image features, and feature compensation is performed on the preliminary dehazed image data. Using feature-compensated dehazed image data, pseudo-clear image data, and unpaired real-world foggy image data, a fusion-stitching semi-supervised learning method is used to train an image dehazing model. Dehazed images are then obtained through the trained image dehazing model.
2. The image dehazing method based on feature aggregation and prior feature compensation according to claim 1, characterized in that, The step of constructing pseudo-sharp image data based on real-world foggy image data involves selecting Enhanced Pixel-to-Pixel Image Dehazing Algorithm (EPDN), Rule-Driven Synthesis-to-Realism Image Dehazing Algorithm (PSD), and Domain Adaptive Image Dehazing Algorithm (DAD). Each of these algorithms is used to perform preliminary processing on the real-world foggy image data, yielding three intermediate dehazing results. These three intermediate results are then combined using a mask to obtain synthesized dehazed image data. Finally, an image domain transfer network is used to perform image domain transfer on the synthesized dehazed image data, resulting in pseudo-sharp image data that approximates the real-world image domain.
3. The image dehazing method based on feature aggregation and prior feature compensation according to claim 1, characterized in that, The step of constructing unpaired real-world foggy image data using a data augmentation method based on a diffusion model involves selecting multiple unpaired cue words and generating multiple unpaired real-world foggy image data that are similar to the style, scene, and fog effect features of the real-world foggy image data through the unpaired cue words.
4. The image dehazing method based on feature aggregation and prior feature compensation according to claim 1, characterized in that, The data consists of 200 real-world foggy images, 200 constructed pseudo-clear images, and 1000 unpaired real-world foggy images.
5. The image dehazing method based on feature aggregation and prior feature compensation according to claim 1, characterized in that, The step involves using a Convolutional Neural Network (CNN) to extract features from real-world foggy image data, performing preliminary dehazing on the extracted features using a physically driven feature aggregation method, and obtaining preliminary dehazed image data. The process involves using a U-shaped architecture CNN to perform preliminary dehazing on the extracted features, and replacing the skip connections in the U-shaped architecture CNN with a physically driven feature aggregation method to obtain preliminary dehazed image data. The processing expression is as follows: in, For the U-shaped architecture convolutional neural network, the first i One upsampling module, These are the coefficients of the defogging physical model, derived from... Uniform feature separation was performed to obtain, For the U-shaped architecture convolutional neural network, the first i One downsampling module; This indicates splitting a vector into two vectors along the channel dimension; This represents the Hadamard product operation; The expression for the defogging physical model is as follows: In the formula, Represents a foggy image. Represents a clear image. Representative transmittance diagram. It represents the global atmospheric light intensity.
6. The image dehazing method based on feature aggregation and prior feature compensation according to claim 1, characterized in that, The step of using multiple image prior processing methods to obtain multiple sets of image features and performing feature compensation on the preliminary dehazed image data includes: obtaining image features using a dark channel prior processing method and a contrast-limited adaptive histogram equalization prior processing method respectively; concatenating the obtained image features with the preliminary dehazed image features to obtain concatenated features and feeding them into the attention module to obtain adaptive weights; and using the concatenated features and adaptive weights to obtain the intermediate dehazed result, as shown in the following expression: In the formula, For preliminary dehazed image data, Image features obtained using dark channel prior processing. Image features obtained using the contrast-limited adaptive histogram equalization prior processing method. These are adaptive weights; This represents the Hadamard product operation; Using the feature extraction module, input the intermediate dehazing results and the initial dehazing image features to obtain the dehazing image data after feature compensation. The expression is as follows: In the formula, For channel attention matrix, For spatial attention matrix, Preliminary features for dehazing images; This indicates the first module based on the attention mechanism; This indicates the second attention-based module; This indicates a convolutional layer.
7. The image dehazing method based on feature aggregation and prior feature compensation according to claim 1, characterized in that, In the step of training the image dehazing model using a fusion and stitching semi-supervised learning method with dehazed image data after feature compensation, pseudo-sharp image data, and unpaired real-scene foggy image data, the real-scene foggy image data and unpaired real-scene foggy image data are divided into four equal parts: top left, bottom left, top right, and bottom right. The top left and bottom right blocks of one image are stitched together with the bottom left and top right blocks of another image. The bottom left and top right blocks of one image are then stitched together with the top left and bottom right blocks of another image. The image dehazing algorithm is used to process the two stitched images multiple times to obtain intermediate results 1, 2, ..., N-1 and the final result N. For the intermediate results, the stitched parts are separated. For the image block results of the real-scene foggy image data, the corresponding pseudo-sharp image data block is used to calculate the loss function. For the image block results of the unpaired real-scene foggy image data, the image block corresponding to the final result N is used to calculate the loss function. The loss function of the image dehazing model is calculated as follows: In the formula, For calculating the loss of image patch results in real-world foggy image data, For image patch results of unpaired real-world foggy image data, partial loss is calculated. This is an intermediate result of image patches from real-world foggy image data. Intermediate results for image patches from unpaired real-world foggy image data. Image blocks that are pseudo-clear image data. Calculate weights for the loss.
8. An image dehazing system based on feature aggregation and prior feature compensation, characterized in that, include: The image data construction module is used to collect real-world foggy image data, construct pseudo-clear image data based on the real-world foggy image data, and construct unpaired real-world foggy image data using a data augmentation method based on a diffusion model. The feature extraction and aggregation module is used to extract features from real-world foggy image data using a convolutional neural network (CNN). The extracted features are then processed using a physical-driven feature aggregation method to perform preliminary dehazing, resulting in preliminary dehazed image data. The prior feature compensation module is used to obtain multiple sets of image features using various prior image processing methods, and to perform feature compensation on the preliminary dehazed image data. The fusion and stitching semi-supervised learning module is used to train an image dehazing model using feature-compensated dehazed image data, pseudo-sharp image data, and unpaired real-world foggy image data. The trained image dehazing model is then used to obtain the dehazed image.
9. An electronic device, characterized in that, include: Memory, storing at least one instruction; and The processor executes instructions stored in the memory to implement the image dehazing method based on feature aggregation and prior feature compensation as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in an electronic device to implement the image dehazing method based on feature aggregation and prior feature compensation as described in any one of claims 1 to 7.