An image dehazing method driven by object detection task

Abstract

This invention relates to the field of image processing technology, addressing the technical problem of poor performance of traditional image dehazing methods in high-quality restoration applications. Specifically, it relates to an image dehazing method driven by object detection tasks. The method includes: inputting a foggy image into a pre-trained object detection network to obtain preliminary object detection prediction results; generating a detection guidance image based on the prediction results; constructing an image dehazing network and inputting the detection guidance image and the foggy image into the image dehazing network to obtain a dehazed image; introducing a high-order task loss function that deeply integrates the object detection task and the dehazing task into the object detection network; and inputting the dehazed image into the object detection network to generate the final object detection result. This invention enables high-quality restoration of images under adverse weather conditions while significantly improving the accuracy of object detection tasks.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image dehazing method driven by object detection task. Background Technology

[0002] Images acquired by outdoor vision systems often suffer severe degradation in contrast and visibility under extreme weather conditions such as fog, heavy rain, and sandstorms. This degradation erodes image quality in multiple dimensions, weakening contrast and sharpness, distorting colors, erasing details, and even burying crucial information, posing significant obstacles to subsequent feature extraction, analysis, and understanding. In target detection tasks, degraded images make target features blurry and difficult to discern, severely impacting the accuracy and robustness of detection algorithms, leading to frequent false negatives and missed detections. Taking military target detection as an example, battlefield smoke or rain can significantly reduce image sharpness and obscure targets due to atmospheric impurities, causing a sharp decline in detection accuracy. Such data distortion has fatal consequences in real-world applications such as military operations. Therefore, extracting effective and robust visual features under extreme weather conditions is crucial for subsequent detection tasks.

[0003] Academic and industrial researchers have focused on visual feature representation and object detection technologies in extreme interference scenarios, particularly image inpainting and object detection under adverse weather conditions. Current research can be broadly categorized into three types: first, training and testing directly on foggy images; second, preprocessing foggy images using existing dehazing algorithms before detection; and third, exploring a unified paradigm that connects low-order image dehazing with high-order object detection tasks. For example, DSNet, TogetherNet, and DH-YOLO learn clean and clear features by adding a feature recovery branch, with the detection backbone network sharing weights with the feature extraction layer of this branch to improve detection performance. However, existing methods often have significant limitations. Some methods focus solely on dehazing, neglecting the improvement in detection accuracy; others overemphasize detection performance, ignoring the quality of the restored image, resulting in poor performance in applications requiring high-quality restoration results. Achieving a precise joint optimization mechanism between low-order and high-order tasks remains a complex and unsolved problem. Therefore, designing a unified optimization paradigm has become a crucial research direction in this field. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an image dehazing method driven by target detection tasks, which solves the technical problem that traditional image dehazing methods perform poorly in high-quality restoration application scenarios.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: an image dehazing method driven by an object detection task, comprising the following steps:

[0006] S1. Input the foggy image into the pre-trained target detection network to obtain preliminary target detection prediction results;

[0007] S2. Generate a detection guidance image based on the target detection prediction result, wherein the pixel value of the target region is set to category ID+1 and the background region is set to 0;

[0008] S3. Construct an image dehazing network, and input the detection guidance image and the foggy image into the image dehazing network to obtain the dehazed image;

[0009] The image dehazing network adopts a U-shaped architecture with skip connections as the backbone network, including a downsampling module for extracting features from foggy images and detection guidance images and outputting fused features, and an upsampling module for gradually restoring details and features in foggy images and generating dehazed images based on fused features, wherein the downsampling module and the upsampling module are configured with multiple layers;

[0010] S4. Introduce a high-order task loss function in the object detection network that deeply integrates the object detection task with the dehazing task;

[0011] S5. Input the dehazed image into the object detection network to generate the final object detection result.

[0012] Furthermore, in step S1, the specific process includes:

[0013] S11. YOLOv7-tiny was selected as the detector for the object detection task for pre-training, and the parameters of the detector were frozen during the process of updating the image dehazing network.

[0014] S12. Input the foggy image into the aforementioned frozen detector to obtain preliminary target detection prediction results.

[0015] Furthermore, each layer of the downsampling module and the upsampling module integrates a physically-aware feature enhancement module;

[0016] The feature enhancement module consists of a two-stage feature extractor. In each stage, the input features are normalized by a batch normalization layer to improve generalization ability. Finally, the output features and input features are connected by residuals to obtain dehazed features.

[0017] Furthermore, in the multi-layered downsampling and upsampling modules, a guided fusion module for extracting shallow information of the original features at a coarse-grained level is introduced in the downsampling module located in the second layer; and a guided attention module for extracting deep information of the original features at a fine-grained level is introduced in the upsampling module located in the second layer.

[0018] Furthermore, in step S3, the process of obtaining the dehazed image includes:

[0019] S31. The foggy image is input into a 3*3 convolutional layer for feature extraction. The foggy image and the detection guidance image are then processed by multiple downsampling modules to extract features and output fused features.

[0020] S32. Input the features extracted by the downsampling module into the symmetrical upsampling module, and use the feature enhancement module to gradually restore the details and features of the foggy image to obtain the defoggy image;

[0021] S33. Input the dehazed image output by the upsampling module into a 3*3 convolutional layer to generate the dehazed restored image.

[0022] Furthermore, the two stages of the feature enhancement module include:

[0023] Phase one includes:

[0024] The normalized features are input into a pointwise convolutional layer and a depthwise convolutional layer to extract features x;

[0025] The normalized features are input into a pointwise convolutional layer and a sigmoid function as the gating signal for feature x;

[0026] A pointwise convolutional layer is used to apply a gated signal to the adjusted feature representation of feature x;

[0027] Phase two includes:

[0028] The output features of stage one are input into the channel attention module to obtain the attention-weighted feature representation;

[0029] The attention-weighted features are input into the physics-based dehazing module, which incorporates an atmospheric scattering model and divides into two branches to predict the model's parameters: global atmospheric light A and the reciprocal of transmittance.

[0030] Branch 1 uses global average pooling to accelerate computation and reduces redundant information in the feature space. It then increases the number of channels through two pointwise convolutional layers, during which GELU is used as the activation function to obtain the prediction parameters.

[0031] Branch 2 increases the number of channels using pointwise convolutional layers, then inputs a 3x3 convolutional layer to extract deep features and uses GELU as the activation function to obtain prediction parameters.

[0032] Based on prediction parameters and According to the formula Interpretable dehazing features were calculated. Where I hazy(x) represents the fogging feature at feature x in the feature space; This is the transmittance estimate for the corresponding feature x in the feature space.

[0033] Furthermore, a guided fusion module is introduced into the downsampling module of the second layer, and the processing includes:

[0034] The guidance features g of the detection guidance image are extracted through pointwise convolutional layers and depthwise convolutional layers;

[0035] Perform upsampling to maintain the size consistency between the original feature f and the guiding feature g;

[0036] The guiding feature g and the dehazing features output by the feature enhancement module will be combined. Adding them together yields the fusion feature;

[0037] After the fused features are extracted through pointwise convolution and depthwise convolution, the weights are adjusted by the Sigmoid function, and the original feature f is preserved through residual connections, thereby achieving the extraction of shallow information of the original feature f at a coarse-grained level.

[0038] Furthermore, a guided attention module is introduced into the upsampling module of the second layer, and the processing includes:

[0039] Pointwise convolutional layers are applied to the original feature f and the guided feature g respectively to extract features, and the features are added and fused together;

[0040] The attention score is calculated using two ReLU activation functions and a pointwise convolutional layer.

[0041] The original feature f is multiplied by the attention score to enhance the feature representation of the key region, and used as the final output feature, thereby realizing the extraction of deep information of the original feature f at a fine level.

[0042] Further, step S4 includes:

[0043] Using MAE loss as image restoration loss L res To optimize the predicted dehazed image, the image restoration loss is expressed as:

[0044]

[0045] In the formula, N is the total number of training samples; J and These represent the clear image and the predicted restored image, respectively.

[0046] The detection loss of the original object detection network is used as the object detection loss L. res The expression for the object detection loss is:

[0047] L det =λbox L box +λ obj L obj +λ cls L cls

[0048] In the formula, L box L obj and L cls Let λ represent the localization loss, confidence loss, and classification loss, respectively; box , λ obj and λ cls This represents the corresponding weighting coefficient;

[0049] By using a weighted hyperparameter λ to balance the dehazing and object detection tasks, the dehazing process is encouraged to update in a direction more conducive to detection. The expression for the higher-order task loss function is:

[0050] L total =L res +λL det

[0051] In the formula, L res For target detection loss; L res This represents the image restoration loss.

[0052] By employing the above technical solution, the present invention provides an image dehazing method driven by object detection task, which has at least the following beneficial effects:

[0053] 1. This invention achieves high-quality restoration of images under severe weather conditions by using a training strategy guided by a fusion module, a attention module, a physical perception feature enhancement module, and a higher-order task loss, while significantly improving the accuracy of target detection tasks.

[0054] 2. This invention inputs the detection guidance image into the guidance fusion module and guidance attention module of the image dehazing network respectively, guiding the dehazing focus to the region containing the object of interest from two levels, coarse and fine, thereby improving the high-order scene understanding ability of the dehazing model to generate high-quality dehazing features.

[0055] 3. This invention introduces a high-order task loss function of the target detection network to deeply integrate the target detection task with the dehazing task. The target detection task drives the optimization of the feature extraction of the image dehazing network, so that the dehazing process is updated in a direction that is conducive to target detection. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0057] Figure 1 This is a flowchart of the image dehazing method in this invention;

[0058] Figure 2 This is a flowchart illustrating the process of obtaining target detection results using an image dehazing network in this invention;

[0059] Figure 3 This is a network structure diagram of the image dehazing network of the present invention;

[0060] Figure 4 This is a network structure diagram of the guided fusion module and the guided attention module in this invention. Detailed Implementation

[0061] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0062] This embodiment proposes an image dehazing method driven by object detection task. Through a training strategy guided by a fusion module, a attention module, a physical perception feature enhancement module, and a higher-order task loss, it achieves high-quality image restoration under adverse weather conditions while significantly improving the accuracy of object detection. Figure 1 As shown, the method includes the following steps:

[0063] S1. Input the foggy image into the pre-trained object detection network to obtain preliminary object detection prediction results. The specific process includes:

[0064] S11. YOLOv7-tiny was selected as the detector for the object detection task for pre-training, and the parameters of the detector were frozen during the process of updating the image dehazing network.

[0065] S12. Input the foggy image into the aforementioned frozen detector to obtain preliminary target detection prediction results.

[0066] S2. Generate a detection guidance image similar to a mask image based on the target detection prediction results, where the pixel value of the target area is set to category ID+1 and the background area is set to 0.

[0067] S3. Construct an image dehazing network, and input the detection guidance image and the foggy image into the image dehazing network to obtain the dehazed image, such as... Figure 2 As shown. Specifically, by inputting a foggy image into a dehazing network that integrates a Physically Aware Feature Enhancement Module (PFEB), physical priors are introduced into the feature space. For example... Figure 3As shown, the image dehazing network employs a U-shaped architecture with skip connections, comprising multiple layers of downsampling modules (encoders) and upsampling modules (decoders). Each layer integrates a physically-aware feature enhancement module (PFEB), specifically:

[0068] The image dehazing network employs a U-shaped architecture with skip connections as its backbone, comprising multi-layered downsampling and upsampling modules. Each layer of both the downsampling and upsampling modules integrates a physically-aware feature enhancement module (PFEB). The downsampling module extracts features from the foggy image and the detection guidance image, outputting fused features. The upsampling module progressively restores details and features from the foggy image to generate the dehazed image. In this embodiment, the process of obtaining the dehazed image includes:

[0069] S31. The foggy image is input into a 3*3 convolutional layer for feature extraction. The foggy image and the detection guidance image are then processed by multiple downsampling modules to extract features and output fused features.

[0070] S32. Input the features extracted by the downsampling module into the symmetrical upsampling module, and use the feature enhancement module (PFEB) to gradually restore the details and features of the foggy image to obtain the defoggy image;

[0071] S33. The dehazed image output by the upsampling module is fed into a 3*3 convolutional layer to further refine the features and generate the final dehazed restored image.

[0072] As a further innovative approach, the Feature Enhancement Module (PFEB) in this embodiment consists of a two-stage feature extractor. In each stage, the input features are normalized by a batch normalization layer to improve generalization ability. Finally, the output features and input features are concatenated via residuals to obtain the dehazed features. Specifically, Phase One includes:

[0073] The normalized features are input into a pointwise convolutional layer and a depthwise convolutional layer to extract features x; the normalized features are input into a pointwise convolutional layer and a sigmoid function as the gating signal for features x; a pointwise convolutional layer is used to apply the gating signal to adjust the feature representation of features x.

[0074] Phase Two includes:

[0075] The output features from Stage 1 are input into the Channel Attention (CA) module to obtain the attention-weighted feature representation. The attention-weighted features are then input into the Physical Perception Dehazing (PFEB) module. The PFEB module incorporates the atmospheric scattering model and is divided into two branches to predict the model's parameters: global atmospheric light A and the reciprocal of transmittance.

[0076] Branch 1 uses global average pooling to accelerate computation and reduces redundant information in the feature space. It then increases the number of channels through two pointwise convolutional layers, during which GELU is used as the activation function to obtain the prediction parameters.

[0077] To prevent the significant information loss caused by global average pooling, branch two first increases the number of channels using pointwise convolutional layers, then extracts deep features using a 3x3 convolutional layer and uses GELU as the activation function to obtain the prediction parameters.

[0078] Based on prediction parameters and According to the formula Interpretable dehazing features were calculated. Where I hazy (x) represents the fogging feature at feature x in the feature space; This is the transmittance estimate for the corresponding feature x in the feature space.

[0079] In the multi-layered downsampling and upsampling modules, a guided fusion module (GFB) is introduced in the second-layer downsampling module to extract shallow information of the original features at a coarse-grained level; a guided attention module (GAB) is introduced in the second-layer upsampling module to extract deep information of the original features at a fine-grained level. Specifically, the guided fusion module (GFB) is introduced in the second-layer downsampling module, such as... Figure 4 As shown. The processing procedure includes:

[0080] The guidance features g of the detection guidance image are extracted through pointwise convolutional layers and depthwise convolutional layers; an upsampling operation is performed to maintain the size consistency between the original features f and the guidance features g; the guidance features g and the dehazing features output by the feature enhancement module (PFEB) are then combined. The fused features are obtained by adding them together. After the fused features are extracted by pointwise convolution and depthwise convolution, the weights are adjusted by the Sigmoid function, and the original feature f is preserved by residual connection, thereby realizing the extraction of shallow information of the original feature f at the coarse-grained level.

[0081] Furthermore, a Guided Attention (GAB) module is introduced into the upsampling module of the second layer, such as... Figure 4 As shown. The processing procedure includes:

[0082] Pointwise convolutional layers are applied to the original feature f and the guiding feature g respectively to extract features and then the features are added and fused. Attention scores are calculated using two ReLU activation functions and pointwise convolutional layers. The original feature f is multiplied by the attention score to enhance the feature representation of key regions and used as the final output feature, thereby realizing the extraction of deep information of the original feature f at a fine level.

[0083] In this embodiment, the detection guidance image is input into the Guided Fusion Module (GFB) and Guided Attention Module (GAB) of the image dehazing network, respectively. This guides the dehazing focus to the region containing the object of interest at both coarse and fine levels, thereby improving the high-order scene understanding ability of the dehazing model and generating high-quality dehazing features.

[0084] S4. Introduce a high-order task loss function in the object detection network that deeply integrates the object detection task with the dehazing task. This embodiment introduces a high-order task loss function into the object detection network to deeply integrate the object detection task with the dehazing task. This allows the object detection task to drive the optimization of the feature extraction of the image dehazing network, updating it in a direction that favors object detection. This includes:

[0085] Using MAE loss as image restoration loss L res To optimize the predicted dehazed image, the image restoration loss is expressed as:

[0086]

[0087] In the formula, N is the total number of training samples; J and These represent the clear image and the predicted restored image, respectively.

[0088] The detection loss of the original object detection network (i.e., YOLOv7-tiny) is used as the object detection loss L. res The expression for the object detection loss is:

[0089] L det =λ box L box +λ obj L obj +λ cls L cls

[0090] In the formula, L box L obj and L cls Let λ represent the localization loss, confidence loss, and classification loss, respectively; box , λ obj and λ cls This represents the corresponding weighting coefficient;

[0091] By using a weighted hyperparameter λ to balance the dehazing and object detection tasks, the dehazing process is encouraged to update in a direction more conducive to detection. Based on ablation experiments on the loss weights, setting λ to 0.4 yields the best results. The expression for the higher-order task loss function is:

[0092] L total =L res +λLdet

[0093] In the formula, L res For target detection loss; L res This represents the image restoration loss.

[0094] S5. Input the dehazed image into the object detection network to generate the final object detection result. Specifically, input the dehazed image into the pre-trained object detection model YOLOv7-tiny with frozen parameters to generate the final object detection result.

[0095] This invention generates high-quality dehazing features by inputting the detection guidance image into the guidance fusion module and guidance attention module of the image dehazing network, respectively, guiding the dehazing focus to the region containing the object of interest at both coarse and fine levels, thereby improving the high-order scene understanding ability of the dehazing model.

[0096] This invention introduces a high-order task loss function of the object detection network to deeply integrate the object detection task with the dehazing task. The object detection task drives the optimization of the feature extraction of the image dehazing network, so that the dehazing process is updated in a direction that is conducive to object detection.

[0097] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, their descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0099] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A target detection task-driven image defogging method, characterized in that, The method includes the following steps: S1. Input the foggy image into the pre-trained target detection network to obtain preliminary target detection prediction results; S2. Generate a detection guidance image based on the target detection prediction result, wherein the pixel value of the target region is set to category ID+1 and the background region is set to 0; S3. Construct an image dehazing network, and input the detection guidance image and the foggy image into the image dehazing network to obtain the dehazed image; The image dehazing network adopts a U-shaped architecture with skip connections as the backbone network, including a downsampling module for extracting features from foggy images and detection guidance images and outputting fused features, and an upsampling module for gradually restoring details and features in foggy images and generating dehazed images based on fused features, wherein the downsampling module and the upsampling module are configured with multiple layers; Each layer of the downsampling module and the upsampling module integrates a physically-aware feature enhancement module; The feature enhancement module consists of a two-stage feature extractor. In each stage, the input features are normalized by a batch normalization layer to improve generalization ability. Finally, the output features and input features are connected by residuals to obtain dehazed features. The feature enhancement module consists of two phases: Phase one includes: The normalized features are input into a pointwise convolutional layer and a depthwise convolutional layer to extract features x; The normalized features are input into a pointwise convolutional layer and a sigmoid function as the gating signal for feature x; A pointwise convolutional layer is used to apply a gated signal to the adjusted feature representation of feature x; Phase two includes: The output features of stage one are input into the channel attention module to obtain the attention-weighted feature representation; The attention-weighted features are input into a physical perception defogging module, the physical perception defogging module introduces an atmospheric scattering model and is divided into two branches to respectively predict parameters of the model, i.e. global atmospheric light A and the reciprocal of transmittance ; Branch one uses global average pooling to accelerate the calculation, and reduce redundant information in the feature space, and then pass through two point convolution layers to increase the number of channels, during which GELU is used as the activation function to obtain the prediction parameters ; Branch 2 increases the number of channels using pointwise convolutional layers, then inputs a 3x3 convolutional layer to extract deep features and uses GELU as the activation function to obtain prediction parameters. ; Based on prediction parameters and According to the formula Interpretable dehazing features were calculated. ,in This represents the fogging feature at feature x in the feature space; This is for estimating the transmittance of the corresponding feature x in the feature space; S4. Introduce a high-order task loss function in the object detection network that deeply integrates the object detection task with the dehazing task; S5. Input the dehazed image into the object detection network to generate the final object detection result.

2. The image dehazing method according to claim 1, characterized in that, In step S1, the specific process includes: S11. YOLOv7-tiny was selected as the detector for the object detection task for pre-training, and the parameters of the detector were frozen during the process of updating the image dehazing network. S12. Input the foggy image into the aforementioned frozen detector to obtain preliminary target detection prediction results.

3. The image dehazing method according to claim 1, characterized in that, In the multi-layered downsampling and upsampling modules, a guided fusion module for extracting shallow information of original features at a coarse-grained level is introduced in the downsampling module located in the second layer; a guided attention module for extracting deep information of original features at a fine-grained level is introduced in the upsampling module located in the second layer.

4. The image dehazing method according to claim 1, characterized in that, In step S3, the process of obtaining the dehazed image includes: S31. The foggy image is input into a 3*3 convolutional layer for feature extraction. The foggy image and the detection guidance image are then processed by multiple downsampling modules to extract features and output fused features. S32. Input the features extracted by the downsampling module into the symmetrical upsampling module, and use the feature enhancement module to gradually restore the details and features of the foggy image to obtain the defoggy image; S33. Input the dehazed image output by the upsampling module into a 3*3 convolutional layer to generate the dehazed restored image.

5. The image dehazing method according to claim 3, characterized in that, A guided fusion module is introduced into the downsampling module of the second layer. The processing includes: Guiding features are extracted from the detected guiding images using pointwise convolutional layers and depthwise convolutional layers. ; Upsampling operation preserves original features With guiding features Dimensional consistency; Guide features Dehazing features output by the feature enhancement module Adding them together yields the fusion feature; After the fused features are extracted through pointwise convolution and depthwise convolution, the weights are adjusted using the Sigmoid function, and the original features are preserved through residual connections. This allows for the extraction of original features at a coarse-grained level. Shallow information.

6. The image dehazing method according to claim 3, characterized in that, A guided attention module is introduced into the upsampling module of the second layer. The processing includes: In original features and guiding features Each pointwise convolutional layer is applied to extract features, and the features are then added and fused. The attention score is calculated using two ReLU activation functions and a pointwise convolutional layer. Original features Multiplying the value by the attention score enhances the feature representation of key regions and serves as the final output feature, thereby enabling the extraction of original features at a fine level. The deeper information.

7. The image dehazing method according to claim 1, characterized in that, Step S4 includes: Using MAE loss as image restoration loss To optimize the predicted dehazed image, the image restoration loss is expressed as: ； In the formula, N is the total number of training samples; and These represent the clear image and the predicted restored image, respectively. The detection loss of the original object detection network is used as the object detection loss. The expression for the object detection loss is: ； In the formula, and These represent localization loss, confidence loss, and classification loss, respectively. , and This represents the corresponding weighting coefficient; Through weight hyperparameters To balance the dehazing and object detection tasks, and to encourage the dehazing process to update in a direction more conducive to detection, the expression for the higher-order task loss function is: ； In the formula, Loss for target detection; This represents the image restoration loss.

Images