Image defogging and detection tool generation method and device, electronic equipment and storage medium

Abstract

The invention discloses an image defogging and detection tool generation method and device, electronic equipment and a storage medium, and the method and device are applied to the electronic equipment and are used for generating an image defogging and detection tool based on a unified collaborative optimization framework. The image defogging and detection tool is a defogging-detection closed-loop feedback system comprising a detection perception defogger and an adaptive detector. The method specifically comprises the steps of obtaining a training data set for training an image defogging and detection tool, wherein the training data set comprises a first data set and a second data set; alternate training is carried out based on the two data sets, and a detection perception demister and a self-adaptive detector are obtained. According to the scheme, a tool for defogging and detecting the image collected by the visual system can be generated, and after the tool is applied to the visual system, effective defogging processing on the image can be effectively improved. Therefore, adverse effects of haze weather on accuracy and reliability of downstream detection tasks can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and more particularly, to a method and device for generating an image defogging and detection tool, an electronic device and a storage medium. BACKGROUND

[0002] Visual systems play an important role in the current traffic control, security, production and other fields, and are increasingly widely deployed and applied. In practical applications, visual systems not only need to work in good weather conditions, but also need to play their due role in fog and haze conditions. Among them, fog and haze weather, due to its widespread existence and serious interference on imaging quality, has become one of the most challenging environmental factors. When fog and haze occur, suspended particles (such as water vapor, dust, pollutants) in the atmosphere will scatter and absorb light, resulting in degradation phenomena such as contrast reduction, color distortion, detail blurring and even information loss in collected images, which can seriously weaken the accuracy and reliability of downstream detection tasks. SUMMARY

[0003] In view of this, the present application provides a method and device for generating an image defogging and detection tool, an electronic device and a storage medium, for generating a tool for defogging processing of images collected by a visual system, so that users can apply it to the visual system to avoid the impact of fog and haze weather on the accuracy and reliability of downstream detection tasks.

[0004] To achieve the above purpose, the present scheme is as follows:

[0005] A method for generating an image defogging and detection tool is applied to an electronic device, for generating an image defogging and detection tool based on a unified collaborative optimization framework, the image defogging and detection tool being a defogging-detection closed-loop feedback system including a detection-aware defogger and an adaptive detector, the method comprising the steps of:

[0006] obtaining a training data set for training the image defogging and detection tool, the training data set including a first data set and a second data set;

[0007] performing model training based on the first data set to obtain the detection-aware defogger;

[0008] performing model training based on the second data set to obtain the adaptive detector;

[0009] When performing model training, the two are alternately trained to achieve collaborative optimization of the detection-aware defogger and the adaptive detector.

[0010] Optionally, the first data set comprises a real fog image data set and a real clear image data set; and the second data set comprises a clear image data set with detection annotation and a haze image data set without annotation.

[0011] Optionally, the model training based on the first data set obtains the detection-aware dehazer.

[0012] The cyclic consistent generative adversarial network is constructed.

[0013] The cyclic consistent generative adversarial network is trained based on the first data set to obtain the detection-aware dehazer.

[0014] Optionally, the cyclic consistent generative adversarial network comprises a haze adding network, a haze removing network, a haze adding discriminator, and a haze removing discriminator.

[0015] Optionally, a plurality of loss functions are used when generating the haze removing network, and the plurality of loss functions comprise a generative adversarial loss function, a cyclic consistent loss function, an identity loss function, and a detection-aware loss function.

[0016] Optionally, the adaptive detector comprises a student detector and a teacher detector with the same network structure.

[0017] Optionally, the unified collaborative optimization framework comprises two collaborative components, which comprise a detection-aware dehazer and an adaptive detector.

[0018] A generation device of an image dehazing and detection tool is applied to an electronic device and is used to generate an image dehazing and detection tool based on a unified collaborative optimization framework. The image dehazing and detection tool method is a dehazing-detection closed-loop feedback system comprising a detection-aware dehazer and an adaptive detector. The generation device comprises:

[0019] A data set acquisition module is designed to acquire a training data set for training the image dehazing and detection tool. The training data set comprises a first data set and a second data set.

[0020] A first training module is designed to perform model training based on the first data set to obtain the detection-aware dehazer.

[0021] A second training module is designed to perform model training based on the second data set to obtain the adaptive detector.

[0022] When the first training module and the second training module perform model training, they perform alternating training to realize collaborative optimization of the detection-aware dehazer and the adaptive detector.

[0023] An electronic device, comprising at least one processor and a memory connected with the processor, wherein:

[0024] The memory is used to store computer programs or instructions;

[0025] The processor is used to execute the computer programs or instructions, so that the electronic device implements the generation method of the image defogging and detection tool as described above.

[0026] A computer-readable storage medium, carrying one or more computer programs, which can be executed by the electronic device, so that the electronic device implements the generation method of the image defogging and detection tool as described above.

[0027] From the above technical solution, the present application discloses a generation method, device, electronic device and storage medium of an image defogging and detection tool, which is applied to an electronic device and used to generate an image defogging and detection tool based on a unified collaborative optimization framework. The image defogging and detection tool is a defogging-detection closed-loop feedback system including a detection-aware defogger and an adaptive detector. Specifically, a training data set for training the image defogging and detection tool is obtained, the training data set includes a first data set and a second data set; model training is performed based on the first data set to obtain a detection-aware defogger; model training is performed based on the second data set to obtain an adaptive detector; during model training, the two are alternately trained to realize collaborative optimization of the detection-aware defogger and the adaptive detector. Through the above scheme, a tool for defogging and detecting the image collected by a visual system can be generated. After being applied to the visual system, the tool can effectively improve the effective defogging processing of the image. Thus, the adverse effects of foggy weather on the accuracy and reliability of downstream detection tasks can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0029] Figure 1 A flowchart of a generation method of an image defogging and detection tool according to an embodiment of the present application;

[0030] Figure 2 A schematic diagram of an image defogging and detection tool according to an embodiment of the present application;

[0031] Figure 3aA schematic diagram of a physical perception generator of an embodiment of the present application

[0032] Figure 3b A schematic diagram of a PAUForme of an embodiment of the present application

[0033] Figure 3c A schematic diagram of a PAU block of an embodiment of the present application

[0034] Figure 4a A schematic diagram of a detection effect of an image defogging and detection tool of the present application

[0035] Figure 4b A schematic diagram of a defogging effect of an image defogging and detection tool of the present application

[0036] Figure 5 A block diagram of a generation device of an image defogging and detection tool of an embodiment of the present application

[0037] Figure 6 A block diagram of an electronic device of an embodiment of the present application DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0039] To solve the problem of haze affecting image processing, image defogging technology has emerged. The core goal is to restore a clear image close to the real scene from a single haze image. Early methods are mainly based on physical models (such as the classic atmospheric scattering model), which achieve restoration by estimating the transmission map and global atmospheric light. In recent years, with the development of deep learning, end-to-end defogging models based on convolutional neural networks (CNN) and generative adversarial networks (GAN) have made significant breakthroughs in visual quality, generating high-contrast, natural color, and detailed images, and are widely used in photography enhancement, video monitoring beautification, and other human visual perception-centered scenarios.

[0040] However, when the dehazing technology is introduced into the human-machine collaborative safety-critical system (such as automatic driving, power inspection robot, border intelligent monitoring, etc.), its evaluation standard must be fundamentally changed. In these scenarios, the image is not only for people to "see", but also for machines to "use". The target detector needs to accurately identify key objects such as vehicles, pedestrians, obstacles, and equipment defects from the dehazed image, and output accurate bounding boxes and class confidence. At this time, semantic fidelity, that is, whether the geometric structure, spatial relationship, and texture semantics of the object in the image are truly preserved, becomes as important as, or even more important than, visual clarity.

[0041] Unfortunately, a large number of experiments show that the visually "beautiful" dehazing results often perform poorly in machine vision tasks. For example, the dehazing model often over-sharpens the edges to improve the contrast, resulting in artifacts in the object outline; or introduces unnatural color bias to restore color, which interferes with the detector's judgment of color-sensitive targets such as traffic signs and signal lights; even worse, "brainstorm" non-existent structures (i.e. structure illusion) in the absence of a true reference, directly causing false detection. These seemingly minor distortions can be amplified into fatal errors in target detection tasks - such as mistaking a blurred guardrail for a pedestrian, or identifying a light spot in the fog as a vehicle, which seriously threatens system safety.

[0042] The root cause is that traditional dehazing methods generally use a separate pipeline architecture of "dehaze-then-detect". This architecture regards dehazing as an independent image preprocessing step, and its optimization target only focuses on image quality indicators at the pixel or perception level, completely ignoring the inherent needs of semantic consistency for downstream detection tasks. The dehazing module has no awareness of the key features required by the detector (such as target boundary continuity, local texture discriminability, and context semantic relevance) when running, while the detector is forced to reason under "black box" input and cannot impose any feedback constraints on the dehazing process. This misalignment of task and optimization targets results in a significant domain gap between the dehazed image and the real clear image in the feature space, causing the detector trained on clear images to perform poorly on dehazed images.

[0043] Therefore, there is an urgent need for an adaptive method that can restore the visual quality of images under foggy conditions while actively preserving or even enhancing the target detection effect, achieving the dual goals of "clear to see and accurate to recognize" for both humans and machines. In view of this, the present application provides the following specific technical solutions for generating an image dehazing and detection tool for image dehazing and detection under foggy conditions, in order to be applied to specific scenarios.

[0044] Figure 1A flowchart of a method for generating an image defogging and detection tool according to an embodiment of the present application.

[0045] As shown in Figure 1 The method for generating an image defogging and detection tool according to an embodiment of the present application is applied to an electronic device, and is used to generate an image defogging and detection tool capable of running on a corresponding platform based on a same coordination optimization framework. The image defogging and detection tool is a monitoring closed-loop feedback system including a detection-aware defogger and an adaptive detector, as shown in Figure 2 The electronic device can be understood as a computer, a server or a cloud platform having data computing capability and information processing capability. The method specifically includes the following steps:

[0046] S1, obtaining a training data set for training an image defogging and detection tool, the training data set including a first data set and a second data set.

[0047] The first data set includes a real foggy image data set and a real clear image data set, and the second data set includes a clear image data set with detection annotation and a haze image data set without annotation.

[0048] The first data set includes 7000 images collected from (Real-world Task-Driven Testing Set, RTTS) and (Unannotated Real-world Hazy Images, URHI) in the foggy image database RESIDE as the real foggy image data set, and more than 10000 images collected from the ADE20K database and (Outdoor Training Set, OTS) in RESIDE as the real clear image data set.

[0049] The second data set includes more than 10,000 images selected from VOC2012, and only retains the annotations of five categories: person, car, bus, bicycle and motorbike, as a clear image data set. In order to support the unpaired learning of the detection task, 5,900 images containing the five categories of instances are further selected from the above-mentioned real foggy image data set through visual confirmation, and an unannotated haze image data set is constructed.

[0050] S2, model training based on the first data set to obtain a detection-aware defogger.

[0051] When the model is trained based on the first data set, the basic framework of the cycle-consistent generative adversarial network CycleGAN is adopted, which includes a fogging network, a defogging network, a fogging discriminator and a defogging discriminator.

[0052] wherein the fogging network and the de-fogging network adopt the same network structure. In order to make the generator explicitly perceive the physical fog formation process, so as to simultaneously improve the de-fogging quality and detection compatibility, the application constructs a physics-aware generator which simultaneously acts as the de-fogging network and the fogging network. As shown in the figure, the physics-aware generator adopts a U-Net architecture, contains three encoder-decoder stages, skip connections, and introduces a ResNet module at the bottleneck layer to enhance the representation ability. Figure 3a

[0053] The fogging discriminator and the de-fogging discriminator also adopt the same network structure, and both rely on the publicly released Vision-aided-gan as the fogging discriminator and the de-fogging discriminator, wherein the cv_type is set to dino, and a DINO-based global discriminator is used to evaluate the overall structure and semantic consistency of the image.

[0054] The specific process of the fogging network and the de-fogging network as the generator is as follows:

[0055] A real clear image is input , and after fogging, a fogged image is generated, and then the image is input into the de-fogging network to generate a cyclic clear image . A real foggy image is input , and after de-fogging, a de-fogged image is obtained, and then it is input into the fogging network to generate a cyclic foggy image .

[0056] When training the de-fogging network, the following loss functions are used:

[0057] (1) The generative adversarial loss function is:

[0058] (2)

[0059] In the formula, denotes the de-fogging network, denotes the fogging network. and are respectively the discriminators in their respective fields for making the generated image more approximate to the real image.

[0060] (2) The cycle-consistency loss function is:

[0061] (3)

[0062] ​wherein, denotes the sum of the Manhattan distance between two items and the LPIPS distance.

[0063] (3) Identity loss function:

[0064] Adding the identity loss function is to make the defogging backbone network, while being able to defog, preserve the structure and details of the original image content as much as possible. The same applies to the haze adding network. For example: give the defogging network a clear image, the output of the defogging network should still be the clear image itself, so the identity loss function is expressed as follows:

[0065] (4)

[0066] (4) Detection-aware loss function:

[0067] Using only the above loss functions for training often produces visually appealing but suboptimal defogging results for the detection task. The fundamental reason is that there is a mismatch between low-level visual attributes (such as contrast, color) and high-level semantic cues (such as object boundaries, scale-invariant features) required by the detector. In addition, GAN-based optimization can introduce subtle artifacts that are difficult for humans to detect but harmful to detection performance.

[0068] To bridge this gap, a detection-aware loss function (detection-aware loss) is proposed, which explicitly preserves semantic information critical to detection during image translation. Specifically, the backbone network of the object detector (YOLOv7 in the implementation process of the present application) is used as a task-specific feature extractor. The core idea is that if the defogging network / haze adding network can preserve detection-related information, then the real clear image and its cyclic reconstruction result should have as small a feature distance as possible under the detector backbone network, while and the fake foggy image should have as large a feature distance as possible. As Figure 1 shown on the left side of the blue module, this is expressed as a contrast learning objective. For the cyclically reconstructed clear image , it is treated as an anchor, the real clear image is the positive sample, and the fake foggy image is the negative sample. For the defogging output (i.e., the fake clear image), a contrast triple is also constructed: taking as the anchor, the input real foggy image The negative samples are used to force feature differences, while the dehazing results generated using the Dark Channel Prior (DCP) are... As a positive sample, DCP can generally preserve object-level structural semantics that are crucial for detection, although it may introduce artifacts in the sky or highlighted areas. Therefore, DCP can serve as a reliable surrogate recovery result for detection in the absence of truly sharp images.

[0069] Considering the critical importance of YOLOv7's multi-scale features for object detection, representations are extracted from layers 24, 37, and 50 of its backbone network (corresponding to features ranging from fine-grained to semantic levels, respectively), and contrastive losses are computed at these stages. Taking cycle-consistent triples as an example, the detection-perceptual loss can be expressed as:

[0070] (5)

[0071] in, This indicates the YOLOv7 backbone network. This refers to the distance to Manhattan. This refers to the temperature parameter.

[0072] S3. Train the model based on the second dataset to obtain the adaptive detector, and train it alternately with the detection-aware defogging system.

[0073] In joint dehazing and detection tasks, a key challenge lies in the extreme scarcity of hazy images with detection annotations in real-world scenarios. While existing cross-domain detection methods typically aim to make detectors perform well directly on hazy inputs, their objectives are fundamentally different: ensuring that the dehazed output is detection-friendly requires the detector to adapt to the distribution of generated dehazed images. To address this, this application proposes an adaptive detector trained in an unpaired semi-supervised paradigm. Given a dataset of clear images with detection annotations... and an unlabeled haze image dataset .in and These represent the bounding box and the category label, respectively. First, a dehazing network is applied to a real haze image. The defogging result was obtained. Then input it into the detection module.

[0074] Inspired by the Mean Teacher method, this application constructs two identical detection networks: a student detector and a teacher detector. The student detector is optimized using gradient descent, while the teacher detector is updated using exponential moving average (EMA). The student detector is supervised using a standard detection loss on real, sharp images.

[0075] (6)

[0076] where, is the GIoU loss for predicting the bounding box, is the Focal loss for classification probability and target confidence.

[0077] For unannotated dehazed images , the teacher detector generates pseudo labels, i.e. keeping the predictions with high confidence as the supervision signal for the student detector. To alleviate the feature degradation or hallucination problem caused by imperfect dehazing, the original hazy images are also fed into the teacher detector, and the detection results are fused with the results from . The fused predictions are passed through non-maximum suppression (NMS) to generate robust pseudo labels. This dual-stream strategy combines the structural semantic information in dehazed images and the low-level robust cues in hazy images, significantly improving the reliability of pseudo labels under domain shift. The resulting unsupervised loss is:

[0078] (7)

[0079] In addition, during training, the teacher detector only processes dehazed images and original hazy images, while the student detector is mainly updated by clear images with real annotations. This domain mismatch can introduce bias and hinder alignment. To alleviate this problem, two complementary losses are introduced: the cycle supervision loss and the consistency loss . In an ideal case, a real clear image after being hazy and then dehazed, should get a reconstructed image which should be highly similar to the original image in both visual and semantic aspects. Therefore, can be supervised using the same detection annotations as , thus providing additional task-aware supervision signals for the student detector.

[0080] (6)

[0081] In addition, although and may have domain-level differences, their detection labels are the same. Therefore, the student detector should produce consistent predictions for the two inputs. To enforce this invariance, a consistency loss function is imposed:

[0082] (7)

[0083] These loss functions effectively bridge the domain gap between real and generated clear images, stabilize the detector training, and reduce overfitting to the dehazing artifacts. The teacher detector has the same initial weights as the student detector, and its weights are updated by EMA.

[0084] In addition, the unified co-optimization framework in the present application includes two co-optimization components, which include a detection-perception dehazing module and an adaptive detection module. Their goals are controlled by the following two composite loss functions, respectively:

[0085] (8)

[0086] (9)

[0087] In view of the fact that jointly minimizing the two losses under the unpaired setting is prone to cause unstable training, an alternating co-training strategy is adopted to decouple and coordinate the two goals. Firstly, the dehazing stage detection network is frozen, and its backbone network is used as a task-perception feature extractor to calculate , ensuring that the dehazed image retains the key semantics required for downstream detection; secondly, in the detection stage, the haze adding network and the dehazing network are frozen to generate the cyclically reconstructed clear image and dehazed image, and input them into the student and teacher detectors to achieve robust learning. This iterative co-optimization mechanism forms a closed-loop feedback system: the dehazer learns to generate images that are both visually realistic and semantically reliable, while the detector adaptively adapts to the specific characteristics of the dehazing domain. Ultimately, the method achieves a principled balance between human perception quality and detection practicality, successfully achieving the dehazing goal of "serving both the human eye and machine vision".

[0088] In one specific embodiment, the present application implements the above-mentioned image dehazing and detection tool generation through the following specific means.

[0089] 1. The input feature map with a size of (1, 3, 256, 256) is first processed by an input convolution to obtain a feature with a size of (1, 32, 256, 256), denoted as , the convolution kernel size is 3x3, the number of convolution kernels is 32, the step is 1, and the padding is 1;

[0090] 2. Then it is sent to the first PAUFormer. The structure of the PAUFormer is as shown in Figure 3b . The PAUFormer includes LayerNorm, Gated FFN and PAU Block.

[0091] 2.1, First, it is processed by the LayerNorm layer to become a feature map with a size of (1, 32, 256, 256), denoted as .

[0092] 2.3, the PAU block is processed, as shown in Figure 3c .

[0093] First , that is , through a global adaptive pooling GAP into a feature map with a size of (1, 32, 1, 1), and then using a convolution, ReLU, a convolution, and a Sigmoid activation function to change into a feature map with a size of (1, 32, 1, 1) . Among them, the first convolution kernel size is 1x1, the number of convolution kernels is 8, the step is 1, and the padding is 0. The second convolution kernel size is 1x1, the number of convolution kernels is 32, the step is 1, and the padding is 0.

[0094] At the same time , first through a convolution, a convolution, ReLU, a convolution, and a Sigmoid to change into a feature map with a size of (1, 32, 256, 256), denoted as , wherein the first convolution kernel size is 3x3, the number of convolution kernels is 32, the step is 1, and the padding is 1. The second convolution kernel size is 3x3, the number of convolution kernels is 4, the step is 1, and the padding is 1. The first convolution kernel size is 3x3, the number of convolution kernels is 32, the step is 1, and the padding is 1.

[0095] According to , a feature map with a size of (1, 32, 256, 256) can be obtained, denoted as , the input and output of the PAU block remain unchanged. At the same time, the output is recorded as .

[0096] 2.4, a feature map with a size of (1, 32, 256, 256) is obtained , wherein is a learnable scaling factor, and the initial value is set to 0.

[0097] 2.5, , through LayerNorm to change into a feature map with a size of (1, 32, 256, 256) .

[0098] 2.6, , through a convolution to change into a feature map with a size of (1, 64, 256, 256) . The convolution kernel size is 1x1, the number of convolution kernels is 64, the step is 1, and the padding is 0.

[0099] 2.7、 After Gated FFN, the feature map size is (1, 32, 256, 256) , the Gated FFN is first chunk operation to change it into two feature maps with size (1, 32, 256, 256), and then element multiplication is performed.

[0100] 2.8, get a feature map with size (1, 32, 256, 256), , is a learnable scaling factor, and the initial value is set to 0.

[0101] The input and output sizes of the PAU Former remain unchanged.

[0102] 3, after the first PAU Former, the feature is still (1, 32, 256, 256).

[0103] 3.1, down-sampling is performed, the feature becomes (1, 32, 128, 128), and then the PAU Former is passed, and the subsequent operations are the same as above.

[0104] When the feature becomes (1, 256, 32, 32), it passes through a residual network containing 9 residual blocks. Each residual block contains two sub-modules: conv_block and conv_fin

[0105] Each of the residual blocks includes two sub-network structures: a main convolution path (conv_block) and a residual post-processing path (conv_fin). Among them, the main convolution path includes in turn:

[0106] (1) ReflectionPad2d, which is used to symmetrically fill the four sides of the input feature map with 1 pixel to avoid introducing boundary artifacts due to convolution operation;

[0107] (2) 3x3 Conv2d, with 256 convolution kernel numbers, step size 1, and padding 1

[0108] (3) GroupNorm, which divides 256 channels into 64 groups, each group with 4 channels, performs group normalization, and enables learnable affine transformation parameters.

[0109] After completing the main convolution path processing, the output result is added to the original input feature map element by element to form a residual connection. Subsequently, the residual result is input to the residual post-processing path, which includes in turn:

[0110] (1) ReLU;

[0111] (2) Another reflection padding layer (ReflectionPad2d), also performing 1-pixel symmetric padding;

[0112] (3) A second 3x3 convolution layer (Conv2d), with 256 convolution kernel numbers, a step of 1, and padding of 1;

[0113] (4) A second group normalization layer (GroupNorm), with the same parameter settings as described above.

[0114] The input and output sizes of the residual block remain unchanged.

[0115] Then, upsampling and PAUFormer are performed, so that the final output becomes a feature of (1, 32, 256, 256), and finally an output convolution is performed to become an image of (1, 3, 256, 256). The convolution kernel size is 3x3, the number of convolution kernels is 32, the step is 1, and the padding is 1.

[0116] As can be seen from the above technical solution, the embodiment provides a method for generating an image defogging and detection tool. The method is applied to an electronic device and is used to generate an image defogging and detection tool based on a unified collaborative optimization framework. The image defogging and detection tool is a defogging-detection closed-loop feedback system including a detection-aware defogger and an adaptive detector. The generation method specifically includes obtaining a training data set for training the image defogging and detection tool, the training data set including a first data set and a second data set; performing model training based on the first data set to obtain the detection-aware defogger; performing model training based on the second data set to obtain the adaptive detector; and alternately training the detection-aware defogger and the adaptive detector during model training to achieve collaborative optimization of the detection-aware defogger and the adaptive detector. Through the above scheme, a tool for defogging and detecting images collected by a visual system can be generated. After the tool is applied to the visual system, effective defogging of images can be improved. Thus, the adverse effects of foggy weather on the accuracy and reliability of downstream detection tasks can be avoided.

[0117] When the image defogging and detection tool described above is applied to the corresponding visual system, good detection effects can be obtained, as shown in FIG. 8, and good defogging effects can be obtained, as shown in FIG. 9, Figure 4a Figure 4b

[0118] ​​The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0119] While operations are depicted in a particular, sequential order, this should not be understood as requiring or

[0120] It is to be understood that the steps of the methods recited in the method embodiments of the present disclosure can be carried out in a different order and / or concurrently with each other. Additional steps can also be employed.

[0121] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0122] Figure 5 A block diagram of an image defogging and detection tool generation apparatus according to an embodiment of the present disclosure.

[0123] As Figure 5As shown, the generation device provided by the embodiment of the present application is applied to an electronic device, and is used for generating an image defogging and detection tool capable of running on a corresponding platform based on a same coordination optimization framework. The generation device specifically comprises a data set acquisition module 10, a first training module 20, and a second training module 30.

[0124] The data set acquisition module is used for acquiring a training data set for training the image defogging and detection tool. The training data set comprises a first data set and a second data set.

[0125] The first data set comprises a real foggy image data set and a real clear image data set, and the second data set comprises a labeled clear image data set and an unlabeled foggy image data set.

[0126] The first training module is used for performing model training based on the first data set to obtain a detection-aware defogger.

[0127] When the model training is performed based on the first data set, the basic framework of a cycle-consistent generative adversarial network (CycleGAN) is adopted. The defogger comprises a fogging network, a defogging network, a fogging discriminator, and a defogging discriminator.

[0128] The fogging network and the defogging network adopt a same network structure. In order to enable the generator to explicitly perceive a physical fog formation process, thereby simultaneously improving the defogging quality and detection compatibility, a physics-aware generator is constructed to simultaneously serve as the defogging network and the fogging network. As shown in FIG. a, the physics-aware generator adopts a U-Net architecture, comprises three encoder-decoder stages, skip connections, and introduces a ResNet module at a bottleneck layer to enhance the representation ability. Figure 2

[0129] The fogging discriminator and the defogging discriminator also adopt a same network structure, and both depend on a publicly released Vision-aided-gan as the fogging discriminator and the defogging discriminator. A DINO-based global discriminator (DINO-based global discriminator) is set as cv_type=dino to evaluate the overall structure and semantic consistency of an image.

[0130] The second training module is used for performing model training based on the second data set to obtain an adaptive detector, and the adaptive detector is alternately trained with the detection-aware defogger.

[0131] ​In the joint dehazing and detection task, a key challenge lies in the extreme scarcity of hazy images with detection annotations in real-world scenarios. While existing cross-domain detection methods usually aim to let the detector perform well directly on hazy inputs, the goal of DeDetect is essentially different: it hopes to ensure that the output after dehazing is friendly to the detection task, which requires the detector to adapt to the distribution of the generated dehazed images. To this end, the present application proposes an adaptive detector trained in a unpaired semi-supervised paradigm. Given a clear image dataset with detection annotations , and an unannotated hazy image dataset . Wherein and represent the bounding box and class label respectively. First, the dehazing network is applied to the real hazy image , to obtain the dehazing result , which is then input into the detection module.

[0132] As can be seen from the above technical solution, the embodiment provides a generation device of an image dehazing and detection tool, which is applied to generating an image dehazing and detection tool based on a unified collaborative optimization framework. The image dehazing and detection tool is a dehazing-detection closed-loop feedback system including a detection-aware dehazer and an adaptive detector. The generation device specifically acquires a training dataset for training the image dehazing and detection tool, the training dataset including a first dataset and a second dataset; performs model training based on the first dataset to obtain the detection-aware dehazer; performs model training based on the second dataset to obtain the adaptive detector; and alternately trains the two during the model training to achieve collaborative optimization of the detection-aware dehazer and the adaptive detector. Through the above scheme, a tool for dehazing and detecting the images collected by a visual system can be generated. After the tool is applied to the visual system, the image can be effectively dehazed. Thus, the adverse effects of foggy weather on the accuracy and reliability of the downstream detection task can be avoided.

[0133] The units described in the embodiments of the present disclosure can be implemented in the form of software or in the form of hardware. Among them, the name of the unit does not constitute a limitation on the unit itself in some cases. For example, the first acquisition unit can also be described as a unit for acquiring at least two internet protocol addresses.

[0134] The functions described above in this paper can be executed at least partially by one or more hardware logic components. For example, non-limitingly, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0135] Figure 6 This is a block diagram of an electronic device according to an embodiment of this application.

[0136] The following is for reference. Figure 6 This document illustrates a structural diagram suitable for implementing the electronic device in the embodiments of this disclosure. The terminal device in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. This electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0137] The electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from an input device 606 into a random access memory (RAM) 603. The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0138] Typically, the following devices can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various devices are shown in the figures, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0139] This application also provides an embodiment of a computer-readable storage medium.

[0140] The computer readable storage medium described above is applied to an electronic device and carries one or more computer programs, when the one or more computer programs are executed by the electronic device, the electronic device generates an image defogging and detection tool based on a unified collaborative optimization framework, and the image defogging and detection tool is a defogging-monitoring closed-loop feedback system including a detection-aware defogger and an adaptive detector. Specifically, a training data set for training the image defogging and detection tool is obtained, the training data set includes a first data set and a second data set; model training is performed based on the first data set to obtain the detection-aware defogger; model training is performed based on the second data set to obtain the adaptive detector; and during model training, the two are alternately trained to achieve collaborative optimization of the detection-aware defogger and the adaptive detector. Through the above scheme, a tool for defogging the image collected by the vision system can be generated, and after the tool is applied to the vision system, the image can be effectively defogged. Thus, the adverse effects of foggy weather on the accuracy and reliability of downstream detection tasks can be avoided.

[0141] It should be noted that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0142] In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination thereof.

[0143] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0144] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to cover all the preferred embodiments and all the changes and modifications falling within the scope of the embodiments of the present application.

[0145] Finally, it should also be noted that, in this document, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or terminal device including the element.

[0146] The above describes the technical solutions provided by the present application in detail. The principles and implementation manners of the present application are described by applying specific examples in this document. The above description of the embodiments is only for helping to understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for generating an image defogging and detection tool, applied to an electronic device, for generating an image defogging and detection tool based on a unified co-optimization framework, the image defogging and detection tool being a defogging-detection closed-loop feedback system including a perception defogger and an adaptive detector, characterized in that, The generation method comprises the steps of: obtaining a training data set for training the image defogging and detection tool, the training data set comprising a first data set and a second data set; based on the first data set, model training is performed to obtain the detection-aware defogger; based on the second data set, model training is performed to obtain the adaptive detector; during model training, the two are alternately trained to achieve the collaborative optimization of the detection-aware defogger and the adaptive detector.

2. The generation method of claim 1, wherein, The first data set comprises a real foggy image data set and a real clear image data set; the second data set comprises a clear image data set with detection annotation and a foggy image data set without annotation.

3. The generation method of claim 1, wherein, The first data set is used to train the model to obtain the detection-aware defogger, wherein a cycle-consistent generative adversarial network is constructed; the cycle-consistent generative adversarial network is trained based on the first data set to obtain the detection-aware defogger.

4. The generation method of claim 3, wherein, The cycle-consistent generative adversarial network comprises a fogging network, a defogging network, a fogging discriminator, and a defogging discriminator.

5. The generation method of claim 4, wherein, A plurality of loss functions are used in generating the defogging network, the plurality of loss functions comprising a generative adversarial loss function, a cycle-consistent loss function, an identity loss function, and a detection-aware loss function.

6. The generation method of claim 1, wherein, The adaptive detector comprises a student detector and a teacher detector with the same network structure.

7. The generation method of claim 1, wherein, The unified collaborative optimization framework comprises two collaborative components, including a detection-aware defogging module and an adaptive detection module.

8. An apparatus for generating an image defogging and detection tool, applied to an electronic device, configured to generate an image defogging and detection tool based on a unified co-optimization framework, the image defogging and detection tool being a defogging-detection closed-loop feedback system comprising a perception defogger and an adaptive detector, characterized in that, The generation device comprises: a data set acquisition module designed to obtain a training data set for training the image defogging and detection tool, the training data set comprising a first data set and a second data set; a first training module designed to train the model based on the first data set to obtain the detection-aware defogger; a second training module designed to train the model based on the second data set to obtain the adaptive detector; during model training of the first training module and the second training module, the two are alternately trained to achieve the collaborative optimization of the detection-aware defogger and the adaptive detector.

9. An electronic device, comprising: The electronic device comprises at least one processor and a memory connected to the processor, wherein: the memory is used to store computer programs or instructions; the processor is used to execute the computer programs or instructions to enable the electronic device to implement the generation method of the image defogging and detection tool according to any one of claims 1-7.

10. A computer readable storage medium, characterized in that, The storage medium carries one or more computer programs, which can be executed by the electronic device, so as to enable the electronic device to implement the generation method of the image defogging and detection tool according to any one of claims 1-7.

Images