Self-adaptive target feature enhanced real-time foggy day road vehicle target detection method

Abstract

The invention discloses a real-time foggy day road vehicle target detection method based on adaptive target feature enhancement, and relates to the technical field of image processing. The method comprises the steps of firstly constructing a training set; then designing a lightweight defogging module and a lightweight adaptive target feature enhancement module, and forming a real-time foggy day road vehicle target detection model together with the target detection network YOLOv5; training a lightweight defogging network step by step by using the constructed training set, and then training a lightweight adaptive target feature enhancement module and a target detection network YOLOv5 end to end; and inputting a foggy day road vehicle image into the constructed model, carrying out real-time detection on a vehicle target in the image, and outputting a detection result. According to the method, the detail features of the foggy day image can be fully restored by using the lightweight defogging network, the target detection performance of the road vehicle in the foggy day is remarkably improved by combining the lightweight adaptive target feature enhancement module, and the network model is relatively small in parameter quantity and meets the real-time requirement.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a real-time foggy road vehicle target detection method with adaptive target feature enhancement. Background Art

[0002] Driven by the Internet of Things, big data, cloud computing, artificial intelligence, and the new infrastructure, my country's smart transportation has entered a new stage of development. Smart transportation aims to provide innovative services for different transportation modes and traffic management, allowing users to use the transportation network more fully, safely, coordinately and efficiently. An important part of smart transportation is road vehicle detection. As a key technology, target detection technology can provide strong support for tasks such as vehicle trajectory tracking and traffic scene recognition, effectively alleviate the pressure of traffic police on road duty, enhance the level of information management, improve traffic operation efficiency, and ensure traffic safety.

[0003] Under good weather conditions, commonly used target detection models such as YOLOv5 can effectively solve the road vehicle detection task. However, road monitoring cameras are often disturbed by fog during the imaging process, which reduces the quality of the acquired images, loses the detailed features of the target objects in the image, reduces the color saturation, reduces the contrast, weakens the texture information, and blurs the contour edges. A large amount of information that is beneficial to target detection is contained in the degraded image in a more implicit way. Directly performing target detection on the unprocessed degraded image will lead to a significant decline in the performance of the target detection model. In response to the above problems, the existing technical solutions can be mainly divided into three categories: one is the method of defogging first and then detecting, the second is the domain adaptive target detection method, and the third is the method of combining image defogging and target detection.

[0004] The steps of the defogging-before-detection method are as follows: construct a synthetic foggy image dataset, train a defogging model on the synthetic foggy image dataset; use a pre-trained image defogging model to defog foggy images; use defogging images to train a target detection model; and send the fog-free image and the defogging image to the target detection model for target detection. This method is the earliest proposed technical solution. Although it can enhance the detail features of foggy images through defogging, thereby improving the target detection performance of foggy images, the generalization effect for real foggy images is poor due to the domain offset between synthetic foggy images and real foggy images.

[0005] In order to solve the problem caused by the inter-domain shift, a domain adaptive target detection method was subsequently proposed. The steps of this method are: construct a synthetic foggy image dataset, and pre-train the defogging model on the synthetic foggy image dataset; on the real foggy dataset, perform adaptive migration training from the synthetic domain to the real domain through physical prior loss or adversarial loss to narrow the gap between domains; use a domain adaptive defogging network to defog the foggy images; use fog-free images and defogging images to train the target detection model; send the defogging images to the target detection model for target detection. This method can improve the target detection effect of real foggy images, but like the method of defogging first and then detecting, it uses human visual senses as the judgment criteria to restore the image, and cannot fully utilize the information related to target detection in the foggy images, so the improvement ability is limited.

[0006] The joint image defogging and target detection method is the latest technical solution. The steps of this method are as follows: construct a synthetic foggy image dataset, pre-train the defogging model on the synthetic foggy image dataset; use fog-free images to train the target detection model; cascade the defogging network and the target detection network, and use fog-free images and synthetic foggy images to train the cascade network end-to-end by combining image defogging loss and target detection loss; send the foggy images to the cascade network for end-to-end target detection. This method can use the implicit information in foggy images to adaptively restore foggy images to images that are conducive to target detection, significantly improving the target detection performance of foggy images. However, due to the constraints of defogging loss, the target feature enhancement of the restored image is not thorough enough, and the number of network model parameters is usually large, which cannot meet the real-time requirements. Summary of the invention

[0007] In view of this, the present invention proposes a real-time foggy road vehicle target detection method with adaptive target feature enhancement. The method can more fully mine potential target features in foggy images and perform adaptive target feature enhancement, significantly improving the target detection performance of foggy road vehicles, and the network model parameters are small, meeting the real-time requirements.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] The real-time foggy road vehicle target detection method with adaptive target feature enhancement comprises the following steps:

[0010] Step 1: synthesize the corresponding image target detection datasets of foggy days with different concentrations based on the open source fog-free road vehicle target detection dataset, and form the defogging network training set and the overall network training set based on the open source fog-free road vehicle target detection dataset, the synthesized corresponding image target detection datasets of foggy days with different concentrations, and the open source real foggy road vehicle target detection dataset;

[0011] Step 2, constructing a real-time foggy road vehicle target detection model, wherein the real-time foggy road vehicle target detection model includes a lightweight defogging module, a lightweight adaptive target feature enhancement module, and a target detection network YOLOv5;

[0012] Step 3, use the defogging network training set to train the lightweight defogging module, the input of the lightweight defogging module is the image in the defogging network training set, and the output is the corresponding refined defogging image. After the training is completed, the current lightweight defogging module is saved;

[0013] Step 4, freeze the parameters of the lightweight defogging module, use the overall network training set to perform end-to-end training on the real-time foggy road vehicle target detection model, the input of the real-time foggy road vehicle target detection model is the image in the overall network training set, and the output is the category and coordinates of the corresponding vehicle target. After the training is completed, save the current real-time foggy road vehicle target detection model;

[0014] Step 5, input the foggy road vehicle image into the trained real-time foggy road vehicle target detection model, obtain the category and coordinates of the vehicle target in the foggy road vehicle image, and complete the real-time foggy road vehicle target detection with adaptive target feature enhancement.

[0015] Furthermore, the specific method of step 1 is:

[0016] Step 101, based on the open-source fog-free road vehicle target detection dataset, synthesize the corresponding image target detection datasets of foggy days with different concentrations:

[0017] I(x)=J(x)e -β(λ)d(x) +A(1-e -β(λ)d(x) );

[0018] Where x represents the pixel position of the image, J(x) represents the image in the open-source fog-free road vehicle target detection dataset, I(x) represents the corresponding image of the synthetic foggy day, β(λ) represents the atmospheric scattering coefficient, d(x) represents the depth map corresponding to J(x) estimated by the monocular image depth estimation algorithm MiDaS, and A represents the atmospheric light at infinity. By adjusting the values ​​of β(λ) and A, the corresponding image target detection datasets of foggy days with different concentrations are synthesized.

[0019] Step 102, the open source fog-free road vehicle target detection dataset and the corresponding image target detection dataset of synthetic fog with different concentrations are combined into a defogging network training set; the open source fog-free road vehicle target detection dataset, the corresponding image target detection dataset of synthetic fog with different concentrations and the open source real foggy road vehicle target detection dataset are combined into an overall network training set.

[0020] Furthermore, the lightweight defogging module in step 2 includes a first-stage encoder, a first-stage decoder, a second-stage encoder, a second-stage decoder, and an image cascade layer;

[0021] Among them, the first-level encoder includes a 1×1 convolution layer with a stride of 1 and an output channel number of 8, a first Relu activation function layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 8, a second Relu activation function layer, a first maximum pooling layer with a stride of 2 and a window size of 3×3, two 3×3 convolution layers with a stride of 1 and an output channel number of 16, a third Relu activation function layer, a second maximum pooling layer with a stride of 2 and a window size of 3×3, a 3×3 convolution layer with a stride of 1 and an output channel number of 32, and a fourth Relu activation function layer.

[0022] The first-stage decoder includes a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a fifth Relu activation function layer, a first cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a sixth Relu activation function layer, a first upsampling layer with a step size of 2, a second cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 8, a seventh Relu activation function layer, a second upsampling layer with a step size of 2, a 3×3 convolution layer with a step size of 1 and an output channel number of 3, and an eighth Relu activation function layer.

[0023] Among them, the output of the fourth Relu activation function layer in the first-stage encoder is the input of the 3×3 convolution layer with a step size of 1 and an output channel number of 32 in the first-stage decoder; the first cascade layer in the first-stage decoder is used to cascade the output of the second maximum pooling layer in the first-stage encoder with the output of the fifth Relu activation function layer in the first-stage decoder, and the second cascade layer in the first-stage decoder is used to cascade the output of the first maximum pooling layer in the first-stage encoder with the output of the first upsampling layer in the first-stage decoder;

[0024] The second-stage encoder includes a 1×1 convolution layer with a stride of 1 and an output channel number of 8, a ninth Relu activation function layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 8, a tenth Relu activation function layer, a third maximum pooling layer with a stride of 2 and a window size of 3×3, a third cascade layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 16, an eleventh Relu activation function layer, a fourth maximum pooling layer with a stride of 2 and a window size of 3×3, a fourth cascade layer, a 3×3 convolution layer with a stride of 1 and an output channel number of 32, and a twelfth Relu activation function layer.

[0025] The third cascade layer in the second-stage encoder is used to cascade the output of the first maximum pooling layer in the first-stage encoder with the output of the third maximum pooling layer in the second-stage encoder, and the fourth cascade layer in the second-stage encoder is used to cascade the output of the second maximum pooling layer in the first-stage encoder with the output of the fourth maximum pooling layer in the second-stage encoder;

[0026] The second stage decoder includes a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a thirteenth Relu activation function layer, a fifth cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a fourteenth Relu activation function layer, a third upsampling layer with a step size of 2, a sixth cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 8, a fifteenth Relu activation function layer, a fourth upsampling layer with a step size of 2, two 3×3 convolution layers with a step size of 1 and an output channel number of 3, and a sixteenth Relu activation function layer.

[0027] Among them, the output of the twelfth Relu activation function layer in the second-level encoder is the input of the 3×3 convolutional layer with a step size of 1 and an output channel number of 32 in the second-level decoder; the fifth cascade layer in the second-level decoder is used to cascade the output of the fifth Relu activation function layer in the first-level decoder, the output of the fourth maximum pooling layer in the second-level encoder, and the output of the thirteenth Relu activation function layer in the second-level decoder; the sixth cascade layer in the second-level decoder is used to cascade the output of the first upsampling layer in the first-level decoder, the output of the third maximum pooling layer in the second-level encoder, and the output of the third upsampling layer in the second-level decoder.

[0028] Furthermore, the lightweight adaptive target feature enhancement module in step 2 includes a hyperparameter prediction module, a guideable image processing module and a gating module.

[0029] Further, the hyperparameter prediction module includes a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a seventeenth Relu activation function layer, a fifth maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 32, an eighteenth Relu activation function layer, a sixth maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a nineteenth Relu activation function layer, a seventh maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 64, a twentieth Relu activation function layer, a eighth maximum pooling layer with a step size of 2 and a window size of 3×3, and a step size of 1 output channel number of 64, the twenty-first Relu activation function layer, the ninth maximum pooling layer with a stride of 2 and a window size of 3×3, the 3×3 convolution layer with a stride of 1 and an output channel number of 128, the twenty-second Relu activation function layer, a global average pooling layer, a first fully connected layer with an output neuron number of 256, and two parallel second fully connected layers and third fully connected layers; the number of output neurons of the second fully connected layer is 15, and the number of output neurons of the third fully connected layer is 6; wherein, the input of the 3×3 convolution layer with a stride of 1 and an output channel number of 16 is the refined dehazed image output by the lightweight dehazing module, the output of the second fully connected layer is the 15 hyperparameters in the derivable image processing module, and the output of the third fully connected layer is the 6 weight parameters in the gating module.

[0030] Furthermore, the guideable image processing module includes six disordered guideable image processors, namely, exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening, and color adjustment; the guideable image processing module receives the refined defogging image output by the lightweight defogging module and 15 hyperparameters output by the hyperparameter prediction module, and outputs the corresponding processed image;

[0031] The exposure adjustment is represented by the following mapping function:

[0032] J exposure (x) = 2 E ×J 精 (x);

[0033] Among them, J 精 (x) represents the refined dehazed image output by the lightweight dehazing module, J exposure (x) is the refined dehazed image after exposure adjustment, and E is the exposure adjustment hyperparameter;

[0034] The white balance is represented by the following mapping function:

[0035] J wb (x)=(W r r(x),Wg g(x),W b b(x));

[0036] Among them, J wb (x) is the fine dehazed image after white balance, W r , W g , W b Respectively represent the white balance hyperparameters of the r, g, and b color channels, r(x), g(x), and b(x) represent the refined dehazed image J 精 (x) pixel value in r, g, b color channels;

[0037] The gamma transform is represented by the following mapping function:

[0038]

[0039] Among them, J gamma (x) is the refined dehazed image after gamma transformation, γ represents the gamma transformation hyperparameter;

[0040] The contrast enhancement is represented by the following mapping function:

[0041] J contrast (x) = α × En (J 精 (x))+(1-α)×J 精 (x);

[0042] Among them, J contrast (x) is the refined dehazed image after contrast enhancement, α represents the contrast enhancement hyperparameter;

[0043]

[0044] Lum(J 精 (x))=0.3r(x)+0.59g(x)+0.11b(x);

[0045] Sigmoid() is a nonlinear activation function, Lum() represents the pixel brightness function, and En() represents the image brightness enhancement function;

[0046] The sharpening is represented by the following mapping function:

[0047] J sharpen (x) = J 精 (x)+λ1(J 精 (x)-Gaussian(J 精 (x)));

[0048] Among them, J sharpen (x) is the sharpened dehazed image, λ1 is the sharpening hyperparameter, and Gaussian() is the Gaussian filter function;

[0049] The color adjustment is represented by the following mapping function:

[0050]

[0051] Among them, J tone (x) is the refined dehazed image after color adjustment, t i Adjust the hyperparameters for the i+1th color, i=0,1,2,...,7, clip(y,0,1) represents the clipping function;

[0052] Furthermore, the gating module receives the six weight parameters output by the hyperparameter prediction module, and is used to perform adaptive weighted fusion on the images processed by the six disordered derivable image processors to obtain an adaptive enhanced image J. 强 (x):

[0053] J 强 (x) = w1J exposure (x)+w2J wb (x)+w3J gamma (x)+w4J contrast (x)+w5J sharpen (x)+w6J tone (x);

[0054] Among them, w1 is the exposure adjustment weight parameter, w2 is the white balance weight parameter, w3 is the gamma transformation weight parameter, w4 is the contrast enhancement weight parameter, w5 is the sharpening weight parameter, and w6 is the color adjustment weight parameter.

[0055] Furthermore, the specific method of step 3 is:

[0056] Step 301: Input the image in the defogging network training set into the 1×1 convolutional layer with a step size of 1 and an output channel number of 8 of the first-stage encoder in the lightweight defogging module, and the eighth ReLU activation function layer of the first-stage decoder of the lightweight defogging module outputs a coarse map K with a channel number of 3. 粗 (x), according to the coarse map K 粗 (x) Calculate the coarse dehazed image corresponding to the image in the dehazed network training set:

[0057] J 粗 (x) = K 粗 (x)×I'(x)-K 粗 (x)+1;

[0058] Where I'(x) represents the image in the dehazing network training set, I'(x) = I(x) or J(x), and I'(x) is an RGB image with 3 channels; J 粗(x) is the rough dehazed image;

[0059] According to the image cascade layer of the lightweight defogging module, the image in the defogging network training set is cascaded with the corresponding coarse defogging image output by the first-level decoder to obtain a cascade image with 6 channels. The cascade image is input into the 1×1 convolution layer with a step size of 1 and an output channel number of 8 in the second-level encoder of the lightweight defogging module. The sixteenth Relu activation function layer of the second-level decoder of the lightweight defogging module outputs a fine map K with a channel number of 3. 精 (x); According to the precise map K 精 (x) Calculate the refined dehazed image corresponding to the image in the dehazed network training set:

[0060] J 精 (x) = K 精 (x)×I'(x)-K 精 (x)+1;

[0061] Step 302, using the open source fog-free road vehicle target detection dataset as the defogging true value label, using the L1 loss function to calculate the loss value between the coarse defogging image and the corresponding defogging true value label, recorded as loss1, using the L2 loss function to calculate the loss value between the fine defogging image and the corresponding defogging true value label, recorded as loss2, and calculating the total loss value Loss corresponding to the lightweight defogging module: Loss = 0.4loss1+0.6loss2; according to the total loss value Loss, the lightweight defogging module is back-propagated and trained using the gradient descent method until the total loss value converges, ending the training and saving the current lightweight defogging module.

[0062] Furthermore, the specific method of step 4 is:

[0063] Step 401, freeze the parameters of the lightweight defogging module, input the images in the overall network training set into the real-time foggy road vehicle target detection model, the lightweight defogging module outputs the refined defogging image to the hyperparameter prediction module and the derivable image processing module, and the hyperparameter prediction module outputs 15 hyperparameters in the derivable image processing module and 6 weight parameters in the gating module;

[0064] The guideable image processing module obtains the refined defogging image after exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening and color adjustment according to the 15 hyperparameters output by the hyperparameter prediction module and the refined defogging image output by the lightweight defogging module;

[0065] The gating module performs weighted fusion on the refined dehazed image after exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening, and color adjustment output by the guideable image processing module according to the six weight parameters output by the hyperparameter prediction module to obtain the corresponding adaptive enhanced image;

[0066] The adaptive enhanced image is input into the target detection network YOLOv5, and the category and coordinates of the vehicle target corresponding to the image in the overall network training set are output. The category and coordinates of the vehicle target contained in the image in the overall network training set are used as the target detection true value label. According to the loss function of the target detection network YOLOv5 itself, the loss value between the output vehicle target category and coordinates and the target detection true value label is calculated. According to the loss value, the gradient descent method is used to perform back propagation training on the hyperparameter prediction module and the target detection network YOLOv5 until the loss value converges, the training is terminated, and the current real-time foggy road vehicle target detection model is saved.

[0067] Due to the adoption of the above technical solution, the beneficial effects of the present invention compared with the prior art are:

[0068] 1. The real-time foggy road vehicle target detection method with adaptive target feature enhancement proposed in the present invention can utilize target detection loss to adaptively enhance image target features, rather than simply restoring foggy degraded images to sensory images, thereby more fully utilizing potential information in foggy images that is beneficial to target detection and significantly improving the target detection performance of vehicles on foggy roads.

[0069] 2. The lightweight defogging network module proposed in the present invention adopts a multi-view structure and uses the principle of atmospheric scattering for defogging. It has excellent defogging effect, far exceeding traditional defogging methods such as dark channel prior defogging method.

[0070] 3. The lightweight adaptive target feature enhancement module proposed in the present invention can adaptively enhance the dehazed image by learning the hyperparameters of the image processing operation, and realize the adaptive fusion of different image processing operations through the gating module, effectively avoiding the problem of over-enhancement of the image caused by sequential image processing.

[0071] 4. The defogging module and adaptive target feature enhancement module proposed in the present invention both adopt a lightweight structure, which can realize real-time target detection of road vehicles in foggy weather, and also have an improvement effect on road vehicle target detection under natural weather conditions and adverse conditions such as sandstorms and dim light. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 The figure is an overall flow chart of the real-time foggy road vehicle target detection method with adaptive target feature enhancement in an embodiment of the present invention.

[0073] Figure 2 Schematic diagram of a real-time foggy road vehicle target detection model in an embodiment of the present invention.

[0074] Figure 3 Schematic diagram of a lightweight defogging module in an embodiment of the present invention.

[0075] Figure 4 Schematic diagram of a lightweight adaptive target feature enhancement module in an embodiment of the present invention.

[0076] Figure 5 The figure is a schematic diagram for comparing the visualization results of vehicle target detection on foggy roads by comparing the method and the embodiment of the present invention. DETAILED DESCRIPTION

[0077] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0078] Real-time foggy road vehicle target detection method with adaptive target feature enhancement, such as Figure 1 As shown, the following steps are included;

[0079] Step 1: synthesize the corresponding image target detection datasets of foggy days with different concentrations based on the open source fog-free road vehicle target detection dataset, and form the defogging network training set and the overall network training set based on the open source fog-free road vehicle target detection dataset, the synthesized corresponding image target detection datasets of foggy days with different concentrations, and the open source real foggy road vehicle target detection dataset;

[0080] Specifically, in this embodiment, the UA-DETRAC fog-free road vehicle target detection dataset and the RTTS real foggy target detection dataset are selected. Both datasets contain road vehicle targets of interest, as well as the categories and coordinates of the corresponding road vehicles.

[0081] Step 2: Build a real-time foggy road vehicle target detection model, such as Figure 2 As shown, the real-time foggy road vehicle target detection model includes a lightweight defogging module, a lightweight adaptive target feature enhancement module and a target detection network YOLOv5;

[0082] Step 3, use the defogging network training set to train the lightweight defogging module, the input of the lightweight defogging module is the image in the defogging network training set, and the output is the corresponding refined defogging image. After the training is completed, the current lightweight defogging module is saved;

[0083] Step 4, freeze the parameters of the lightweight defogging module, use the overall network training set to perform end-to-end training on the real-time foggy road vehicle target detection model, the input of the real-time foggy road vehicle target detection model is the image in the overall network training set, and the output is the category and coordinates of the corresponding vehicle target. After the training is completed, save the current real-time foggy road vehicle target detection model;

[0084] Step 5, input the foggy road vehicle image into the trained real-time foggy road vehicle target detection model, obtain the category and coordinates of the vehicle target in the foggy road vehicle image, and complete the real-time foggy road vehicle target detection with adaptive target feature enhancement.

[0085] Furthermore, the specific method of step 1 is:

[0086] Step 101, based on the open-source fog-free road vehicle target detection dataset, synthesize the corresponding image target detection datasets of foggy days with different concentrations:

[0087] I(x)=J(x)e -β(λ)d(x) +A(1-e -β(λ)d(x) );

[0088] Where x represents the pixel position of the image, J(x) represents the image in the open-source fog-free road vehicle target detection dataset, I(x) represents the corresponding image of the synthetic foggy day, β(λ) represents the atmospheric scattering coefficient, d(x) represents the depth map corresponding to J(x) estimated by the monocular image depth estimation algorithm MiDaS, and A represents the atmospheric light at infinity. By adjusting the values ​​of β(λ) and A, the corresponding image target detection datasets of foggy days with different concentrations are synthesized.

[0089] Specifically, in this embodiment, the atmospheric light A∈{0.7, 0.8, 0.9} and the atmospheric scattering coefficient β(λ)∈{0.5, 1.0, 1.5, 2.0, 3.0, 4.0} are taken. By adjusting the values ​​of β(λ) and A, 18 foggy image target detection data sets of different concentrations are synthesized.

[0090] Step 102, the open source fog-free road vehicle target detection dataset and the corresponding image target detection dataset of synthetic fog with different concentrations are combined into a defogging network training set; the open source fog-free road vehicle target detection dataset, the corresponding image target detection dataset of synthetic fog with different concentrations and the open source real foggy road vehicle target detection dataset are combined into an overall network training set.

[0091] Furthermore, if Figure 3 As shown, the lightweight defogging module in step 2 includes a first-level encoder, a first-level decoder, a second-level encoder, a second-level decoder, and an image cascade layer;

[0092] Among them, the first-level encoder includes a 1×1 convolution layer with a stride of 1 and an output channel number of 8, a first Relu activation function layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 8, a second Relu activation function layer, a first maximum pooling layer with a stride of 2 and a window size of 3×3, two 3×3 convolution layers with a stride of 1 and an output channel number of 16, a third Relu activation function layer, a second maximum pooling layer with a stride of 2 and a window size of 3×3, a 3×3 convolution layer with a stride of 1 and an output channel number of 32, and a fourth Relu activation function layer.

[0093] The first-stage decoder includes a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a fifth Relu activation function layer, a first cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a sixth Relu activation function layer, a first upsampling layer with a step size of 2, a second cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 8, a seventh Relu activation function layer, a second upsampling layer with a step size of 2, a 3×3 convolution layer with a step size of 1 and an output channel number of 3, and an eighth Relu activation function layer.

[0094] Among them, the output of the fourth Relu activation function layer in the first-stage encoder is the input of the 3×3 convolution layer with a step size of 1 and an output channel number of 32 in the first-stage decoder; the first cascade layer in the first-stage decoder is used to cascade the output of the second maximum pooling layer in the first-stage encoder with the output of the fifth Relu activation function layer in the first-stage decoder, and the second cascade layer in the first-stage decoder is used to cascade the output of the first maximum pooling layer in the first-stage encoder with the output of the first upsampling layer in the first-stage decoder;

[0095] The second-stage encoder includes a 1×1 convolution layer with a stride of 1 and an output channel number of 8, a ninth Relu activation function layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 8, a tenth Relu activation function layer, a third maximum pooling layer with a stride of 2 and a window size of 3×3, a third cascade layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 16, an eleventh Relu activation function layer, a fourth maximum pooling layer with a stride of 2 and a window size of 3×3, a fourth cascade layer, a 3×3 convolution layer with a stride of 1 and an output channel number of 32, and a twelfth Relu activation function layer.

[0096] The third cascade layer in the second-stage encoder is used to cascade the output of the first maximum pooling layer in the first-stage encoder with the output of the third maximum pooling layer in the second-stage encoder, and the fourth cascade layer in the second-stage encoder is used to cascade the output of the second maximum pooling layer in the first-stage encoder with the output of the fourth maximum pooling layer in the second-stage encoder;

[0097] The second stage decoder includes a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a thirteenth Relu activation function layer, a fifth cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a fourteenth Relu activation function layer, a third upsampling layer with a step size of 2, a sixth cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 8, a fifteenth Relu activation function layer, a fourth upsampling layer with a step size of 2, two 3×3 convolution layers with a step size of 1 and an output channel number of 3, and a sixteenth Relu activation function layer.

[0098] Among them, the output of the twelfth Relu activation function layer in the second-level encoder is the input of the 3×3 convolutional layer with a step size of 1 and an output channel number of 32 in the second-level decoder; the fifth cascade layer in the second-level decoder is used to cascade the output of the fifth Relu activation function layer in the first-level decoder, the output of the fourth maximum pooling layer in the second-level encoder, and the output of the thirteenth Relu activation function layer in the second-level decoder; the sixth cascade layer in the second-level decoder is used to cascade the output of the first upsampling layer in the first-level decoder, the output of the third maximum pooling layer in the second-level encoder, and the output of the third upsampling layer in the second-level decoder.

[0099] Furthermore, the lightweight adaptive target feature enhancement module in step 2 includes a hyperparameter prediction module, a guideable image processing module and a gating module.

[0100] Furthermore, if Figure 4As shown, the hyperparameter prediction module includes a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a seventeenth Relu activation function layer, a fifth maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 32, an eighteenth Relu activation function layer, a sixth maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a nineteenth Relu activation function layer, a seventh maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 64, a twentieth Relu activation function layer, a eighth maximum pooling layer with a step size of 2 and a window size of 3×3, and a step size of 1 and an output channel number of 6 4, the twenty-first Relu activation function layer, the ninth maximum pooling layer with a stride of 2 and a window size of 3×3, the 3×3 convolution layer with a stride of 1 and an output channel number of 128, the twenty-second Relu activation function layer, a global average pooling layer, a first fully connected layer with an output neuron number of 256, and two parallel second fully connected layers and third fully connected layers; the number of output neurons of the second fully connected layer is 15, and the number of output neurons of the third fully connected layer is 6; wherein, the input of the 3×3 convolution layer with a stride of 1 and an output channel number of 16 is the refined dehazed image output by the lightweight dehazing module, the output of the second fully connected layer is the 15 hyperparameters in the derivable image processing module, and the output of the third fully connected layer is the 6 weight parameters in the gating module.

[0101] Furthermore, the guideable image processing module includes six disordered guideable image processors, namely, exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening, and color adjustment; the guideable image processing module receives the refined defogging image output by the lightweight defogging module and 15 hyperparameters output by the hyperparameter prediction module, and outputs the corresponding processed image;

[0102] The exposure adjustment is represented by the following mapping function:

[0103] J exposure (x) = 2 E ×J 精 (x);

[0104] Among them, J 精 (x) represents the refined dehazed image output by the lightweight dehazing module, J exposure (x) is the refined dehazed image after exposure adjustment, and E is the exposure adjustment hyperparameter;

[0105] The white balance is represented by the following mapping function:

[0106] J wb (x)=(W r r(x),Wg g(x),W b b(x));

[0107] Among them, J wb (x) is the fine dehazed image after white balance, W r , W g , W b Respectively represent the white balance hyperparameters of the r, g, and b color channels, r(x), g(x), and b(x) represent the refined dehazed image J 精 (x) pixel value in r, g, b color channels;

[0108] The gamma transform is represented by the following mapping function:

[0109]

[0110] Among them, J gamma (x) is the refined dehazed image after gamma transformation, γ represents the gamma transformation hyperparameter;

[0111] The contrast enhancement is represented by the following mapping function:

[0112] J contrast (x) = α × En (J 精 (x))+(1-α)×J 精 (x);

[0113] Among them, J contrast (x) is the refined dehazed image after contrast enhancement, α represents the contrast enhancement hyperparameter;

[0114]

[0115] Lum(J 精 (x))=0.3r(x)+0.59g(x)+0.11b(x);

[0116] Sigmoid() is a nonlinear activation function, Lum() represents the pixel brightness function, and En() represents the image brightness enhancement function;

[0117] The sharpening is represented by the following mapping function:

[0118] J sharpen (x) = J 精 (x)+λ1(J 精 (x)-Gaussian(J 精 (x)));

[0119] Among them, J sharpen (x) is the sharpened dehazed image, λ1 is the sharpening hyperparameter, and Gaussian() is the Gaussian filter function;

[0120] The color adjustment is represented by the following mapping function:

[0121]

[0122] Among them, J tone (x) is the refined dehazed image after color adjustment, t i Adjust the hyperparameters for the i+1th color, i=0,1,2,...,7, clip(y,0,1) represents the clipping function;

[0123] Furthermore, the gating module receives the six weight parameters output by the hyperparameter prediction module, and is used to perform adaptive weighted fusion on the images processed by the six disordered derivable image processors to obtain an adaptive enhanced image J. 强 (x):

[0124] J 强 (x) = w1J exposure (x)+w2J wb (x)+w3J gamma (x)+w4J contrast (x)+w5J sharpen (x)+w6J tone (x);

[0125] Among them, w1 is the exposure adjustment weight parameter, w2 is the white balance weight parameter, w3 is the gamma transformation weight parameter, w4 is the contrast enhancement weight parameter, w5 is the sharpening weight parameter, and w6 is the color adjustment weight parameter.

[0126] Furthermore, the specific method of step 3 is:

[0127] Step 301: Input the image in the defogging network training set into the 1×1 convolutional layer with a step size of 1 and an output channel number of 8 of the first-stage encoder in the lightweight defogging module, and the eighth ReLU activation function layer of the first-stage decoder of the lightweight defogging module outputs a coarse map K with a channel number of 3. 粗 (x), according to the coarse map K 粗 (x) Calculate the coarse dehazed image corresponding to the image in the dehazed network training set:

[0128] J 粗 (x) = K 粗 (x)×I'(x)-K 粗 (x)+1;

[0129] Where I'(x) represents the image in the dehazing network training set, I'(x) = I(x) or J(x), and I'(x) is an RGB image with 3 channels; J 粗(x) is the rough dehazed image;

[0130] According to the image cascade layer of the lightweight defogging module, the image in the defogging network training set is cascaded with the corresponding coarse defogging image output by the first-level decoder to obtain a cascade image with 6 channels. The cascade image is input into the 1×1 convolution layer with a step size of 1 and an output channel number of 8 in the second-level encoder of the lightweight defogging module. The sixteenth Relu activation function layer of the second-level decoder of the lightweight defogging module outputs a fine map K with a channel number of 3. 精 (x); According to the precise map K 精 (x) Calculate the refined dehazed image corresponding to the image in the dehazed network training set:

[0131] J 精 (x) = K 精 (x)×I'(x)-K 精 (x)+1;

[0132] Step 302, using the open source fog-free road vehicle target detection dataset as the defogging true value label, using the L1 loss function to calculate the loss value between the coarse defogging image and the corresponding defogging true value label, recorded as loss1, using the L2 loss function to calculate the loss value between the fine defogging image and the corresponding defogging true value label, recorded as loss2, and calculating the total loss value Loss corresponding to the lightweight defogging module: Loss = 0.4loss1+0.6loss2; according to the total loss value Loss, the lightweight defogging module is back-propagated and trained using the gradient descent method until the total loss value converges, ending the training and saving the current lightweight defogging module.

[0133] Specifically, the simplified atmospheric scattering model can be expressed as:

[0134] I(x)=J(x)t(x)+A(1-t(x));

[0135] Where I(x) represents the foggy image, J(x) represents the fog-free image, A represents the atmospheric intensity, t(x) represents the transmission map, and x represents the image pixel.

[0136] According to the atmospheric scattering model, the defogged image can be obtained:

[0137]

[0138] By combining the parameters to be predicted t(x) and A into the unknown quantity K(x), the formula can be transformed into:

[0139] J(x)=K(x)I(x)-K(x)+1;

[0140]

[0141] Calculate the mapping parameter K(x) of the foggy image and use the above formula to defog the foggy image.

[0142] Furthermore, the specific method of step 4 is:

[0143] Step 401, freeze the parameters of the lightweight defogging module, input the images in the overall network training set into the real-time foggy road vehicle target detection model, the lightweight defogging module outputs the refined defogging image to the hyperparameter prediction module and the derivable image processing module, and the hyperparameter prediction module outputs 15 hyperparameters in the derivable image processing module and 6 weight parameters in the gating module;

[0144] The guideable image processing module obtains the refined defogging image after exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening and color adjustment according to the 15 hyperparameters output by the hyperparameter prediction module and the refined defogging image output by the lightweight defogging module;

[0145] The gating module performs weighted fusion on the refined dehazed image after exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening, and color adjustment output by the guideable image processing module according to the six weight parameters output by the hyperparameter prediction module to obtain the corresponding adaptive enhanced image;

[0146] The adaptive enhanced image is input into the target detection network YOLOv5, and the category and coordinates of the vehicle target corresponding to the image in the overall network training set are output. The category and coordinates of the vehicle target contained in the image in the overall network training set are used as the target detection true value label. According to the loss function of the target detection network YOLOv5 itself, the loss value between the output vehicle target category and coordinates and the target detection true value label is calculated. According to the loss value, the gradient descent method is used to perform back propagation training on the hyperparameter prediction module and the target detection network YOLOv5 until the loss value converges, the training is terminated, and the current real-time foggy road vehicle target detection model is saved.

[0147] Specifically, Figure 5 As shown, Figure 5 The six images on the left are visualization results of vehicle target detection on foggy roads under different fog concentrations. Figure 5 The six images on the right are visualization results of vehicle target detection on foggy roads using this method under foggy conditions of different concentrations. It can be seen that this method has better detection effect and stronger robustness for vehicle targets on foggy roads.

[0148] In summary, the present invention can use a lightweight defogging module to quickly defog foggy road vehicle images, restore the overall detail information of foggy images, and use the target detection task to guide the lightweight adaptive target feature enhancement module to adaptively enhance the effective features of foggy road vehicle targets, thereby greatly improving the detection performance of foggy road vehicle targets. Compared with the existing technical solutions, the present invention has better detection effect for foggy road vehicle targets, stronger robustness, and meets real-time requirements. It can be applied in smart transportation and provide a new solution for foggy road vehicle target detection.

[0149] Those skilled in the art will appreciate that the embodiments described are intended to help readers understand the principles of the present invention, and should be understood that the scope of protection of the present invention is not limited to the embodiments described. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A real-time foggy road vehicle target detection method with adaptive target feature enhancement, characterized in that: The steps include: Step 1: synthesize the corresponding image target detection datasets of foggy days with different concentrations based on the open source fog-free road vehicle target detection dataset, and form the defogging network training set and the overall network training set based on the open source fog-free road vehicle target detection dataset, the synthesized corresponding image target detection datasets of foggy days with different concentrations, and the open source real foggy road vehicle target detection dataset; Step 2, constructing a real-time foggy road vehicle target detection model, wherein the real-time foggy road vehicle target detection model includes a lightweight defogging module, a lightweight adaptive target feature enhancement module, and a target detection network YOLOv5; Step 3, use the defogging network training set to train the lightweight defogging module, the input of the lightweight defogging module is the image in the defogging network training set, and the output is the corresponding refined defogging image. After the training is completed, the current lightweight defogging module is saved; Step 4, freeze the parameters of the lightweight defogging module, use the overall network training set to perform end-to-end training on the real-time foggy road vehicle target detection model, the input of the real-time foggy road vehicle target detection model is the image in the overall network training set, and the output is the category and coordinates of the corresponding vehicle target. After the training is completed, save the current real-time foggy road vehicle target detection model; Step 5, input the foggy road vehicle image into the trained real-time foggy road vehicle target detection model, obtain the category and coordinates of the vehicle target in the foggy road vehicle image, and complete the real-time foggy road vehicle target detection with adaptive target feature enhancement.

2. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 1 is characterized in that: The specific method of step 1 is: Step 101, based on the open-source fog-free road vehicle target detection dataset, synthesize the corresponding image target detection datasets of foggy days with different concentrations: I(x)=J(x)e -β(λ)d(x) +A(1-e -β(λ)d(x) ); Where x represents the pixel position of the image, J(x) represents the image in the open-source fog-free road vehicle target detection dataset, I(x) represents the corresponding image of the synthetic foggy day, β(λ) represents the atmospheric scattering coefficient, d(x) represents the depth map corresponding to J(x) estimated by the monocular image depth estimation algorithm MiDaS, and A represents the atmospheric light at infinity. By adjusting the values ​​of β(λ) and A, the corresponding image target detection datasets of foggy days with different concentrations are synthesized. Step 102, the open source fog-free road vehicle target detection dataset and the corresponding image target detection dataset of synthetic fog with different concentrations are combined into a defogging network training set; the open source fog-free road vehicle target detection dataset, the corresponding image target detection dataset of synthetic fog with different concentrations and the open source real foggy road vehicle target detection dataset are combined into an overall network training set.

3. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 1 is characterized in that: The lightweight defogging module in step 2 includes a first-stage encoder, a first-stage decoder, a second-stage encoder, a second-stage decoder, and an image cascade layer; Among them, the first-level encoder includes a 1×1 convolution layer with a stride of 1 and an output channel number of 8, a first Relu activation function layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 8, a second Relu activation function layer, a first maximum pooling layer with a stride of 2 and a window size of 3×3, two 3×3 convolution layers with a stride of 1 and an output channel number of 16, a third Relu activation function layer, a second maximum pooling layer with a stride of 2 and a window size of 3×3, a 3×3 convolution layer with a stride of 1 and an output channel number of 32, and a fourth Relu activation function layer. The first-stage decoder includes a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a fifth Relu activation function layer, a first cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a sixth Relu activation function layer, a first upsampling layer with a step size of 2, a second cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 8, a seventh Relu activation function layer, a second upsampling layer with a step size of 2, a 3×3 convolution layer with a step size of 1 and an output channel number of 3, and an eighth Relu activation function layer. Among them, the output of the fourth Relu activation function layer in the first-stage encoder is the input of the 3×3 convolution layer with a step size of 1 and an output channel number of 32 in the first-stage decoder; the first cascade layer in the first-stage decoder is used to cascade the output of the second maximum pooling layer in the first-stage encoder with the output of the fifth Relu activation function layer in the first-stage decoder, and the second cascade layer in the first-stage decoder is used to cascade the output of the first maximum pooling layer in the first-stage encoder with the output of the first upsampling layer in the first-stage decoder; The second-stage encoder includes a 1×1 convolution layer with a stride of 1 and an output channel number of 8, a ninth Relu activation function layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 8, a tenth Relu activation function layer, a third maximum pooling layer with a stride of 2 and a window size of 3×3, a third cascade layer, two 3×3 convolution layers with a stride of 1 and an output channel number of 16, an eleventh Relu activation function layer, a fourth maximum pooling layer with a stride of 2 and a window size of 3×3, a fourth cascade layer, a 3×3 convolution layer with a stride of 1 and an output channel number of 32, and a twelfth Relu activation function layer. The third cascade layer in the second-stage encoder is used to cascade the output of the first maximum pooling layer in the first-stage encoder with the output of the third maximum pooling layer in the second-stage encoder, and the fourth cascade layer in the second-stage encoder is used to cascade the output of the second maximum pooling layer in the first-stage encoder with the output of the fourth maximum pooling layer in the second-stage encoder; The second stage decoder includes a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a thirteenth Relu activation function layer, a fifth cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a fourteenth Relu activation function layer, a third upsampling layer with a step size of 2, a sixth cascade layer, a 3×3 convolution layer with a step size of 1 and an output channel number of 8, a fifteenth Relu activation function layer, a fourth upsampling layer with a step size of 2, two 3×3 convolution layers with a step size of 1 and an output channel number of 3, and a sixteenth Relu activation function layer. Among them, the output of the twelfth Relu activation function layer in the second-level encoder is the input of the 3×3 convolutional layer with a step size of 1 and an output channel number of 32 in the second-level decoder; the fifth cascade layer in the second-level decoder is used to cascade the output of the fifth Relu activation function layer in the first-level decoder, the output of the fourth maximum pooling layer in the second-level encoder, and the output of the thirteenth Relu activation function layer in the second-level decoder; the sixth cascade layer in the second-level decoder is used to cascade the output of the first upsampling layer in the first-level decoder, the output of the third maximum pooling layer in the second-level encoder, and the output of the third upsampling layer in the second-level decoder.

4. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 3 is characterized in that: The lightweight adaptive target feature enhancement module in step 2 includes a hyperparameter prediction module, a guideable image processing module and a gating module.

5. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 4 is characterized in that: The hyperparameter prediction module includes a 3×3 convolution layer with a step size of 1 and an output channel number of 16, a seventeenth Relu activation function layer, a fifth maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 32, an eighteenth Relu activation function layer, a sixth maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 32, a nineteenth Relu activation function layer, a seventh maximum pooling layer with a step size of 2 and a window size of 3×3, a 3×3 convolution layer with a step size of 1 and an output channel number of 64, a twentieth Relu activation function layer, an eighth maximum pooling layer with a step size of 2 and a window size of 3×3, a step size of 1 and an output channel number of 64 A 3×3 convolutional layer, a twenty-first Relu activation function layer, a ninth maximum pooling layer with a stride of 2 and a window size of 3×3, a 3×3 convolutional layer with a stride of 1 and an output channel number of 128, a twenty-second Relu activation function layer, a global average pooling layer, a first fully connected layer with 256 output neurons, and two parallel second fully connected layers and a third fully connected layer; the number of output neurons of the second fully connected layer is 15, and the number of output neurons of the third fully connected layer is 6; wherein, the input of the 3×3 convolutional layer with a stride of 1 and an output channel number of 16 is the refined dehazed image output by the lightweight dehazing module, the output of the second fully connected layer is the 15 hyperparameters in the differentiable image processing module, and the output of the third fully connected layer is the 6 weight parameters in the gating module.

6. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 5 is characterized in that: The guideable image processing module includes six disordered guideable image processors: exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening, and color adjustment; the guideable image processing module receives the refined defogging image output by the lightweight defogging module and the 15 hyperparameters output by the hyperparameter prediction module, and outputs the corresponding processed image; The exposure adjustment is represented by the following mapping function: J exposure (x)=2 E ×J 精 (x); Among them, J 精 (x) represents the refined dehazed image output by the lightweight dehazing module, J exposure (x) is the refined dehazed image after exposure adjustment, and E is the exposure adjustment hyperparameter; The white balance is represented by the following mapping function: J wb (x)=(W r r(x),W g g(x),W b b(x)); Among them, J wb (x) = is the fine dehazed image after white balance, W r , W g , W b Respectively represent the white balance hyperparameters of the r, g, and b color channels, r(x), g(x), and b(x) represent the refined dehazed image J 精 (x) pixel value in r, g, b color channels; The gamma transform is represented by the following mapping function: Among them, J gamma (x) is the refined dehazed image after gamma transformation, γ represents the gamma transformation hyperparameter; The contrast enhancement is represented by the following mapping function: J contrast (x)=α×E(J 精 (x))+(1−α)×J 精 (x)? Among them, J contrast (x) is the refined dehazed image after contrast enhancement, α represents the contrast enhancement hyperparameter; Lum(J 精 (x))=0.3r(x)+0.59g(x)+0.11b(x); Sigmoid() is a nonlinear activation function, Lum() represents the pixel brightness function, and En() represents the image brightness enhancement function; The sharpening is represented by the following mapping function: J sharpen (x)=J 精 (x)+λ1(J 精 (x)-Gaussian(J 精 (x))); Among them, J sharpen (x) is the sharpened dehazed image, λ1 is the sharpening hyperparameter, and Gaussian() is the Gaussian filter function; The color adjustment is represented by the following mapping function: Among them, J tone (x) is the refined dehazed image after color adjustment, t i Adjust the hyperparameters for the i+1th color, i=0,1,2,...,7, clip(y,0,1) represents the clipping function; 7. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 6 is characterized in that: The gating module receives the six weight parameters output by the hyperparameter prediction module and is used to perform adaptive weighted fusion on the images processed by the six disordered derivable image processors to obtain an adaptive enhanced image J. 强 (x): J 强 (x)=w1J exposure (x)+w2J wb (x)+w3J gamma (x)+w4J contrast (x)+w5J sharpen (x)+w6J tone (x)? Among them, w1 is the exposure adjustment weight parameter, w2 is the white balance weight parameter, w3 is the gamma transformation weight parameter, w4 is the contrast enhancement weight parameter, w5 is the sharpening weight parameter, and w6 is the color adjustment weight parameter.

8. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 7 is characterized in that: The specific method of step 3 is: Step 301: Input the image in the defogging network training set into the 1×1 convolutional layer with a step size of 1 and an output channel number of 8 of the first-stage encoder in the lightweight defogging module, and the eighth ReLU activation function layer of the first-stage decoder of the lightweight defogging module outputs a coarse map K with a channel number of 3. 粗 (x), according to the coarse map K 粗 (x) Calculate the coarse dehazed image corresponding to the image in the dehazed network training set: J 粗 (x)=K 粗 (x)×I'(x)-K 粗 (x)+1; Where I'(x) represents the image in the dehazing network training set, I'(x) = I(x) or J(x), and I'(x) is an RGB image with 3 channels; J 粗 (x) is the rough dehazed image; According to the image cascade layer of the lightweight defogging module, the image in the defogging network training set is cascaded with the corresponding coarse defogging image output by the first-level decoder to obtain a cascade image with 6 channels. The cascade image is input into the 1×1 convolution layer with a step size of 1 and an output channel number of 8 in the second-level encoder of the lightweight defogging module. The sixteenth Relu activation function layer of the second-level decoder of the lightweight defogging module outputs a fine map K with a channel number of 3. 精 (x); According to the precise map K 精 (x) Calculate the refined dehazed image corresponding to the image in the dehazed network training set: J 精 (x)=K 精 (x)×I'(x)-K 精 (x)+1; Step 302: Use the open-source fog-free road vehicle target detection dataset as the defogging true value label, use the L1 loss function to calculate the loss value between the coarse defogging image and the corresponding defogging true value label, recorded as loss1, use the L2 loss function to calculate the loss value between the fine defogging image and the corresponding defogging true value label, recorded as loss2, and calculate the total loss value Loss corresponding to the lightweight defogging module: Loss = 0.4loss1 + 0.6loss2; According to the total loss value Loss, the gradient descent method is used to perform back propagation training on the lightweight dehazing module until the total loss value converges, the training is ended, and the current lightweight dehazing module is saved.

9. The real-time foggy road vehicle target detection method with adaptive target feature enhancement according to claim 8 is characterized in that: The specific method of step 4 is: Step 401, freeze the parameters of the lightweight defogging module, input the images in the overall network training set into the real-time foggy road vehicle target detection model, the lightweight defogging module outputs the refined defogging image to the hyperparameter prediction module and the derivable image processing module, and the hyperparameter prediction module outputs 15 hyperparameters in the derivable image processing module and 6 weight parameters in the gating module; The guideable image processing module obtains the refined defogging image after exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening and color adjustment according to the 15 hyperparameters output by the hyperparameter prediction module and the refined defogging image output by the lightweight defogging module; The gating module performs weighted fusion on the refined dehazed image after exposure adjustment, white balance, gamma transformation, contrast enhancement, sharpening, and color adjustment output by the guideable image processing module according to the six weight parameters output by the hyperparameter prediction module to obtain the corresponding adaptive enhanced image; The adaptive enhanced image is input into the target detection network YOLOv5, and the category and coordinates of the vehicle target corresponding to the image in the overall network training set are output. The category and coordinates of the vehicle target contained in the image in the overall network training set are used as the target detection true value label. According to the loss function of the target detection network YOLOv5 itself, the loss value between the output vehicle target category and coordinates and the target detection true value label is calculated. According to the loss value, the gradient descent method is used to perform back propagation training on the hyperparameter prediction module and the target detection network YOLOv5 until the loss value converges, the training is terminated, and the current real-time foggy road vehicle target detection model is saved.

Images