Foggy day image vehicle detection method based on cyclic generative adversarial network and interest domain enhancement

Abstract

The invention discloses a foggy day image vehicle detection method based on a cyclic generative adversarial network and interest domain enhancement, and the method comprises the steps: obtaining a road vehicle image under a foggy day condition, and converting the foggy day image into a fogless image through the cyclic generative adversarial network; constructing a feature extraction network to extract a feature map of the fog-free image, generating a candidate frame through a region generation network, and processing the candidate frame through a clustering algorithm to obtain a clustered rectangular frame and generate a mask map of the region of interest; constructing a mask feature extraction network, and extracting features of the mask graph of the region of interest; a feature fusion module is constructed to splice and fuse the feature maps; and constructing a detection architecture comprising uncertainty minimization query selection, a lightweight decoder and a prediction network to realize target detection and classification. According to the method, the image quality and visibility can be improved, the region-of-interest mask graph is generated through target detection and a clustering algorithm, and the specific region of the image is effectively concerned, so that the detection precision is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle targets in foggy weather, and in particular to a vehicle detection method for foggy weather images based on a cyclic generative adversarial network and domain of interest enhancement. Background Art

[0002] With the rapid development of intelligent transportation systems and autonomous driving technologies, vehicle detection, as one of the core technologies, plays an important role in traffic monitoring, driving assistance, and intelligent navigation. However, in practical applications, complex weather conditions pose severe challenges to the performance of vehicle detection systems. Among them, foggy environments have become one of the main difficulties in vehicle detection in autonomous driving and intelligent monitoring due to low visibility, light attenuation, and contrast reduction. On the one hand, the quality of images captured by ordinary cameras is greatly reduced, especially the vehicle contours and detail information are blocked or blurred by fog, making it difficult for traditional detection algorithms based on edge, texture, or color features to work stably. On the other hand, fog causes optical attenuation and color distortion of images, which makes it difficult for deep learning-based vehicle detection models to extract effective features from low-quality images, thereby affecting the detection accuracy and robustness of the model.

[0003] In foggy environments, some studies have attempted to improve vehicle detection performance through image preprocessing techniques. For example, defogging algorithms can improve image quality by enhancing image contrast and restoring true colors. However, defogging algorithms themselves have problems such as high computational complexity, poor real-time performance, and the introduction of artifacts, making them difficult to be directly applied in efficient vehicle detection scenarios. In addition, relying solely on image enhancement technology is still difficult to completely solve the problem of missing vehicle appearance information caused by foggy weather.

[0004] In recent years, deep learning-based vehicle detection methods have developed rapidly and have gradually become the mainstream means to solve vehicle detection problems in complex scenarios. Through models such as convolutional neural networks and Transformer, researchers can learn multi-level semantic features from large-scale data and significantly improve the robustness and adaptability of vehicle detection. However, current deep learning methods still face challenges in the task of vehicle detection in foggy weather. Specifically, due to the difficulty in collecting foggy data and the high cost of annotation, existing training data sets often lack sufficient coverage of foggy scenes, resulting in insufficient out-of-domain generalization capabilities of the model in practical applications. In addition, specific optical properties under foggy conditions, such as scattering and attenuation of reflected light, may cause the model to be ineffective in detecting long-distance vehicles.

[0005] In order to solve the above problems, methods for fusing multi-sensor data have also been proposed in the prior art. For example, the multi-modal fusion method combining lidar, millimeter-wave radar and camera can effectively alleviate the dependence of foggy environment on single visual information. However, the hardware cost of the multi-sensor fusion method is relatively high, and it still faces technical bottlenecks in data synchronization and cross-modal information fusion. Summary of the invention

[0006] The purpose of the present invention is to disclose a vehicle detection method for foggy images based on a cyclic generative adversarial network and domain of interest enhancement, which can simplify the necessary hardware conditions required for detection, while enhancing the semantic features of the image and improving the robustness and accuracy of detection.

[0007] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0008] A vehicle detection method for foggy images based on a cyclic generative adversarial network and interest domain enhancement, the method comprising:

[0009] S1, obtain a road vehicle image under foggy conditions, and obtain a fog-free road vehicle image through a cyclic generative adversarial network;

[0010] S2, constructing a feature extraction network to extract deep features of fog-free road vehicle images, inputting multiple fog-free image feature maps of different sizes into a region generation network to generate candidate boxes of different sizes and positions, processing the candidate boxes through a clustering algorithm, obtaining clustered rectangular boxes and generating a mask map of the region of interest;

[0011] S3, constructing a mask image feature extraction network, downsampling the mask image of the region of interest, and obtaining a mask image feature map corresponding to the size of the haze-free image feature map;

[0012] S4, constructing a feature fusion module to perform feature splicing and fusion of the extracted haze-free image feature map and mask image feature map within the same scale and between different scales;

[0013] S5, construct an uncertainty minimization query selection method, screen the query results generated by the target detection model, and select the query results with the smallest uncertainty; use a lightweight decoder to perform multi-level decoding processing on the fused features of step S4, and input the decoding results into the target detection model built based on the prediction network to generate the final detection output, including target category and bounding box information.

[0014] Step S1 further comprises:

[0015] The road camera collects images of road vehicles under foggy conditions and converts them into fog-free road vehicle images through a cyclic generative adversarial network.

[0016] Label the vehicle images on fog-free roads, generate a data set, and randomly divide the data set into a training set and a test set according to a preset ratio.

[0017] Step S2 further comprises:

[0018] A feature extraction network was built based on the CSPDarknet53 network to extract features from fog-free road vehicle images and output image feature maps of various sizes.

[0019] The extracted feature map is input into the region generation network to generate candidate boxes of different sizes and positions, and then the candidate boxes are clustered through the clustering algorithm using the following formula:

[0020]

[0021] box new =box+m(box)

[0022] Where box = (x center ,y center ,w,h) represents the center coordinates, width and height of the candidate box; box j represents the jth candidate box; K(box j -box) is the kernel function used to calculate box and box j r is the bandwidth parameter, which controls the perception range and neighborhood size of the kernel function; m(box) is the mean shift vector, which indicates the offset of the box to the density center; N(box) represents the set of candidate boxes in the neighborhood of the current box; box new is the updated candidate box position; the coordinates of the upper left corner and lower right corner of the rectangular box are:

[0023]

[0024] Among them, left_top is the coordinate of the upper left corner of the rectangular box; right_bottom is the coordinate of the lower right corner of the rectangular box; min(·) is the minimum value operation, and max(·) is the maximum value operation; box j (1) represents the x-axis coordinate x of the center of the candidate box center ;box j (2) represents the y-axis coordinate y of the center of the candidate box center ;

[0025] After clustering, the rectangular box is used as the region of interest to design the corresponding mask map.

[0026] As a preferred example, after clustering, the process of designing a corresponding mask map using a rectangular box as a region of interest includes the following steps:

[0027] Construct an empty image with background pixels δ and size W×H, where W is the width resolution of the input image and H is the height resolution of the input image;

[0028] The candidate boxes obtained by clustering are processed in the corresponding area of ​​the empty image, and the pixel value in the corresponding area is set to α;

[0029] The processed image is fused with the fog-free road vehicle image to obtain the mask map of the region of interest.

[0030] As a preferred example, the feature extraction network includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module and an AIFI module connected in sequence;

[0031] The first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer all include convolution operations, batch normalization, and SiLU function activation;

[0032] The first C2f module, the second C2f module, the third C2f module and the fourth C2f module are all composed of 3 groups of residual modules; the input is divided into two branches after passing through the 1×1 convolution layer to twice the original number of channels, and the output result of the first branch is directly passed to the splicing module; the output result of the second branch enters the first group of residual modules for processing, and the result of the first group of residual modules is split into two parts, one part is passed to the splicing module, and the other part enters the second group of residual modules for processing; the output result of the second group of residual modules is split into two parts, one part is passed to the splicing module again, and the other part enters the third group of residual modules, and the output result of the third group of residual modules is directly passed to the splicing module, spliced ​​with the output result of the first branch passed to the splicing module, the partial output result of the first group of residual modules, and the partial output result of the second group of residual modules, and finally enters the 1×1 convolution layer;

[0033] The AIFI module generates a two-dimensional sine-cosine position code that matches the input size using the output features of the fourth C2f module through a position encoder, converts the image data shape from a 4D tensor to a 3D tensor, and then passes the transformed image data and the generated position code into the transformer encoder for feature interaction and processing; finally, the result is readjusted to a 4D tensor shape, and the processed haze-free image feature map is output.

[0034] As a preferred example, in step S3, the mask map feature extraction network uses five pooling layers to downsample the mask map, so that the size of the output mask map is consistent with the feature map output by the haze-free image feature extraction network.

[0035] Step S4 further comprises:

[0036] For image feature maps And mask feature map The inter-scale feature fusion module is used for splicing and fusion, and the fusion operation is as follows:

[0037]

[0038] Among them, ISFM(·) represents inter-scale feature fusion; Concat(·) represents the concatenation operator; M3, M1 and M2 represent the fusion features of the feature fusion modules in three scales, respectively, and are expressed as:

[0039]

[0040] in, is the fusion feature of IFM; CBS1(·) indicates the convolution operation with a convolution kernel of 1, batch standardization operation and activation function operation; CBS3(·) indicates the convolution operation with a convolution kernel of 3, batch standardization operation and activation function operation; UP(·) indicates the upsampling operation; IFM(·) indicates the intra-scale feature fusion, and its operation is as follows:

[0041]

[0042] Among them, Flatten(·) represents the flattening operation, Conv(·) represents the convolution operation, and RepB(·) represents the module operation consisting of three sets of heavy parameter convolutions.

[0043] As a preferred example, in step S5, a query selection method for minimizing uncertainty is constructed, and the process of selecting the query result with the minimum uncertainty by screening the query results generated by the target detection model includes the following steps:

[0044] For multiple queries generated during the target detection process, the uncertainty value of each query is calculated based on its prediction score distribution or confidence; the uncertainty value is determined by the entropy or other uncertainty measurement function of the category prediction score corresponding to the query;

[0045] According to the calculated uncertainty value, one or more queries with the smallest uncertainty are selected from all queries as valid queries for subsequent detection tasks;

[0046] The final detection results are generated using the filtered uncertainty queries.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The vehicle detection method for foggy images based on a cyclic generative adversarial network and interest domain enhancement of the present invention can generate fog-free images according to the cyclic generative adversarial network, thereby improving image quality and visibility, and generate a region of interest mask map through target detection and clustering algorithms, which can effectively focus on specific areas of the image, thereby making the detection accuracy higher; at the same time, it can simplify the necessary hardware conditions required for detection, while enhancing the semantic features of the image, and improving the robustness and accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the overall process of a vehicle detection method for foggy images based on a cyclic generative adversarial network and interest domain enhancement according to an embodiment of the present invention;

[0050] Figure 2 is a before-and-after comparison diagram of a recurrent generative adversarial network according to an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of a candidate frame image generated by a region generation network RPN according to an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of a rectangular frame image clustered by a clustering algorithm according to an embodiment of the present invention;

[0053] Figure 5 is a schematic diagram of a mask image of a region of interest according to an embodiment of the present invention;

[0054] Figure 6 is a schematic diagram of a feature fusion module according to an embodiment of the present invention;

[0055] Figure 7 is a schematic diagram of the overall network structure of an embodiment of the present invention;

[0056] Figure 8 Schematic diagram of the detection results of the embodiment of the present invention. DETAILED DESCRIPTION

[0057] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0058] like Figure 1-8 As shown, a vehicle detection method for foggy images based on a cyclic generative adversarial network and interest domain enhancement according to an embodiment of the present invention includes:

[0059] S1: Collect foggy image data on the highway through road cameras. And convert the foggy images into fog-free road images through the cycle generative adversarial network CycleGAN, such as Figure 2 As shown in the figure, a dataset containing 25,048 highway images and their corresponding labels is created, and the dataset is randomly divided into training and test sets in a ratio of 8:2.

[0060] S2: Construct a feature extraction network to extract the deep features of the fog-free image, then generate candidate boxes through the region generation network, process the candidate boxes through the clustering algorithm, obtain the clustered rectangular box and generate a mask map of the area of ​​interest.

[0061] The feature extraction network uses the CSPDarknet53 network. The haze-free image is downsampled by 32 times when it passes through the network for feature extraction, and the final three-layer feature map is output for subsequent operations. When the output image resolution is W×H×3, three image features of different sizes are output, denoted as and The output feature map scales are W is the width resolution of the image, and H is the height resolution of the image. For example, when the input image resolution is 640×640, the output feature map scale is 80×80×256, 40×40×512, and 20×20×1024.

[0062] For the CSPDarknet53 network, it includes convolutional layers, residual modules, C2f modules, and the SPPF module is replaced by the AIFI module. The convolutional layer includes convolution operations, batch normalization and SiLU function activation; the residual module reduces the input by half the number of channels through a 1×1 convolutional layer, and then increases it to the original number of channels through a 3×3 convolutional layer, and finally adds it to the input; the C2f module divides the input into two branches after it passes through a 1×1 convolutional layer to twice the original number of channels. Each C2f module includes 3 groups of residual modules; the input is divided into two branches after it passes through a 1×1 convolutional layer to twice the original number of channels. The output result of the first branch is directly passed to the splicing module; the output result of the second branch enters the first group of residual modules for processing. The results of the first group of residual modules are split into two parts according to the number of channels 1:1, one part is passed to the splicing module, and the other part enters the second group of residual modules for processing; the output result of the second group of residual modules is split into two parts, one part is passed to the splicing module again The output of the third residual module is directly passed to the splicing module, and the output of the first branch of the splicing module, the partial output of the first residual module, and the partial output of the second residual module are spliced, and finally enter the 1×1 convolution layer; the AIFI module generates a two-dimensional sine-cosine position code that matches the input size through the position encoder, and then converts the image data shape from a 4D tensor (batch_size, channels, height, width) to a 3D tensor (batch_size, height*width, channels), and then passes the transformed image data and the generated position code to the encoder of the transformer for feature interaction and processing. Finally, the result is resized to a 4D tensor shape to output the processed image features.Specific network structure: the first layer is a convolution layer with a kernel size of 3×3, a step size of 2, a padding of 2, a number of channels of 64, and an output size of 320×320×64; the second layer is a convolution layer with a kernel size of 3×3, a step size of 2, a number of channels of 128, and an output size of 160×160×128; the third layer is a C2f module, which consists of 3 groups of residual modules, and the output size is 160×160×128; the fourth layer is a convolution layer with a kernel size of 3×3, a step size of 2, a number of channels of 256, and an output size of 80×80×256; the fifth layer is a C2f module, which consists of 3 groups of residual modules. The output size is 80×80×256; the sixth layer is a convolutional layer with a convolution kernel size of 3×3, a stride of 2, a number of channels of 512, and an output size of 40×40×512; the seventh layer is a C2f module, which consists of 3 groups of residual modules and has an output size of 40×40×512; the eighth layer is a convolutional layer with a convolution kernel size of 3×3, a stride of 2, a number of channels of 1024, and an output size of 20×20×1024; the ninth layer is a C2f module, which consists of 3 groups of residual modules and has an output size of 20×20×1024; the tenth layer is an AIFI module with an output size of 20×20×1024.

[0063] Then the candidate box is generated by the region generation network, such as Figure 3 As shown, the candidate boxes are then clustered using the Meanshift clustering algorithm. The specific clustering process is as follows:

[0064]

[0065] Where box = (x center ,y center ,w,h) represents the center coordinates, width and height of the candidate box; box j represents the jth candidate box; K(box j -box) is the kernel function used to calculate box and box j r is the bandwidth parameter, which controls the perception range and neighborhood size of the kernel function; m(box) is the mean shift vector, which indicates the offset of the box to the density center; N(box) represents the set of candidate boxes in the neighborhood of the current box; box new is the updated candidate box position. The coordinates of the upper left corner and lower right corner of the rectangular box are:

[0066]

[0067] Among them, left_top is the coordinate of the upper left corner of the rectangular box; right_bottom is the coordinate of the lower right corner of the rectangular box; min(·) is the minimum value operation, and max(·) is the maximum value operation; box j(1) represents the x-axis coordinate of the center of the candidate box, that is, x center ;box j (2) represents the y-axis coordinate of the center of the candidate box, i.e., y center .

[0068] After clustering, the rectangular box is used as the region of interest to design the corresponding mask map. The specific process includes:

[0069] First, construct an empty image with background pixels δ and size W×H;

[0070] Then, the candidate boxes obtained by clustering are processed in the corresponding area of ​​the empty image, and the pixel value in the corresponding area is set to α;

[0071] Finally, the processed image is fused with the haze-free image to obtain the mask map of the region of interest.

[0072] S3: Construct a mask map feature extraction network. The mask map of the region of interest is downsampled through the feature extraction network to obtain a feature map of the corresponding size to the feature map of the haze-free image. Since the region of interest only contains the region coordinate information, the mask map feature extraction network mainly uses five pooling layers to downsample the mask map so that the output mask map size is consistent with the feature map output by the haze-free image feature extraction network, so as to facilitate subsequent fusion operations. That is, the output mask map sizes are respectively That is, the sizes of the extracted feature maps are 80×80×3, 40×40×3, and 20×20×3, respectively.

[0073] There are five maximum pooling layers in the pooling layer network for the ROI mask image. If the input is a 640×640×3 ROI mask image, the output sizes are 320×320×3, 160×160×3, 80×80×3, 40×40×3, and 20×20×3 respectively.

[0074] S4: Construct a feature fusion module to perform feature splicing and fusion of the extracted feature maps within the same scale and between different scales. The specific operations are as follows:

[0075] For image feature maps And mask feature map The inter-scale fusion module (ISFM) is used for splicing and fusion. The fusion operation is as follows:

[0076]

[0077] Among them, ISFM(·) represents inter-scale feature fusion; Concat(·) represents the concatenation operator; M3, M1 and M2 represent the fusion features of three intra-scale fusion modules (IFM), which can be expressed as:

[0078]

[0079] in, is the fusion feature of IFM; CBS1(·) indicates the convolution operation with a convolution kernel of 1, batch standardization operation and activation function operation; CBS3(·) indicates the convolution operation with a convolution kernel of 3, batch standardization operation and activation function operation; UP(·) indicates the upsampling operation; IFM(·) indicates the intra-scale feature fusion, and its operation is as follows:

[0080]

[0081] Among them, Flatten(·) represents a flattening operation, Conv(·) represents a convolution operation, and RepB(·) represents a module operation consisting of three sets of heavy parameter convolutions (RepConv).

[0082] The specific fusion process is:

[0083] The input image feature map Image feature map after upsampling and after convolution layer with convolution kernel size of 1×1 And the mask feature map The intra-scale feature fusion module is used for fusion. After the input features are spliced, the intra-scale fusion module is divided into two branches. One branch passes through a convolution layer with a convolution kernel size of 1×1, and then passes through three layers of RepBlock, and is added and flattened with the other branch that only passes through a convolution layer with a convolution kernel size of 1×1, which is recorded as a feature map. After a convolution layer with a convolution kernel size of 1×1, and then up-sampled, the image feature map is the same as the image feature map after the convolution layer with a convolution kernel size of 1×1. And the mask feature map The IFM module is used for fusion, which is recorded as feature map M3; after M3 passes through a convolution layer with a convolution kernel size of 3×3, it is combined with the feature map after passing through a convolution layer with a convolution kernel size of 1×1. Use the IFM module for fusion and record it as a feature map After a convolution layer with a convolution kernel size of 3×3, the image feature map And the mask feature map Use the IFM module to fuse and record it as feature map M1. Finally, the feature map M1, M3 performs splicing and outputs the fused data.

[0084] S5: Construct a detection architecture including uncertainty minimization query selection, lightweight decoder and prediction network to achieve target detection and classification. Specifically, construct an uncertainty minimization query selection method, filter the queries (Query) generated by the target detection model, and select the query with the smallest uncertainty to enhance the stability and accuracy of the detection results, including the following steps:

[0085] (1) Query uncertainty measurement: For multiple queries generated during the object detection process, the uncertainty value of each query is calculated based on its prediction score distribution or confidence. The uncertainty value is determined by the entropy or other uncertainty measurement function of the category prediction score corresponding to the query.

[0086] (2) Selection of queries with the minimum uncertainty: Based on the calculated uncertainty value, one or more queries with the minimum uncertainty are selected from all queries as valid queries for subsequent detection tasks.

[0087] (3) Optimization of detection results: Use the filtered low-uncertainty queries to generate the final detection results, thereby reducing the false detection and missed detection problems caused by high-uncertainty queries and improving the performance of the detection model.

[0088] The detection architecture of the lightweight decoder and prediction network is constructed to perform multi-level decoding processing on the input features and generate the final detection output, including the target category and bounding box information. The algorithm is compared with other algorithms, and the final detection results are shown in Table 1:

[0089] Table 1 Detection results of the present invention and other target detection networks on the self-made vehicle dataset

[0090]

[0091] Table 1 shows the detection results of the present invention and other target detection networks on the self-made vehicle data set. It can be seen that the present invention has the highest detection accuracy in AP for each type of vehicle, and the overall mAP index reaches 79.35%, which is better than other target detection algorithms. And for special trucks that are difficult to monitor, the improvement is very high, and the improvement accuracy exceeds 10%, which is enough to prove the superiority of the present invention. Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 8 The text part has no effect on the technical solution of the present invention, and is only used to show the image processing process and the final effect.

[0092] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0093] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0097] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A vehicle detection method for foggy images based on a recurrent generative adversarial network and interest domain enhancement, characterized in that: The method comprises: S1, obtain a road vehicle image under foggy conditions, and obtain a fog-free road vehicle image through a cyclic generative adversarial network; S2, constructing a feature extraction network to extract deep features of fog-free road vehicle images, inputting multiple fog-free image feature maps of different sizes into a region generation network to generate candidate boxes of different sizes and positions, processing the candidate boxes through a clustering algorithm, obtaining clustered rectangular boxes and generating a mask map of the region of interest; S3, constructing a mask image feature extraction network, downsampling the mask image of the region of interest, and obtaining a mask image feature map corresponding to the size of the haze-free image feature map; S4, constructing a feature fusion module to perform feature splicing and fusion of the extracted haze-free image feature map and mask image feature map within the same scale and between different scales; S5, construct an uncertainty minimization query selection method, screen the query results generated by the target detection model, and select the query results with the smallest uncertainty; use a lightweight decoder to perform multi-level decoding processing on the fused features of step S4, and input the decoding results into the target detection model built based on the prediction network to generate the final detection output, including target category and bounding box information.

2. The vehicle detection method for foggy images based on cyclic generative adversarial network and interest domain enhancement according to claim 1 is characterized in that: Step S1 further comprises: The road camera collects images of road vehicles under foggy conditions and converts them into fog-free road vehicle images through a cyclic generative adversarial network. Label the vehicle images on fog-free roads, generate a data set, and randomly divide the data set into a training set and a test set according to a preset ratio.

3. The vehicle detection method for foggy images based on cyclic generative adversarial network and interest domain enhancement according to claim 1 is characterized in that: Step S2 further comprises: A feature extraction network was built based on the CSPDarknet53 network to extract features from fog-free road vehicle images and output image feature maps of various sizes. The extracted feature map is input into the region generation network to generate candidate boxes of different sizes and positions, and then the candidate boxes are clustered through the clustering algorithm using the following formula: box new =box+m(box) Where box = (x center ,y center ,w,h) represents the center coordinates, width and height of the candidate box; box j represents the jth candidate box; K(box j -box) is the kernel function used to calculate box and box j r is the bandwidth parameter, which controls the perception range and neighborhood size of the kernel function; m(box) is the mean shift vector, which indicates the offset of the box to the density center; N(box) represents the set of candidate boxes in the neighborhood of the current box; box new is the updated candidate box position; the coordinates of the upper left corner and lower right corner of the rectangular box are: Among them, left_top is the coordinate of the upper left corner of the rectangular box; right_bottom is the coordinate of the lower right corner of the rectangular box; min(·) is the minimum value operation, and max(·) is the maximum value operation; box j (1) represents the x-axis coordinate x of the center of the candidate box center ;box j (2) represents the y-axis coordinate y of the center of the candidate box center ; After clustering, the rectangular box is used as the region of interest to design the corresponding mask map.

4. The vehicle detection method for foggy images based on cyclic generative adversarial network and interest domain enhancement according to claim 3 is characterized in that: After clustering, the process of designing the corresponding mask map using the rectangular box as the region of interest includes the following steps: Construct an empty image with background pixels δ and size W×H, where W is the width resolution of the input image and H is the height resolution of the input image; The candidate boxes obtained by clustering are processed in the corresponding area of ​​the empty image, and the pixel value in the corresponding area is set to α; The processed image is fused with the fog-free road vehicle image to obtain the mask map of the area of ​​interest.

5. The vehicle detection method for foggy images based on cyclic generative adversarial network and interest domain enhancement according to claim 1 is characterized in that: The feature extraction network includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a second C2f module, a fourth convolutional layer, a third C2f module, a fifth convolutional layer, a fourth C2f module and an AIFI module connected in sequence; The first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer all include convolution operations, batch normalization, and SiLU function activation; The first C2f module, the second C2f module, the third C2f module and the fourth C2f module are each composed of three groups of residual modules; the input is divided into two branches after passing through a 1×1 convolution layer to double the original number of channels, and the output result of the first branch is directly passed to the splicing module; the output result of the second branch enters the first group of residual modules for processing, and the result of the first group of residual modules is split into two parts, one part is passed to the splicing module, and the other part enters the second group of residual modules for processing; The output results of the second group of residual modules are split into two parts. One part is passed to the splicing module again, and the other part enters the third group of residual modules. The output results of the third group of residual modules are directly passed to the splicing module, and spliced ​​with the output results of the first branch passed to the splicing module, the partial output results of the first group of residual modules, and the partial output results of the second group of residual modules, and finally enter the 1×1 convolution layer; The AIFI module generates a two-dimensional sine-cosine position code that matches the input size using the output features of the fourth C2f module through a position encoder, converts the image data shape from a 4D tensor to a 3D tensor, and then passes the transformed image data and the generated position code into the transformer encoder for feature interaction and processing; finally, the result is readjusted to a 4D tensor shape, and the processed haze-free image feature map is output.

6. The vehicle detection method for foggy images based on cyclic generative adversarial network and interest domain enhancement according to claim 1 is characterized in that: In step S3, the mask map feature extraction network uses five pooling layers to downsample the mask map so that the size of the output mask map is consistent with the feature map output by the haze-free image feature extraction network.

7. The vehicle detection method for foggy images based on cyclic generative adversarial network and interest domain enhancement according to claim 1 is characterized in that: Step S4 further comprises: For image feature maps And mask feature map The inter-scale feature fusion module is used for splicing and fusion, and the fusion operation is as follows: Among them, ISFM(·) represents inter-scale feature fusion; Concat(·) represents the concatenation operator; M3, M1 and M2 represent the fusion features of the feature fusion modules in three scales, respectively, and are expressed as: in, is the fusion feature of IFM; CBS1(·) indicates the convolution operation with a convolution kernel of 1, batch standardization operation and activation function operation; CBS3(·) indicates the convolution operation with a convolution kernel of 3, batch standardization operation and activation function operation; UP(·) indicates the upsampling operation; IFM(·) indicates the intra-scale feature fusion, and its operation is as follows: Among them, Flatten(·) represents the flattening operation, Conv(·) represents the convolution operation, and RepB(·) represents the module operation consisting of three sets of heavy parameter convolutions.

8. The vehicle detection method for foggy images based on cyclic generative adversarial network and interest domain enhancement according to claim 1 is characterized in that: In step S5, a query selection method for minimizing uncertainty is constructed. The query results generated by the target detection model are screened, and the process of selecting the query result with the minimum uncertainty includes the following steps: For multiple queries generated during the target detection process, the uncertainty value of each query is calculated based on its prediction score distribution or confidence; the uncertainty value is determined by the entropy or other uncertainty measurement function of the category prediction score corresponding to the query; According to the calculated uncertainty value, one or more queries with the smallest uncertainty are selected from all queries as valid queries for subsequent detection tasks; The final detection results are generated using the filtered uncertainty queries.

Images