A dual-branch fusion-based severe weather traffic scene target detection method

Abstract

The present application relates to a kind of based on dual-branch fusion adverse weather traffic scene target detection method, belong to intelligent transportation, automatic driving visual perception and target detection technical field.The method includes: S1: obtaining clean traffic scene image, and it is artificially synthesized and pretreated to adverse weather, and obtain the adverse weather degraded image to be detected;S2: construct dual-branch progressive fusion detection network, adverse weather degradation feature extraction branch is used to directly extract degradation feature from degraded image;Restoration guide feature extraction branch is used to carry out unified recovery modeling to weather degradation mapping in degraded image, obtains recovery guide feature;S3: the degraded feature and recovery guide feature are input into adverse weather degradation feature extraction branch and obtain fusion detection feature;S4: using the progressive fusion training strategy of alternate freezing and loss function carries out training;S5: fusion detection feature is input into detection head, and detection result is output after confidence screening and non-maximum suppression.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent transportation, autonomous driving visual perception and target detection technology, and relates to a target detection method for adverse weather traffic scenarios based on dual-branch fusion. Background Technology

[0002] Traffic scene object detection is a fundamental component of autonomous driving environmental perception, assisted driving safety warnings, and road traffic monitoring. Vehicle vision systems need to stably identify and locate road participants such as pedestrians, bicycles, vehicles, motorcycles, and buses to provide reliable input for path planning, collision warnings, speed control, and traffic situation analysis.

[0003] Under clear, high-visibility conditions, target detection methods based on convolutional neural networks and single-stage detection frameworks can achieve high detection accuracy and real-time performance. However, in adverse weather conditions such as fog, rain, snow, low-light conditions at night, and mixed rain, fog, snow, and fog, camera images are prone to degradation phenomena such as decreased contrast, blurred target edges, attenuated texture details, rain occlusion, snow occlusion, light scattering, and local reflections. These degradations make it difficult for the detection network to obtain stable target contours, target textures, and contextual semantic responses.

[0004] In existing technologies, one type of method directly performs target detection on severe weather images and enhances detection features through attention mechanisms, multi-scale feature fusion, or Transformer structures. This type of method can avoid the propagation of recovery artifacts, but it lacks explicit modeling of weather degradation information. When fog, rain lines, or snow cover cause severe information loss, it is prone to missed detections, false detections, and bounding box offsets.

[0005] Another type of approach employs a cascaded framework of image restoration followed by object detection. This involves first improving visual quality through dehazing, deraining, desnowing, or low-light enhancement models, and then inputting the restored image into the detector. While this approach can improve image visibility, image restoration models typically optimize for peak signal-to-noise ratio, structural similarity, or perceptual quality, while object detection optimizes for category discrimination and bounding box localization. This inconsistency in objective functions leads to problems. Over-restoration can also introduce artifacts, texture smoothing, or color shifts, interfering with downstream feature extraction.

[0006] Another approach attempts to perform end-to-end joint training of image restoration and object detection. However, if the coupling between the restoration and detection branches is too strong, noise generated by the restoration branch in the early stages of training can disrupt the feature learning of the detection branch. Conversely, if the coupling is too weak, it becomes difficult to fully utilize the degradation correction information provided by the restoration branch. Therefore, achieving a balance between utilizing restoration information, preserving detection semantics, and suppressing task conflicts remains a key challenge in object detection during adverse weather and traffic scenarios.

[0007] Therefore, there is a need for a target detection method for traffic scenarios in severe weather, which can simultaneously learn weather degradation recovery features and degradation domain target semantic features, and reduce the conflict between image restoration tasks and target detection tasks through progressive training and task consistency constraints, thereby improving the detection accuracy and robustness under complex weather conditions. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a target detection method for traffic scenes in severe weather based on bi-branch fusion. By using a bi-branch progressive fusion detection network, a dynamic inverted bottleneck module, a bi-branch backbone network, an attention fusion module, an alternating freezing progressive fusion training strategy, and a robust consistency loss are combined to improve the accuracy and robustness of target detection in traffic scenes under rain, snow, fog, and combined severe weather conditions.

[0009] To achieve the above objectives, the present invention provides the following technical solution: A target detection method for severe weather traffic scenes based on dual-branch fusion, specifically including the following steps: S1: Obtain clean traffic scene images and artificially synthesize them for severe weather to construct clean image-degraded image pairs. Perform scale normalization and pixel normalization preprocessing on the images in the clean image-degraded image pairs to obtain the severe weather degraded images to be detected. S2: Construct a dual-branch progressive fusion detection network. This network is based on the YOLO single-stage detection framework, retains the detection head and bounding box regression structure, and sets the backbone feature extraction part as a dual-branch structure, namely, a severe weather degradation feature extraction branch and a restoration guidance feature extraction branch. The severe weather degradation feature extraction branch is used to directly extract the degradation features corresponding to the target contour, target texture, target scale, weather disturbance response, and contextual semantics from the severe weather degradation image to be detected. The restoration guidance feature extraction branch is used to perform unified restoration modeling on the weather degradation mapping in the severe weather degradation image to be detected to obtain restoration guidance features. S3: Input the degradation features and recovery guidance features into the attention fusion module in the severe weather degradation feature extraction branch for weighted fusion to obtain the fused detection features; S4: A progressive fusion training strategy with alternating freezes and a total loss function consisting of detection loss, recovery loss and robust consistency loss are adopted to perform staged collaborative optimization of object detection, image restoration and feature consistency. S5: Input the fused detection features into the detection head, and output the final detection results after confidence filtering and non-maximum suppression, including traffic target category probability, target confidence and detection box coordinates.

[0010] Further, in step S1, the clean image-degraded image pair consists of a clean traffic scene image and a corresponding degraded image; the degraded image is obtained by artificially synthesizing and mapping the clean traffic scene image under severe weather conditions, and the weather degradation synthesis parameters of the artificially synthesized and mapped severe weather conditions include at least one of fog concentration parameters, rain line density parameters, raindrop occlusion parameters, snowflake occlusion parameters, low illumination disturbance parameters, and composite weather degradation parameters; the preprocessing includes scale normalization and pixel normalization, adjusting the degraded image to the input size of the detection network and performing pixel normalization processing to obtain the severe weather degradation image to be detected.

[0011] Furthermore, in step S2, the recovery guidance feature extraction branch adopts an encoder-decoder structure, which includes a dynamic inverted bottleneck module, a DG-C2f module, and a prompt word guidance module. The prompt word guidance module generates a weather prompt vector based on the global degradation representation of the input image, and uses the weather prompt vector to modulate the coding features, thereby uniformly recovering fog, rain lines, snowflakes, low light, and composite degradation.

[0012] in, The input image represents a degraded image due to severe weather. and These represent the convolution kernel as follows: and Convolution operation, This represents a lightweight feature aggregation module. Represents the dynamic bottleneck module. The module is guided by key prompts. Indicates splicing along the channel. Represents upsampling, Represents downsampling; This represents a degraded image caused by severe weather. Shallow degradation features obtained through downsampling operations and extraction by the DG-C2f module. Indicates by The mid-layer degradation features were further obtained through downsampling operations and extraction by the DG-C2f module. Indicates by Further, the high-level degradation features were obtained through downsampling operations and extraction by the DG-C2f module. Indicates by Compressed high-level features obtained after downsampling operations; Indicates by Initial deep repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The first layer of decoded and fused features is obtained by splicing and merging the features, followed by convolution operation with a 1×1 kernel. Indicates by The second-layer decoding and repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The second layer of decoded and fused features is obtained by concatenating and fusing the features, followed by a 1×1 convolution operation. Indicates by The third-layer decoding and repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The third layer of decoded and fused features is obtained by splicing and merging, followed by convolution operation with a 1×1 kernel; Indicates by Detailed recovery features obtained after processing by the DG-C2f module; Indicates by Residual repair features obtained by convolution operation with a 3×3 kernel; This indicates that the image will be degraded due to severe weather. and ( and The dimensions remain consistent because , Neither upsampling nor downsampling changes the dimension, only adjusting the number of channels; upsampling and downsampling are each performed 3 times, equivalent to no change in dimension. The residual image features are obtained by element-wise summing the results of convolution operations with a 3×3 kernel. Indicates will The recovery boot characteristics obtained after processing by the dynamic bottleneck module; Represents the convolutional coding layer. This represents the decoding and reconstruction layer.

[0013] Furthermore, in step S2, the severe weather degradation feature extraction branch includes a dynamic inverted bottleneck module, an attention fusion module, and an SPPF module, where SPPF represents fast spatial pyramid pooling; the dynamic inverted bottleneck module in the severe weather degradation feature extraction branch is used to dynamically and adaptively enhance the feature of the severe weather degradation image to be detected, thereby obtaining degradation features; The attention fusion module generates fusion weights based on degradation features and recovery guidance features, and performs feature fusion based on the fusion weights to obtain fused features. When there is excessive smoothing or artifacts in the recovery guidance features, the attention fusion module increases the weight of degradation features. When the degradation image to be detected in severe weather has severe fogging, rain or snow occlusion, or low contrast, the attention fusion module increases the weight of recovery guidance features. The fused features are then processed by the SPPF module to obtain fused detection features.

[0014] Furthermore, the specific expression for the severe weather degradation feature extraction branch is as follows:

[0015] in, Indicates fusion detection features, Represents the dynamic bottleneck module. Represents dynamic convolution. This indicates fast spatial pyramid pooling. Indicates that the convolution kernel is Ordinary convolution, This represents the attention fusion module.

[0016] Furthermore, the dynamic inverted bottleneck module includes a pre-convolution, a channel splitting structure, at least two parallel dynamic inverted bottleneck units, a feature concatenation structure, and a post-convolution; the input image features are split into at least two branch features along the channel dimension after the pre-convolution, and each branch feature is input into the corresponding dynamic inverted bottleneck unit for enhancement processing. The enhanced branch features are fused by the feature concatenation structure and then mapped to degenerate features by the post-convolution. The dynamic inverted bottleneck unit includes a cascaded structure of 3×3 dynamic convolution, 1×1 ordinary convolution, and 3×3 dynamic convolution. It generates conditional convolution weights through global average pooling, multilayer perceptron, and activation functions, enabling the dynamic convolution to adaptively adjust the convolution response according to the input image features, thereby enhancing the expression of target features under different weather degradation intensities and target scales.

[0017] Furthermore, in step S4, the progressive fusion training strategy of alternating freezing includes a recovery perception pre-training stage and a progressive detection optimization stage; in the recovery perception pre-training stage, pre-trained detection weights are loaded, the severe weather degradation feature extraction branch is frozen, and the recovery guidance feature extraction branch and attention fusion module are trained; in the progressive detection optimization stage, the recovery guidance feature extraction branch is frozen, and the SPPF module, attention fusion module, one or more dynamic inverted bottleneck modules, and complete severe weather degradation feature extraction branch are unfrozen in sequence according to the number of training rounds.

[0018] Furthermore, in step S4, the total number of training rounds in the alternating freeze progressive fusion training strategy is...E The threshold for the first stage is The gradual thawing interval is When the number of training rounds At that time, the severe weather degradation feature extraction branch was frozen and jointly trained using detection loss, recovery loss, and robust consistency loss; when the number of training rounds... At that time, freeze the recovery guide feature extraction branch, and every At least one structural unit in the feature extraction branch of the severe weather detection during the thawing cycle.

[0019] Further, in step S4, the detection loss is used to constrain the target category probability, target confidence, and bounding box regression; the recovery loss is used to constrain the pixel difference between the recovered image and the corresponding clean image; the robust consistency loss is used to constrain the consistency of the channel probability distribution obtained after global average pooling and normalization of the degraded features and the recovered guiding features; the detection loss, recovery loss, and robust consistency loss are each assigned corresponding weights, and the weights of the recovery loss and robust consistency loss change with the training phase.

[0020] Furthermore, in step S5, the fused detection features are input into the detection head to first obtain candidate detection results, including the traffic target category probability, target confidence, and bounding box coordinates corresponding to the candidate boxes. During the inference process, low-confidence candidate boxes are first removed according to the confidence threshold, and then duplicate candidate boxes are removed according to the non-maximum suppression threshold to obtain the final traffic scene target detection results.

[0021] The beneficial effects of this invention are as follows: 1) Enhanced complementary capabilities of dual-branch features. This invention directly extracts target contours, textures, scales, and contextual semantics through the severe weather degradation feature extraction branch, models weather degradation mapping by restoring the guided feature extraction branch, and utilizes an attention fusion module for adaptive coupling to improve the collaborative expression capability of degradation suppression information and detection semantic information.

[0022] 2) Enhanced adaptability to complex weather degradation. The recovery guidance feature extraction branch of this invention utilizes a prompt guidance module, a dynamic inverted bottleneck module, and a DG-C2f module to perform unified recovery modeling for fog, rain lines, snowflakes, low light, and composite degradation, thereby improving the model's adaptability to various types of severe weather traffic scenarios.

[0023] 3) Enhanced dynamic lightweight feature representation capability. This invention employs a dynamic inverted bottleneck module and a DG-C2f lightweight feature transformation structure, which improves the network's adaptive feature representation capability for different degradation intensities, target scales, and local regions while controlling the number of parameters and computational cost.

[0024] 4) Improved stability of multi-task collaborative training. This invention adopts a progressive fusion training strategy of alternating freezing, first stabilizing and restoring the guiding feature extraction branch, and then gradually releasing the learning ability of the severe weather degradation feature extraction branch, thereby alleviating the optimization conflict between image restoration tasks and object detection tasks.

[0025] 5) Improved robustness of traffic target detection in severe weather. This invention reduces the interference of recovery noise, oversmoothing, and artifacts on the detection results by constraining the consistency of the channel probability distribution of the degenerate features and the recovery guidance features through robust consistency loss, thereby improving the accuracy and robustness of traffic target detection under complex and severe weather conditions.

[0026] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is an overall flowchart of the target detection method for severe weather traffic scenarios based on dual-branch progressive fusion, as described in this invention. Figure 2 A schematic diagram of the backbone structure of a dual-branch progressive fusion detection network; Figure 3 This is a schematic diagram of the dynamic inverted bottleneck module structure; Figure 4 This is a comparison of the detection results of the method proposed in this invention under snow and fog conditions. Detailed Implementation

[0028] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0029] Please see Figures 1-3 This invention provides a target detection method for severe weather traffic scenes based on dual-branch fusion, specifically including the following steps: S1: Obtain clean traffic scene images, perform artificial synthesis of severe weather on the clean traffic scene images to construct clean image-degraded image pairs, and perform scale normalization and pixel normalization preprocessing on the clean image-degraded image pairs to obtain the degraded image to be detected.

[0030] The expressions for the road image set, the image to be detected, and the preprocessing procedure are as follows:

[0031] in, This represents a collection of images depicting clean traffic scenes. Indicates the first A clean traffic scene image. This indicates the number of images representing a clean traffic scene. and These represent the height and width of the original clean traffic scene image, respectively. Represents the three-channel image space. Indicates preprocessing mapping, and These represent the height and width of the detection network input, respectively. This represents an artificially synthesized mapping of severe weather. Represents the set of composite parameters for weather degradation. Represents a set of degraded images. Represents a clean image - degraded image pair; This represents the set of data augmentation parameters, including random scaling parameters, color perturbation parameters, and flipping parameters; This represents the severe weather degradation image to be detected as input to the detection network.

[0032] S2: Construct a dual-branch progressive fusion detection network, including a recovery-guided feature extraction branch, a severe weather degradation feature extraction branch, and a detection head, and introduce robust consistency loss during the training phase.

[0033] (1) Using the recovery guidance feature extraction branch composed of the prompt word guidance module, the dynamic inverted bottleneck module and the DG-C2f module, the weather degradation mapping in the clean image-degraded image pair is uniformly restored and modeled to obtain the recovery guidance features.

[0034] A prompt-guided module is used to uniformly model various weather degradation types, enabling the same recovery branch to adapt to fog, rain, snow, low light, and mixed weather scenarios. The expression is as follows:

[0035] in, The input image represents a degraded image due to severe weather. and These represent the convolution kernel as follows: and Convolution operation, This represents a lightweight feature aggregation module. Represents the dynamic bottleneck module. The module is guided by key prompts. Indicates splicing along the channel. Represents upsampling, Represents downsampling; This represents a degraded image caused by severe weather. Shallow degradation features obtained through downsampling operations and extraction by the DG-C2f module. Indicates by The mid-layer degradation features were further obtained through downsampling operations and extraction by the DG-C2f module. Indicates by Further, the high-level degradation features were obtained through downsampling operations and extraction by the DG-C2f module. Indicates by Compressed high-level features obtained after downsampling operations; Indicates by Initial deep repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The first layer of decoded and fused features is obtained by splicing and merging the features, followed by convolution operation with a 1×1 kernel. Indicates by The second-layer decoding and repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The second layer of decoded and fused features is obtained by concatenating and fusing the features, followed by a 1×1 convolution operation. Indicates by The third-layer decoding and repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The third layer of decoded and fused features is obtained by splicing and merging, followed by convolution operation with a 1×1 kernel; Indicates by Detailed recovery features obtained after processing by the DG-C2f module; Indicates by Residual repair features obtained by convolution operation with a 3×3 kernel; This indicates that the image will be degraded due to severe weather. and The residual-enhanced image features are obtained by element-wise summing the results of convolution operations with a 3×3 kernel. Indicates will The recovery boot characteristics obtained after processing by the dynamic bottleneck module; Represents the convolutional coding layer. This represents the decoding and reconstruction layer.

[0036] (2) The severe weather degradation feature extraction branch includes the dynamic inverted bottleneck module, the attention fusion module and the SPPF module.

[0037] In the severe weather degradation feature extraction branch, the dynamic inverted bottleneck module utilizes dynamic convolution to adaptively change the convolution kernel response with the input degradation features. It expands the dimensionality of intermediate feature representation through the inverted bottleneck structure and preserves the original detection semantics through residual connections. The expression is:

[0038] in, Represents the features of the input image. Represents dynamic convolution. Indicates splitting along the channel number, Indicates that the convolution kernel is Convolution operation, Represents the number of dynamic bottleneck units. Indicates by After convolution kernel The initial mapping features obtained from the operation, and They represent respectively by The first branch features and the second branch features obtained from the split. This represents the adaptive enhancement feature output by the dynamic bottleneck module. The attention fusion module generates fusion weights based on the degradation features and the recovery guidance features, and generates fusion features based on the fusion weights. When there is excessive smoothing or artifacts in the recovery guidance features, the attention fusion module increases the weight of the degradation features. When the image to be detected in severe weather degradation has severe fogging, rain or snow occlusion, or low contrast, the attention fusion module increases the weight of the recovery guidance features.

[0039] S3: The attention fusion module generates fusion weights based on degradation features and restoration guidance features, and performs feature fusion based on these weights to obtain fused features. These fused features are then processed by the SPPF module to obtain fused detection features. The attention fusion module dynamically controls the contribution ratio of restoration guidance features and degradation detection features, enabling the network to utilize restoration guidance information more in severely degraded regions and retain more target semantic information in degraded images in regions with restoration artifacts.

[0040] The expression for extracting degradation features due to severe weather is as follows:

[0041] in, Indicates fusion detection features, Represents the dynamic bottleneck module. Represents dynamic convolution. This indicates fast spatial pyramid pooling. Indicates that the convolution kernel is Ordinary convolution; Representing the attention fusion module, the expression is:

[0042] in, This is the initial degradation-recovery fusion feature for degradation detection features and recovery guidance features. To restore the guiding characteristics, For degradation detection features, This represents the degenerate-recovery fusion context features obtained by performing global average pooling in the width direction and global average pooling in the height direction on the initial degenerate-recovery fusion features, and then mapping them through a convolution operation with a 1×1 kernel. and They respectively represent the following: According to the number of degradation detection feature channels and restore the number of characteristic channels The degraded weight features and restored weight features obtained after channel splitting are: Indicates by and The degenerate-recovery fused attention weights are generated by element-wise multiplication and a sigmoid activation function. This indicates element-wise multiplication; This indicates that the attention weights of the degradation detection features are applied based on the degradation-recovery fusion features. and restore guidance features The degradation-recovery fusion features obtained after adaptive weighted fusion This represents splicing along the channel direction; This indicates that global average pooling is performed along the width direction. This indicates that global average pooling is performed along the height direction. This represents an adjustment of the order of dimensions.

[0043] S4: During training, a progressive fusion training strategy with alternating freezes is adopted, and the total loss function consisting of detection loss, recovery loss and robust consistency loss is used for staged optimization.

[0044] Furthermore, in step S3, a progressive fusion training strategy with alternating freezes is used to optimize the model in stages, and the robustness of the model under severe weather scenarios is improved by using a total loss function consisting of detection loss, recovery loss, and robust consistency loss. The specific training strategy and loss function are as follows: During training, the model weights pre-trained on the COCO dataset are first loaded, and the severe weather degradation feature extraction branch is frozen, retaining only the recovery guidance branch for training. In the early stages of training, when the number of training epochs is less than 100, the model primarily optimizes the recovery guidance branch. The total loss function in this stage consists of the detection loss, recovery loss, and robust consistency loss, each weighted by its corresponding weight coefficient to enhance the model's ability to guide the recovery of degraded images while maintaining object detection capabilities. During training, the weights of each loss term are updated every 20 training epochs, gradually adjusting the contribution of different loss terms during training to achieve phased optimization.

[0045] After 100 training epochs, the weights of the recovery guidance branch are frozen, and some modules in the severe weather degradation feature extraction branch are gradually unfrozen, allowing detection-related network structures to participate in joint optimization. Specifically, when the number of training epochs is not less than 100, the SPPF module and a fusion module consisting of a dynamic inverted bottleneck module and a dynamic convolution module are unfrozen; when the number of training epochs is not less than 120, the AFM module and another fusion module consisting of a dynamic inverted bottleneck module and a dynamic convolution module are further unfrozen; when the number of training epochs is not less than 140, the complete severe weather degradation feature extraction branch is unfrozen, allowing the recovery guidance branch and the severe weather degradation feature extraction branch to be jointly adapted at a higher level. In this stage, the model is mainly optimized based on the detection loss to enhance target detection performance in complex degradation scenarios.

[0046] Through the aforementioned alternating freeze and progressive thawing strategy, the model prioritizes learning recovery guidance capabilities in the early stages of training. In the later stages, it gradually introduces branches for extracting degradation features under severe weather conditions and joint optimization of cross-module fusion structures, thereby avoiding mutual interference between different tasks in the early stages of end-to-end joint training. This training strategy promotes the gradual alignment of recovery guidance features and degradation features, improves the network's robustness under severe weather and complex degradation conditions, and ultimately yields a dual-branch progressive fusion detection network.

[0047] The expressions for the total model loss, detection loss, recovery loss, and robust consistency loss are as follows:

[0048] Among them, detection loss Mainly composed of bounding box loss Classification loss With focus loss composition; , and These represent the weights of the bounding box loss, classification loss, and distribution focus loss, respectively. This represents the number of positive samples. and These represent the predicted bounding box and the ground truth bounding box, respectively. Indicates the first The positive sample belongs to the th . Predicted probability of class Indicates category label, Indicates the number of categories. , , , These represent the left, top, right, and bottom distances from the prediction box to the anchor point, respectively. Indicates the first j The true continuous distance values ​​in each direction. Represents true continuous distance The right-hand discrete integer points, Represents true continuous distance The discrete integer points on the left, Indicates the first i The positive sample in the th... j The prediction in each direction is an integer discrete point on the left. The probability of; Indicates the first i A positive sample is predicted as a right-side discrete integer point in the j-th direction. The probability of.

[0049]

[0050] in, and These represent recovery loss and robust consistency loss, respectively. and These represent the restored image and the clean image, respectively. and represent the normalized probability distributions of the degradation features and the recovery-guided features, respectively. and These represent the batch size and the number of channels, respectively. Represents the total loss of the model. Represents the detection loss weighting coefficient. Represents the recovery loss weighting coefficient , Represents the robustness loss weight coefficient .

[0051] S5: Input the fused detection features into the detection head, and output the final traffic scene target detection results after confidence filtering and non-maximum suppression, including target category probability, target confidence and detection box coordinates. Figure 4 The results of the proposed method under snow and fog conditions are compared, where (a)-(d) are the detection results of the proposed method, (e)-(h) are the YOLOv8 detection results, and (i)-(l) are the target detection ground truth values.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A target detection method for severe weather traffic scenes based on dual-branch fusion, characterized in that, The method specifically includes the following steps: S1: Acquire clean traffic scene images and perform severe weather artificial synthesis to construct clean image-degraded image pairs. Preprocess the images in the clean image-degraded image pairs to obtain the severe weather degraded images to be detected. The clean image-degraded image pairs consist of clean traffic scene images and corresponding degraded images. The degraded images are obtained by mapping the clean traffic scene images through severe weather artificial synthesis. The weather degradation synthesis parameters of the severe weather artificial synthesis mapping include at least one of the following: fog concentration parameters, rain line density parameters, raindrop occlusion parameters, snowflake occlusion parameters, low light disturbance parameters, and composite weather degradation parameters. S2: Construct a dual-branch progressive fusion detection network. This network is based on the YOLO single-stage detection framework, retains the detection head and bounding box regression structure, and sets the backbone feature extraction part as a dual-branch structure, namely, a severe weather degradation feature extraction branch and a restoration guidance feature extraction branch. The severe weather degradation feature extraction branch is used to directly extract the degradation features corresponding to the target contour, target texture, target scale, weather disturbance response, and contextual semantics from the severe weather degradation image to be detected. The restoration guidance feature extraction branch is used to perform unified restoration modeling on the weather degradation mapping in the severe weather degradation image to be detected to obtain restoration guidance features. The recovery guidance feature extraction branch adopts an encoder-decoder structure, which includes a dynamic inverted bottleneck module, a DG-C2f module, and a prompt word guidance module. The prompt word guidance module generates a weather prompt vector based on the global degradation representation of the input image, and uses the weather prompt vector to modulate the coding features, thereby uniformly recovering fog, rain lines, snowflakes, low light, and composite degradation. The severe weather degradation feature extraction branch includes a dynamic inverted bottleneck module, an attention fusion module, and an SPPF module, where SPPF stands for Fast Spatial Pyramid Pooling. The dynamic inverted bottleneck module in the severe weather degradation feature extraction branch is used to dynamically and adaptively enhance the feature of the severe weather degradation image to be detected, thereby obtaining degradation features. The attention fusion module generates fusion weights based on the degradation features and the recovery guidance features, and performs feature fusion based on the fusion weights to obtain fused features. When there is excessive smoothing or artifacts in the recovery guidance features, the attention fusion module increases the weight of the degradation features. When the severe weather degradation image to be detected has severe fogging, rain and snow occlusion, or low contrast, the attention fusion module increases the weight of the recovery guidance features. The fused features are then processed by the SPPF module to obtain the fused detection features. S3: Input the degradation features and recovery guidance features into the attention fusion module in the severe weather degradation feature extraction branch for weighted fusion to obtain the fused detection features; S4: A progressive fusion training strategy with alternating freezes and a total loss function consisting of detection loss, recovery loss and robust consistency loss are adopted to perform staged collaborative optimization of object detection, image restoration and feature consistency. The progressive fusion training strategy of alternating freezing includes a recovery perception pre-training stage and a progressive detection optimization stage. In the recovery perception pre-training stage, pre-trained detection weights are loaded, the severe weather degradation feature extraction branch is frozen, and the recovery guidance feature extraction branch and attention fusion module are trained. In the progressive detection optimization stage, the recovery guidance feature extraction branch is frozen, and the SPPF module, attention fusion module, one or more dynamic inverted bottleneck modules, and complete severe weather degradation feature extraction branch are unfrozen in sequence according to the number of training rounds. The detection loss is used to constrain the target category probability, target confidence, and bounding box regression; the recovery loss is used to constrain the pixel difference between the recovered image and the corresponding clean image; the robust consistency loss is used to constrain the consistency of the channel probability distribution obtained after global average pooling and normalization of the degraded features and the recovery guiding features; the detection loss, recovery loss, and robust consistency loss are each assigned corresponding weights, and the weights of the recovery loss and robust consistency loss change with the training phase; S5: Input the fused detection features into the detection head, and output the final traffic scene target detection results after confidence filtering and non-maximum suppression, including traffic target category probability, target confidence and detection box coordinates.

2. The target detection method for severe weather traffic scenes based on dual-branch fusion according to claim 1, characterized in that, In step S1, the preprocessing includes scale normalization and pixel normalization. The degraded image is adjusted to the input size of the detection network and pixel normalization is performed to obtain the degraded image of severe weather to be detected.

3. The target detection method for severe weather traffic scenes based on dual-branch fusion according to claim 1, characterized in that, In step S2, the recovery guidance feature extraction branch is represented as follows: in, The input image represents a degraded image due to severe weather. and These represent the convolution kernel as follows: and Convolution operation, This represents a lightweight feature aggregation module. Represents the dynamic bottleneck module. The module is guided by key prompts. Indicates splicing along the channel. Represents upsampling, Represents downsampling; This represents a degraded image caused by severe weather. Shallow degradation features obtained through downsampling operations and extraction by the DG-C2f module. Indicates by The mid-layer degradation features were further obtained through downsampling operations and extraction by the DG-C2f module. Indicates by Further, the high-level degradation features were obtained through downsampling operations and extraction by the DG-C2f module. Indicates by Compressed high-level features obtained after downsampling operations; Indicates by Initial deep repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The first layer of decoded and fused features is obtained by splicing and merging the features, followed by convolution operation with a 1×1 kernel. Indicates by The second-layer decoding and repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The features are then concatenated and fused, and then subjected to a second layer of decoding and fusion through a 1×1 convolution operation. Indicates by The third-layer decoding and repair features obtained after processing by the DG-C2f module; Indicates will After enhancement by the prompt word guidance module and the dynamic bottleneck module, the upsampling and encoding stages are performed. The third layer of decoded and fused features is obtained by splicing and merging, followed by convolution operation with a 1×1 kernel; Indicates by Detailed recovery features obtained after processing by the DG-C2f module; Indicates by Residual repair features obtained by convolution operation with a 3×3 kernel; This indicates that the image will be degraded due to severe weather. and The residual-enhanced image features are obtained by element-wise summing the results of convolution operations with a 3×3 kernel. Indicates will The recovery boot characteristics obtained after processing by the dynamic bottleneck module; Represents the convolutional coding layer. This represents the decoding and reconstruction layer.

4. The target detection method for severe weather traffic scenes based on dual-branch fusion according to claim 1, characterized in that, The specific expression for the severe weather degradation feature extraction branch is as follows: in, Indicates fusion detection features, Represents the dynamic bottleneck module. Represents dynamic convolution. This indicates fast spatial pyramid pooling. Indicates that the convolution kernel is Ordinary convolution, This represents the attention fusion module.

5. The target detection method for severe weather traffic scenes based on dual-branch fusion according to claim 1 or 3, characterized in that, The dynamic inverted bottleneck module includes a pre-convolution, a channel splitting structure, at least two parallel dynamic inverted bottleneck units, a feature concatenation structure, and a post-convolution. The input image features are split into at least two branch features along the channel dimension after the pre-convolution. Each branch feature is input into the corresponding dynamic inverted bottleneck unit for enhancement processing. The enhanced branch features are fused by the feature concatenation structure and then mapped to degenerate features by the post-convolution. The dynamic inverted bottleneck unit includes a cascaded structure of 3×3 dynamic convolution, 1×1 ordinary convolution, and 3×3 dynamic convolution. It generates conditional convolution weights through global average pooling, multilayer perceptron, and activation functions, enabling the dynamic convolution to adaptively adjust the convolution response according to the input image features, thereby enhancing the expression of target features under different weather degradation intensities and target scales.

6. The target detection method for severe weather traffic scenes based on dual-branch fusion according to claim 1, characterized in that, In step S4, the total number of training rounds in the alternating freeze progressive fusion training strategy is: E The threshold for the first stage is The gradual thawing interval is When the number of training rounds At that time, the severe weather degradation feature extraction branch was frozen and jointly trained using detection loss, recovery loss, and robust consistency loss; when the number of training rounds... At that time, freeze the recovery guide feature extraction branch, and every At least one structural unit in the feature extraction branch of the severe weather detection during the thawing cycle.

7. The target detection method for severe weather traffic scenes based on dual-branch fusion according to claim 1, characterized in that, In step S5, the fused detection features are input into the detection head to first obtain candidate detection results, including the traffic target category probability, target confidence, and bounding box coordinates corresponding to the candidate boxes. During the inference process, low-confidence candidate boxes are first removed according to the confidence threshold, and then duplicate candidate boxes are removed according to the non-maximum suppression threshold to obtain the final traffic scene target detection results.

Images