A traffic sign detection method based on gated fusion and pseudo-label dynamic screening

Abstract

This invention discloses a traffic sign detection method based on gating fusion and dynamic pseudo-label selection, comprising the following steps: S1: Data preparation and preprocessing, acquiring a small number of labeled traffic sign images and a large number of unlabeled traffic sign images; S2: Constructing a gating feature fusion mechanism to optimize and recall candidate boxes; S3: Constructing a pseudo-label dynamic selection module based on category distribution, fusing statistical priors and semantic priors, dynamically adjusting the pseudo-label selection thresholds for each category, and generating high-quality pseudo-labels; S4: Introducing an adaptive weight loss mechanism to dynamically weight pseudo-labels to reduce noise impact; S5: Iterative training using a teacher-student framework to output traffic sign detection results. This invention improves small target detection capability through gating fusion, alleviates category imbalance through dynamic selection, and reduces the impact of pseudo-label noise, significantly improving detection accuracy and generalization performance under conditions of limited labeling.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent transportation technology, specifically relating to a traffic sign detection method based on gating fusion and dynamic screening of pseudo-labels. Background Technology

[0002] With the rapid development of intelligent driving technology, traffic sign detection, as a crucial task in environmental perception, has a significant impact on driving safety due to its accuracy and real-time performance. Semi-supervised learning, by utilizing a small amount of labeled data alongside a large amount of unlabeled data, provides an effective way to alleviate the problem of insufficient data. However, existing semi-supervised object detection methods still have certain limitations: on the one hand, most methods use a uniform confidence threshold for pseudo-label screening, which is difficult to adapt to uneven class distribution, easily leading to overfitting of the head class and insufficient recognition ability of the tail class; on the other hand, in complex traffic scenarios, the feature representation of small-scale or distant targets is weak, their prediction confidence is low, and they are easily misclassified as background, resulting in missed detections.

[0003] Furthermore, existing methods typically lack effective feature fusion mechanisms for feature utilization, and noise is inevitably present in pseudo-labels. Without reasonable screening and weighting mechanisms, this can negatively impact model training. While some improved methods based on two-stage detectors can enhance detection performance to some extent, their complex structure and high computational cost make them difficult to meet the real-time requirements of practical applications.

[0004] Therefore, how to improve the quality of pseudo-labels, alleviate class imbalance, and enhance the detection capability of multi-scale, especially small targets, while reducing noise interference under the semi-supervised learning framework has become an urgent technical problem to be solved in the field of traffic sign detection. Summary of the Invention

[0005] The purpose of this invention is to address the problems of uneven category distribution, easy omission of small targets, and large pseudo-label noise in existing semi-supervised traffic sign detection technologies. This invention proposes a traffic sign detection method based on gating fusion and dynamic pseudo-label screening. This method improves the quality of pseudo-labels and enhances the multi-scale feature expression capability through a category distribution-aware dynamic pseudo-label screening mechanism and a gating feature fusion strategy, thereby improving detection accuracy and model generalization performance, while reducing dependence on labeled data.

[0006] The present invention adopts the following technical solution:

[0007] A traffic sign detection method based on gating fusion and dynamic pseudo-label filtering includes the following steps:

[0008] S1: Data collection and preprocessing, acquiring a small number of labeled traffic sign images and a large number of unlabeled traffic sign images;

[0009] S2: Construct a gated feature fusion mechanism to optimize and recall candidate boxes by fusing features at different scales and calculating the confidence gain before and after fusion;

[0010] S3: Construct a pseudo-label dynamic filtering module based on category distribution, integrate statistical prior and semantic prior, dynamically adjust the pseudo-label filtering threshold of each category, and generate high-quality pseudo-labels;

[0011] S4: Introduce an adaptive weight loss mechanism to dynamically weight pseudo-labels to reduce the impact of noise;

[0012] S5: Iterative training is performed using a teacher-student framework to output traffic sign detection results.

[0013] Preferably, in step S1, for the small number of labeled traffic sign images and the large number of unlabeled traffic sign images, blurry images with glare or low resolution are deleted. Then, the Labelimg tool is used to refine the annotations on the labeled images, including the traffic sign category (e.g., prohibition, instruction, warning), bounding box position, and scale information. Finally, the labeled dataset is divided into training, validation, and test sets in an 8:1:1 ratio, ensuring that the validation and test sets are fully labeled and evaluated independently. Unlabeled data is only used in the semi-supervised training phase, working in conjunction with the labeled training set to improve model performance.

[0014] Preferably, in step S2, a gated feature fusion mechanism is constructed to optimize and recall candidate boxes by fusing features at different scales and calculating the confidence gain before and after fusion. Specifically, this includes the following:

[0015] (1) Construct a gated fusion feature pyramid. The teacher model extracts features from the original scale image and the downsampled scale image respectively to obtain the original features and downsampled features of the corresponding level; for the first level... Each feature layer first converts the original features... With downsampling features The data is concatenated along the channel dimension, and then a fusion weight is generated using a gating function. :

[0016] (1)

[0017] in, Indicates the first Gated fusion weights of each feature layer This represents the Sigmoid activation function. This represents a multilayer perceptron. This indicates a global average pooling operation. Represents the original scale image at the 1st... Features of layer output, Indicates the downsampled scale image at the 1st... Features of the layer output;

[0018] (2) Feature fusion generation: Based on the gated fusion weights, features of different scales are adaptively weighted to obtain fused scale features. :

[0019] (2)

[0020] in, Indicates the first The network integrates scale features from multiple feature layers. This approach enables the network to adaptively retain detailed information from the original scale features and semantic information from the downsampled scale features, based on the target scale features.

[0021] (3) Candidate box matching: prediction is performed based on the original features and the fused scale features to obtain the original set of prediction boxes. and fusion prediction box set For candidate boxes in the original predicted box set that are located in the fuzzy region Candidate box matching relationships are established by calculating the intersection-union ratio (IU) with each predicted box in the fused predicted box set.

[0022] (3)

[0023] in, Indicates the relationship with the original candidate box The best-matching fusion prediction box, This represents the cross-union ratio between candidate boxes; when the maximum cross-union ratio is greater than the preset matching threshold, it is determined that the original candidate box and the fused prediction box correspond to the same target.

[0024] (4) Calculate the joint confidence score before and after fusion. Calculate the product of the classification confidence score and the localization quality confidence score for both the original candidate boxes and the fused candidate boxes to obtain the joint confidence score before and after fusion:

[0025]

[0026] in, This represents the joint confidence level of the original candidate boxes. This represents the joint confidence level of the fused candidate boxes. and These represent the classification confidence of the candidate boxes under the original features and the fused features, respectively. and These represent the localization quality scores of the candidate boxes under the original features and the fused features, respectively.

[0027] (5) Candidate box generation, the fusion confidence gain is defined as:

[0028] (6)

[0029] in, This represents the joint confidence gain of the candidate box in the merged view relative to the original view. Candidate boxes Joint confidence level in the fusion pyramid Candidate boxes Joint confidence in the original view;

[0030] When the confidence gain corresponding to the fuzzy sample Greater than the preset threshold This indicates that the target's prediction is unstable in the original view due to scale differences or insufficient feature representation, but is effectively enhanced in the fused view. Therefore, candidate boxes that meet the above conditions are selected from the fuzzy candidate box set, and the higher-quality predicted boxes in the fused view are used as candidate boxes. At the same time, candidate boxes that meet the high confidence condition in the fused feature pyramid prediction results are also directly retained. The two types of candidate boxes are merged into the preferred candidate boxes as the final candidate box set used for subsequent pseudo-label dynamic screening and model training.

[0031] The above steps utilize the gain of joint confidence before and after fusion to perform secondary verification on candidate boxes that are in a blurred state in the original view, thereby recalling small targets that are easily missed at the original scale, improving the quality of false labels and the recall rate of traffic sign detection.

[0032] Preferably, the pseudo-label dynamic filtering module based on category distribution in S3 is defined as follows:

[0033] (1) Category semantic mapping and text embedding construction: In view of the problem that traffic sign category labels are usually engineering abbreviations and lack direct semantic expression, a classification label space is constructed. To Natural Language Description Space mapping function And embed the mapped category description into the preset prompt template to generate the first Text embedding of classes :

[0034] (7)

[0035] in, Indicates the first Natural language semantic descriptions corresponding to traffic sign category codes. Indicates the first Text feature embedding of categories; this step is used to enhance the semantic expressive power of category text features and provide semantic priors for subsequent category distribution estimation;

[0036] (2) Based on CLIP semantic distribution estimation, the teacher model is used to perform preliminary detection on samples in the unlabeled dataset, and the predicted bounding boxes are cropped and input into the CLIP image encoder to extract the first bounding box. Visual features of each sample ; By calculating visual features Text embedding with various categories The similarity between them determines which sample belongs to the first group. Semantic prediction probability of class:

[0037] (8)

[0038] in, Represents the cosine similarity function. Indicates the temperature coefficient. Indicates the total number of categories. Indicates sample Belongs to the Semantic prediction probability of the class;

[0039] Statistical aggregation of the semantic prediction probabilities of the unlabeled sample subset is performed to obtain the first... Prior semantic distribution of classes This semantic distribution prior is used to supplement the empirical distribution information obtained only from labeled data statistics, so as to improve the robustness of long-tail category distribution estimation.

[0040] (3) Fusion of statistical prior and semantic prior, combining the first prior obtained from the statistical analysis of labeled data. Class Statistical Prior Compared with the first estimate obtained by CLIP Class semantic prior We perform weighted fusion to obtain the calibrated prior distribution of categories. :

[0041] (9)

[0042] in, For distributed fusion weights, This indicates the first number obtained based on statistics from labeled data. Similar to empirical distribution, The first one obtained based on CLIP semantic reasoning Class semantic distribution, Indicates the fused first Prior distribution of class categories;

[0043] Through the above fusion mechanism, the supervisory accuracy of the labeled data is preserved, and open-world semantic knowledge is introduced to perform zero-sample calibration for long-tail categories.

[0044] (4) Dynamic threshold calculation based on category distribution, for category Let the predicted score generated by the teacher model on the unlabeled set be... And sorted in descending order as ,in, This represents the total number of unlabeled candidate samples. Indicates category The The predicted scores are sorted in descending order.

[0045] To filter out low-reliability samples near the decision boundary, a positive sample confidence interval ratio is introduced. and the proportion of negative sample confidence intervals And based on the fused category prior Dynamically determine the first positive sample threshold of class and negative sample threshold :

[0046] (10)

[0047] in, Indicates the first The threshold for positive samples in pseudo-label filtering. Indicates the first The threshold for negative samples in pseudo-label filtering, and ;

[0048] The above thresholds change dynamically with the category distribution: for the head category, because For larger values, the threshold position is shifted forward accordingly to suppress excessive false positives; for tail categories, because... The threshold is relatively small, and it automatically adapts to the lower confidence distribution, thereby improving the recall rate;

[0049] (5) Pseudo-label dynamic screening rules for unlabeled samples The teacher model outputs its category Predicted scores Then the pseudo-label of the sample Determined by the following rules:

[0050] (11)

[0051] in, Indicates the sample Determined as the first Positive samples, Indicates that it is determined to be the first Negative samples, This indicates that the sample is in the uncertainty interval and will not participate in the construction of the unsupervised supervision signal for this category in the current iteration;

[0052] By using the above filtering rules, the number of pseudo-labels in each category is kept consistent with the distribution of the real categories, thus avoiding overfitting of the head category by a uniform global threshold and systematic suppression of the tail category.

[0053] Preferably, the adaptive weight Focal Loss in S4 is defined as follows:

[0054] (1) Adaptive weighted construction of unsupervised classification loss: For unlabeled data, based on the pseudo-labeled samples obtained after S2 candidate box selection and S3 dynamic screening, Focal Loss based on adaptive weights is used as the unsupervised classification loss to provide differentiated supervision for pseudo-labeled samples with different reliability; The classification loss for each pseudo-labeled sample is defined as:

[0055] (12)

[0056] in, Focal Loss, representing adaptive weights, is used as the unsupervised classification loss. Represents the student model for the first The predicted probability of the target category for each sample. This represents the focusing parameters of Focal Loss. Indicates the first Adaptive weights for each sample;

[0057] (2) The dynamic calculation of adaptive sample weights differs from the traditional method that directly uses absolute classification confidence as sample weights. The adaptive sample weights in this invention... The value is determined by the prediction result of the teacher model under the fused view and the dynamic positive sample threshold corresponding to the category to which the sample belongs, and is defined as follows:

[0058] (13)

[0059] in, This indicates that the teacher model is displayed in the fused view for the first... The classification score of each sample. This represents the dynamic positive sample threshold corresponding to the predicted class of the sample. Determined by the pseudo-label dynamic filtering module based on category distribution in step S3;

[0060] By outputting scores from the teacher model Relative to category threshold Normalization allows sample weights to no longer depend on a uniform absolute score standard, but rather on the degree to which the sample transcends the decision boundary of its class, thus more accurately reflecting the true reliability of pseudo-labels.

[0061] Preferably, the iterative training process of the teacher-student framework in S5 is defined as follows:

[0062] (1) Teacher-student collaborative training: This invention adopts a semi-supervised framework for collaborative training of the teacher model and the student model. During the training process, the input data is divided into two parts: labeled data and unlabeled data. The labeled data is directly used for supervised training, while the unlabeled data is first predicted by the teacher model to generate pseudo-labels, which are then used to guide the student model's learning.

[0063] Specifically, for labeled data, the student model directly uses real annotations to calculate supervised loss; for unlabeled data, the teacher model first combines the candidate box optimization gating fusion mechanism in step S2 and the pseudo-label dynamic filtering module based on category distribution in step S3 to generate high-quality pseudo-labels; then, the student model performs unsupervised learning based on the generated pseudo-labels and completes parameter optimization by combining the adaptive weight loss mechanism in step S4.

[0064] (2) Supervised training with labeled data: For labeled data, the student model is trained using the standard object detection loss function. The supervised loss includes classification loss and localization loss.

[0065] (14)

[0066] in, This indicates supervised loss on labeled data. This represents a supervised classification loss that uses binary cross-entropy loss to minimize the classification error between the predicted class and the true label. This indicates that GIoU has supervised localization loss;

[0067] (3) Generation and training of pseudo-labels for unlabeled data: For unlabeled data, the teacher model first outputs the detection results in the original view and the fused view, and calculates the double reset confidence gain before and after fusion through the gating feature fusion mechanism in step S2, and selects and recalls candidate boxes in the fuzzy area in the original view; then, through the category distribution dynamic filtering module in step S3, the category-level distribution constraint is applied to the candidate pseudo-labels, thereby generating the final high-quality pseudo-label set for training.

[0068] The student model calculates unsupervised loss based on the aforementioned pseudo-labels. This unsupervised loss includes unsupervised classification loss, unsupervised localization loss, and IoU consistency loss, expressed as follows:

[0069] (15)

[0070] in, This represents the unsupervised loss on unlabeled data. This indicates that the Focal Loss based on adaptive weights defined in step S4 is used as the unsupervised classification loss. This indicates the unsupervised localization loss of GIoU. The binary cross-entropy loss is used as the IoU consistency loss;

[0071] (4) Overall joint optimization: In each training iteration, the total loss of the student model consists of the supervised loss on labeled data and the unsupervised loss on unlabeled data, and its expression is:

[0072] (16)

[0073] in, This represents the total training loss of the student model. Indicates monitoring losses, This indicates unsupervised loss;

[0074] Through the above joint optimization method, the student model can simultaneously utilize the precise supervision information provided by a small number of real labeled samples and the potential structural information contained in a large number of unlabeled samples, thereby improving the model's adaptability to scenarios with uneven distribution of traffic sign categories and multi-scale target changes.

[0075] (5) Iterative update of teacher model parameters: During training, the teacher model parameters are not directly updated through backpropagation, but are iteratively updated by the student model parameters through exponential moving average (EMA), the expression of which is:

[0076] (17)

[0077] in, Indicates the first Teacher model parameters at the next iteration This represents the teacher model parameters at the previous iteration. This represents the student model parameters in the current iteration. This represents the coefficient of the exponential moving average, and its value is 0.9996.

[0078] Through the above update mechanism, the teacher model can maintain a smoother and more stable parameter evolution process than the student model, thereby generating more reliable pseudo-labels and avoiding the direct amplification and accumulation of early prediction noise in the student model.

[0079] Beneficial Effects: This invention addresses the problems of high pseudo-label noise, uneven category distribution, and easy omission of small targets in existing semi-supervised traffic sign detection. It improves the utilization rate of tail categories by constructing a dynamic pseudo-label screening module based on category distribution and adaptively adjusting the pseudo-label thresholds for each category. Furthermore, it enhances the detection capability of small-scale traffic signs by constructing a candidate box optimization gating fusion mechanism to optimize and recall fuzzy candidate boxes. Finally, it reduces the interference of noisy pseudo-labels on the training process by introducing an adaptive weight loss mechanism. Therefore, this invention achieves high-precision traffic sign detection with a small number of labeled samples, effectively reduces manual annotation costs, and improves the stability and reliability of detection results. Attached Figure Description

[0080] Figure 1 This is a flowchart of a traffic sign detection method based on gating fusion and dynamic screening of pseudo-labels.

[0081] Figure 2 This is a schematic diagram of different traffic sign examples.

[0082] Figure 3 This is a diagram of the overall structure of the semi-supervised model.

[0083] Figure 4 This is the architecture diagram of the gating feature fusion module.

[0084] Figure 5 This is a structural diagram of the pseudo-label dynamic filtering module based on category distribution.

[0085] Figure 6 Real label image

[0086] Figure 7 This is a screenshot of the detection results for the original model.

[0087] Figure 8 This is a diagram illustrating the detection effect of the traffic sign detection method proposed in this application. Detailed Implementation

[0088] The present invention will be further described in detail below with reference to specific embodiments.

[0089] Please see Figure 1 A traffic sign detection method based on gating fusion and dynamic pseudo-label filtering includes the following steps:

[0090] S1: Data Preparation and Preprocessing

[0091] First, a small number of labeled traffic sign images and a large number of unlabeled traffic sign images are acquired. These images are then cleaned by deleting blurry images with significant reflections, severe occlusion, or low resolution to avoid low-quality samples interfering with subsequent pseudo-label generation and detection model training. Subsequently, the LabelImg tool is used to refine the annotations on the labeled images. The annotation information includes at least the traffic sign category, bounding box position, and scale information. The traffic sign categories include various types such as prohibition, instruction, and warning, collectively forming a traffic sign category set, such as... Figure 2 As shown, the labeled dataset was then divided into training, validation, and test sets in an 8:1:1 ratio. This ensured that both the validation and test sets were fully labeled and used to independently evaluate model performance; unlabeled data only participated in the semi-supervised training process and was used in conjunction with the labeled training set. This data organization method is consistent with the description of data preparation and training settings in the paper.

[0092] In this embodiment, a data augmentation strategy combining weak and strong augmentation is employed during training. Weak augmentation includes at least mosaic enhancement; strong augmentation includes mosaic enhancement, multi-scale scaling, random color enhancement, grayscale conversion, Gaussian blurring, and multiple sets of cutout operations with different parameters. Since traffic signs have semantic differences in the left-right direction, horizontal flip augmentation is not used; that is, the horizontal flip probability is set to 0. The augmentation parameter configuration is shown in Table 1.

[0093] Table 1 Description of data augmentation

[0094]

[0095]

[0096] S2: Candidate box selection strategy combining gating feature fusion

[0097] Please see Figure 3 and Figure 4 This step constructs a gated fusion feature pyramid to jointly model the prediction results of the teacher model under the original view and the fused view. It also performs secondary screening and recall of fuzzy candidate boxes based on the double reset confidence gain before and after fusion, so as to improve the detection capability of small targets, distant targets and targets with drastic scale changes.

[0098] (1) Construction of Gated Fusion Feature Pyramid

[0099] The teacher model performs forward inference on both the original scale image and the downsampled scale image, and the backbone network extracts the original features and downsampled features of the corresponding layers. For the first... Each feature layer first concatenates the original features and downsampled features along the channel dimension, and then generates fusion weights through a gating function consisting of global average pooling and a multilayer perceptron. The expression for this fusion weights is:

[0100] (1)

[0101] in, Indicates the first Gated fusion weights of each feature layer This represents the Sigmoid activation function. This represents a multilayer perceptron. This indicates a global average pooling operation. Represents the original scale image at the 1st... Features of layer output, Indicates the downsampled scale image at the 1st... Features of the layer output.

[0102] Furthermore, the first... Fusion scale features of each feature layer:

[0103] (2)

[0104] in, Indicates the first The network fuses scale features across multiple feature layers. This approach enables the network to adaptively retain detailed information from the original scale features and semantic information from the downsampled scale features, based on the target scale features.

[0105] (2) Candidate box matching

[0106] Detection is performed based on the original features and the fused features respectively, resulting in a set of original predicted bounding boxes. With fusion prediction box set For candidate boxes in the original predicted box set that are located in the blurred region. Candidate box matching relationships are established by calculating the intersection-union ratio (IU) with each predicted box in the fused predicted box set.

[0107] (3)

[0108] in, Indicates the relationship with the original candidate box The best-matching fusion prediction box, This represents the intersection-union ratio (IURR) between candidate boxes. When the maximum IURR is greater than a preset matching threshold, the original candidate box and the fused predicted box are determined to correspond to the same target.

[0109] (3) Calculation of reliability before and after fusion

[0110] To simultaneously measure the classification reliability and localization reliability of candidate boxes, the joint confidence of the original candidate boxes and the fused candidate boxes is calculated separately:

[0111]

[0112] in, This represents the joint confidence level of the original candidate boxes. This represents the joint confidence level of the fused candidate boxes. and These represent the classification confidence of the candidate boxes under the original features and the fused features, respectively. and These represent the localization quality scores of the candidate boxes under the original features and the fused features, respectively.

[0113] (5) Candidate box recall based on confidence gain

[0114] We define the fusion confidence gain as:

[0115] (6)

[0116] in, This represents the joint confidence gain of the candidate box in the merged view relative to the original view. Candidate boxes Joint confidence level in the fusion pyramid Candidate boxes Joint confidence in the original view.

[0117] When the confidence gain corresponding to the fuzzy sample Greater than the preset threshold This indicates that the target's prediction was unstable in the original view due to scale differences or insufficient feature representation, but was effectively enhanced in the fused view. Therefore, candidate boxes that meet the above conditions are selected from the fuzzy candidate box set, and the higher-quality predicted boxes in the fused view are used as candidate boxes. Simultaneously, candidate boxes that meet the high-confidence condition in the fused feature pyramid prediction results are also directly retained. We merge the two types of candidate boxes into a preferred candidate box set as the final candidate box set used for subsequent pseudo-label dynamic selection and model training.

[0118] The above steps utilize the gain of joint confidence before and after fusion to perform secondary verification on candidate boxes that are in a blurred state in the original view, thereby recalling small targets that are easily missed at the original scale, improving the quality of false labels and the recall rate of traffic sign detection.

[0119] S3: Pseudo-label dynamic filtering module based on category distribution

[0120] Please see Figure 3 and Figure 5 The purpose of this step is to address the problem of severely imbalanced category distribution in traffic sign detection tasks. Unlike existing methods that use a uniform global threshold to filter pseudo-labels, this invention integrates statistical priors of label data with semantic priors of a visual-language model, dynamically adjusting the filtering thresholds for pseudo-labels of each category to make the generated pseudo-labels more consistent with the real data in terms of category distribution.

[0121] (1) Category semantic mapping and text embedding construction

[0122] Because traffic sign category labels are often engineering abbreviations and lack direct semantic expression, a space for abbreviation labels is constructed. To Natural Language Description Space mapping function And embed the mapped category description into the preset prompt template to generate the first Text embedding of classes :

[0123] (7)

[0124] in, Indicates the first Natural language semantic descriptions corresponding to traffic sign category codes. Indicates the first Text feature embedding for each class. This step enhances the semantic expressiveness of the text features for each class, providing a semantic prior for subsequent class distribution estimation.

[0125] (2) Semantic distribution estimation based on CLIP

[0126] The teacher model is used to perform preliminary detection on samples in the unlabeled dataset, and the predicted bounding boxes are cropped and input into the CLIP image encoder to extract the first bounding box. Visual features of each sample By calculating visual features Text embedding with various categories The similarity between them determines which sample belongs to the first group. Semantic prediction probability of class:

[0127] (8)

[0128] in, Represents the cosine similarity function. Indicates the temperature coefficient. Indicates the total number of categories. Indicates sample Belongs to the Semantic prediction probability of the class.

[0129] (3) Fusion of statistical prior and semantic prior

[0130] The first one obtained from the statistics of labeled data Class Statistical Prior Compared with the first estimate obtained by CLIP Class semantic prior We perform weighted fusion to obtain the calibrated prior distribution of categories. :

[0131] (9)

[0132] in, For distributed fusion weights, This indicates the first number obtained based on statistics from labeled data. Similar to empirical distribution, The first one obtained based on CLIP semantic reasoning Class semantic distribution, Indicates the fused first Prior distribution of class categories.

[0133] (4) Dynamic threshold calculation based on category distribution

[0134] For category Let the predicted score generated by the teacher model on the unlabeled set be... And sorted in descending order as ,in, This represents the total number of unlabeled candidate samples. Indicates category The The predicted scores are sorted in descending order.

[0135] To filter out low-reliability samples near the decision boundary, a positive sample confidence interval ratio is introduced. and the proportion of negative sample confidence intervals And based on the fused category prior Dynamically determine the first positive sample threshold of class and negative sample threshold :

[0136] (10)

[0137] in, Indicates the first The threshold for positive samples in pseudo-label filtering. Indicates the first The threshold for negative samples in pseudo-label filtering, and .

[0138] The above thresholds change dynamically with the category distribution: for the head category, because For larger values, the threshold position is shifted forward accordingly to suppress excessive false positives; for tail categories, because... The threshold is relatively small, and it automatically adapts to the lower confidence distribution, thereby improving the recall rate.

[0139] (5) Pseudo-label dynamic filtering rules

[0140] For unlabeled samples The teacher model outputs its category Predicted scores Then the pseudo-label of the sample Determined by the following rules:

[0141] (11)

[0142] in, Indicates the sample Determined as the first Positive samples, Indicates that it is determined to be the first Negative samples, This indicates that the sample is in the uncertainty interval and will not participate in the construction of the unsupervised supervision signal for this category in the current iteration.

[0143] S4: Adaptive Weight Loss Optimization Mechanism

[0144] This step addresses the unavoidable noise problem in pseudo-labeled samples by introducing Focal Loss based on adaptive weights. Different loss weights are assigned to pseudo-labeled samples with different reliability, thereby suppressing noise interference and improving training stability.

[0145] (1) Adaptive weighting of unsupervised classification loss

[0146] For unlabeled data, based on the pseudo-labeled samples obtained after S2 candidate box selection and S3 dynamic filtering, Focal Loss with adaptive weights is used as the unsupervised classification loss to provide differentiated supervision for pseudo-labeled samples with different reliability. The classification loss for each pseudo-labeled sample is defined as:

[0147] (12)

[0148] in, Focal Loss, representing adaptive weights, is used as the unsupervised classification loss. Represents the student model for the first The predicted probability of the target category for each sample. This represents the focusing parameters of Focal Loss. Indicates the first Adaptive weights for each sample.

[0149] (2) Adaptive weighting of unsupervised classification loss

[0150] Unlike traditional methods that directly use absolute classification confidence as sample weights, this invention uses adaptive sample weights. The value is determined by the prediction result of the teacher model under the fused view and the dynamic positive sample threshold corresponding to the category to which the sample belongs, and is defined as follows:

[0151] (13)

[0152] in, This indicates that the teacher model is displayed in the fused view for the first... The classification score of each sample. This represents the dynamic positive sample threshold corresponding to the predicted class of the sample. It is determined by the pseudo-label dynamic filtering module based on category distribution in step S3.

[0153] S5: Iterative Training and Detection Output within a Teacher-Student Framework

[0154] Please see Figure 3 This invention employs a semi-supervised framework that uses a teacher model and a student model for collaborative training. During training, the input data is divided into two parts: labeled data and unlabeled data. The labeled data is directly used to calculate the supervised loss, while the unlabeled data is used by the teacher model to generate high-quality pseudo-labels based on the candidate box selection strategy in step S2 and the dynamic filtering module in step S3, which are then used to guide the student model's learning.

[0155] (1) Supervised training with labeled data

[0156] For labeled data, the student model is trained under supervision using a standard object detection loss function, whereby the supervision loss includes classification loss and localization loss:

[0157] (14)

[0158] in, This indicates supervised loss on labeled data. This represents a supervised classification loss that uses binary cross-entropy loss to minimize the classification error between the predicted class and the true label. This indicates that GIoU has supervised localization loss.

[0159] (2) Generation and training of pseudo-labels for unlabeled data

[0160] For unlabeled data, the teacher model first outputs detection results in the original view and the fused view, and calculates the double reset confidence gain before and after fusion through the gating feature fusion mechanism in step S2, and selects and recalls candidate boxes in the fuzzy area in the original view; then, through the category distribution dynamic filtering module in step S3, category-level distribution constraints are applied to the candidate pseudo-labels, thereby generating a high-quality pseudo-label set for training.

[0161] The student model calculates unsupervised loss based on the aforementioned pseudo-labels. This unsupervised loss includes unsupervised classification loss, unsupervised localization loss, and IoU consistency loss, expressed as follows:

[0162] (15)

[0163] in, This represents the unsupervised loss on unlabeled data. This indicates that the Focal Loss based on adaptive weights defined in step S4 is used as the unsupervised classification loss. This indicates the unsupervised localization loss of GIoU. The binary cross-entropy loss is represented as the IoU consistency loss.

[0164] (3) Overall joint optimization

[0165] In each training iteration, the total loss of the student model consists of the supervised loss on labeled data and the unsupervised loss on unlabeled data, expressed as:

[0166] (16)

[0167] in, This represents the total training loss of the student model. Indicates monitoring losses, This indicates unsupervised loss.

[0168] Through the above joint optimization method, the student model can simultaneously utilize the precise supervision information provided by a small number of real labeled samples and the potential structural information contained in a large number of unlabeled samples, thereby improving the model's adaptability to scenarios with uneven distribution of traffic sign categories and multi-scale target changes.

[0169] (4) Iterative update of teacher model parameters

[0170] During training, the teacher model parameters are not directly updated via backpropagation, but are iteratively updated by the student model parameters using an exponential moving average (EMA) method, the expression of which is:

[0171] (17)

[0172] in, Indicates the first Teacher model parameters at the next iteration This represents the teacher model parameters at the previous iteration. This represents the student model parameters in the current iteration. This represents the coefficient of the exponential moving average, and its value is 0.9996.

[0173] Table 2 Comparison of Model Accuracy

[0174]

[0175] As shown in Table 2, in the traffic sign dataset, randomly selected data sets were used, with labeled data accounting for 1%, 2%, 5%, and 10% respectively, and the remainder being unlabeled data. With 1%, 2%, 5%, and 10% labeled data, the present invention achieved mAP (mean Average Precision) of 11.5%, 18.9%, 26.6%, and 36.3%, respectively. Therefore, this application can effectively reduce the manual cost of data annotation and improve the detection accuracy of the original model.

[0176] Please see Figure 6 , Figure 7 and Figure 8 , Figure 6 To detect the true label of the image, Figure 7 Original model detection results; Figure 8 The image shows the detection model performance of the semi-supervised object detection method proposed in this application. Visualization of the detection results reveals that the original baseline model suffers from detection errors, while the improved algorithm in this application effectively addresses these issues.

[0177] This application addresses the issues of uneven class distribution and missed detection of small targets in semi-supervised traffic sign detection by constructing a semi-supervised learning framework based on dynamic pseudo-label selection and candidate box optimization gating fusion. By setting up a pseudo-label dynamic selection module based on class distribution, and fusing statistical priors with the semantic priors of the CLIP visual-language model, the thresholds for pseudo-labels of each class are adaptively adjusted, improving the utilization rate of pseudo-labels in the tail category and alleviating the model's overfitting to the head category. Furthermore, by constructing a candidate box optimization strategy combining gating feature fusion, and optimizing and recalling fuzzy candidate boxes based on double-reset confidence gains before and after fusion, the detection capability for distant traffic signs is enhanced. Finally, by introducing an adaptive weight loss mechanism, the interference of noisy pseudo-labels on the training process is reduced, improving the stability of model training and the reliability of detection results. Therefore, this application can achieve high-accuracy traffic sign detection with a small number of labeled samples and can effectively reduce the cost of manual annotation, demonstrating good robustness and application value.

[0178] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A traffic sign detection method based on gating fusion and dynamic pseudo-label screening, characterized in that: Includes the following steps: S1: Data collection and preprocessing, acquiring a small number of labeled traffic sign images and a large number of unlabeled traffic sign images; S2: Construct a gated feature fusion mechanism to optimize and recall candidate boxes by fusing features at different scales and calculating the confidence gain before and after fusion; S3: Construct a pseudo-label dynamic filtering module based on category distribution, integrate statistical prior and semantic prior, dynamically adjust the pseudo-label filtering threshold of each category, and generate high-quality pseudo-labels; S4: Introduce an adaptive weight loss mechanism to dynamically weight pseudo-labels to reduce the impact of noise; S5: Iterative training is performed using a teacher-student framework to output traffic sign detection results.

2. The traffic sign detection method based on gating fusion and pseudo-label dynamic screening according to claim 1, characterized in that: In step S1, for the small number of labeled traffic sign images and the large number of unlabeled traffic sign images, blurry images with reflections or low resolution are deleted. Then, the Labelimg tool is used to refine the labeled images, and the annotation content includes traffic sign category, bounding box position and scale information. Finally, the labeled dataset is divided into training set, validation set and test set in a ratio of 8:1:1 to ensure that the validation set and test set are fully labeled and evaluated independently. The unlabeled data is only used in the semi-supervised training stage and is used in conjunction with the labeled training set to improve model performance.

3. The traffic sign detection method based on gating fusion and pseudo-label dynamic screening according to claim 2, characterized in that: S2 establishes a gated feature fusion mechanism, which optimizes and recalls candidate boxes by fusing features at different scales and calculating the confidence gain before and after fusion. Specifically, it includes the following: (1) Construct a gated fusion feature pyramid. The teacher model extracts features from the original scale image and the downsampled scale image respectively to obtain the original features and downsampled features of the corresponding level. For the Each feature layer first converts the original features... With downsampling features The data is concatenated along the channel dimension, and then a fusion weight is generated using a gating function. : （1） in, Indicates the first Gated fusion weights of each feature layer This represents the Sigmoid activation function. This represents a multilayer perceptron. This indicates a global average pooling operation. Represents the original scale image at the 1st... Features of layer output, Indicates the downsampled scale image at the 1st... Features of the layer output; (2) Feature fusion generation: Based on the gated fusion weights, features of different scales are adaptively weighted to obtain fused scale features. : （2） in, Indicates the first The network integrates scale features from multiple feature layers. This approach enables the network to adaptively retain detailed information from the original scale features and semantic information from the downsampled scale features, based on the target scale features. (3) Candidate box matching: prediction is performed based on the original features and the fused scale features to obtain the original set of prediction boxes. and fusion prediction box set For candidate boxes in the original predicted box set that are located in the fuzzy region Candidate box matching relationships are established by calculating the intersection-union ratio (IU) with each predicted box in the fused predicted box set. （3） in, Indicates the relationship with the original candidate box The best-matching fusion prediction box, This represents the cross-union ratio between candidate boxes; when the maximum cross-union ratio is greater than the preset matching threshold, it is determined that the original candidate box and the fused prediction box correspond to the same target. (4) Calculate the joint confidence score before and after fusion. Calculate the product of the classification confidence score and the localization quality confidence score for both the original candidate boxes and the fused candidate boxes to obtain the joint confidence score before and after fusion: ， in, This represents the joint confidence level of the original candidate boxes. This represents the joint confidence level of the fused candidate boxes. and These represent the classification confidence of the candidate boxes under the original features and the fused features, respectively. and These represent the localization quality scores of the candidate boxes under the original features and the fused features, respectively. (5) Candidate box generation, the fusion confidence gain is defined as: （6） in, This represents the joint confidence gain of the candidate box in the merged view relative to the original view. Candidate boxes Joint confidence level in the fusion pyramid Candidate boxes Joint confidence in the original view; When the confidence gain corresponding to the fuzzy sample Greater than the preset threshold This indicates that the target's prediction is unstable in the original view due to scale differences or insufficient feature representation, but is effectively enhanced in the fused view. Therefore, candidate boxes that meet the above conditions are selected from the fuzzy candidate box set, and the higher-quality predicted boxes in the fused view are used as candidate boxes. At the same time, candidate boxes that meet the high confidence condition in the fused feature pyramid prediction results are also directly retained. The two types of candidate boxes are merged into the preferred candidate boxes as the final candidate box set used for subsequent pseudo-label dynamic screening and model training. The above steps utilize the gain of joint confidence before and after fusion to perform secondary verification on candidate boxes that are in a blurred state in the original view, thereby recalling small targets that are easily missed at the original scale, improving the quality of false labels and the recall rate of traffic sign detection.

4. The traffic sign detection method based on gating fusion and pseudo-label dynamic screening according to claim 3, characterized in that: The pseudo-label dynamic filtering module based on category distribution in S3 is defined as follows: (1) Category semantic mapping and text embedding construction: In view of the problem that traffic sign category labels are usually engineering abbreviations and lack direct semantic expression, a classification label space is constructed. To Natural Language Description Space mapping function And embed the mapped category description into the preset prompt template to generate the first Text embedding of classes : （7） in, Indicates the first Natural language semantic descriptions corresponding to traffic sign category codes. Indicates the first Text feature embedding of categories; this step is used to enhance the semantic expressive power of category text features and provide semantic priors for subsequent category distribution estimation; (2) Based on CLIP semantic distribution estimation, the teacher model is used to perform preliminary detection on samples in the unlabeled dataset, and the predicted bounding boxes are cropped and input into the CLIP image encoder to extract the first bounding box. Visual features of each sample ; By calculating visual features Text embedding with various categories The similarity between them determines which sample belongs to the first group. Semantic prediction probability of class: （8） in, Represents the cosine similarity function. Indicates the temperature coefficient. Indicates the total number of categories. Indicates sample Belongs to the Semantic prediction probability of the class; Statistical aggregation of the semantic prediction probabilities of the unlabeled sample subset is performed to obtain the first... Prior semantic distribution of classes This semantic distribution prior is used to supplement the empirical distribution information obtained only from labeled data statistics, so as to improve the robustness of long-tail category distribution estimation. (3) Fusion of statistical prior and semantic prior, combining the first prior obtained from the statistical analysis of labeled data. Class Statistical Prior Compared with the first estimate obtained by CLIP Class semantic prior We perform weighted fusion to obtain the calibrated prior distribution of categories. : （9） in, For distributed fusion weights, This indicates the number of statistics obtained based on labeled data. Similar to empirical distribution, This indicates the first result obtained based on CLIP semantic reasoning. Class semantic distribution, Indicates the fused first Prior distribution of class categories; Through the above fusion mechanism, the supervisory accuracy of the labeled data is preserved, and open-world semantic knowledge is introduced to perform zero-sample calibration for long-tail categories. (4) Dynamic threshold calculation based on category distribution, for category Let the predicted score generated by the teacher model on the unlabeled set be... And sorted in descending order as ,in, This represents the total number of unlabeled candidate samples. Indicate category The The predicted scores are sorted in descending order. To filter out low-reliability samples near the decision boundary, a positive sample confidence interval ratio is introduced. and the proportion of negative sample confidence intervals And based on the fused category prior Dynamically determine the first positive sample threshold of class and negative sample threshold : （10） in, Indicates the first The threshold for positive samples in pseudo-label filtering. Indicates the first The threshold for negative samples in pseudo-label filtering, and ; The above thresholds change dynamically with the category distribution: for the head category, because For larger values, the threshold position is shifted forward accordingly to suppress excessive false positives; for tail categories, because... The threshold is relatively small, and it automatically adapts to the lower confidence distribution, thereby improving the recall rate; (5) Pseudo-label dynamic screening rules for unlabeled samples The teacher model outputs its category Predicted scores Then the pseudo-label of the sample Determined by the following rules: （11） in, Indicates the sample Determined as the first Positive samples, Indicates that it is determined to be the first Negative samples, This indicates that the sample is in the uncertainty interval and will not participate in the construction of the unsupervised supervision signal for this category in the current iteration; By using the above filtering rules, the number of pseudo-labels in each category is kept consistent with the distribution of the real categories, thus avoiding overfitting of the head category by a uniform global threshold and systematic suppression of the tail category.

5. The traffic sign detection method based on gating fusion and pseudo-label dynamic screening according to claim 4, characterized in that: The adaptive weight Focal Loss in S4 is defined as follows: (1) Adaptive weighted construction of unsupervised classification loss: For unlabeled data, based on the pseudo-labeled samples obtained after S2 candidate box selection and S3 dynamic screening, Focal Loss based on adaptive weights is used as the unsupervised classification loss to provide differentiated supervision for pseudo-labeled samples with different reliability; The classification loss for each pseudo-labeled sample is defined as: （12） in, Focal Loss, representing adaptive weights, is used as the unsupervised classification loss. Represents the student model for the first The predicted probability of the target category for each sample. This represents the focusing parameters of Focal Loss. Indicates the first Adaptive weights for each sample; (2) Dynamic calculation of sample adaptive weights, sample adaptive weights The value is determined by the prediction result of the teacher model under the fused view and the dynamic positive sample threshold corresponding to the category to which the sample belongs, and is defined as follows: （13） in, This indicates that the teacher model is displayed in the fused view for the first... The classification score of each sample. This represents the dynamic positive sample threshold corresponding to the predicted class of the sample. Determined by the pseudo-label dynamic filtering module based on category distribution in step S3; By outputting scores from the teacher model Relative to category threshold Normalization is performed so that sample weights no longer depend on a uniform absolute score standard, but rather on the degree to which the sample transcends the decision boundary of its class, thus more accurately reflecting the true reliability of pseudo-labels.

6. The traffic sign detection method based on gating fusion and pseudo-label dynamic screening according to claim 5, characterized in that: The iterative training process of the teacher-student framework in S5 is defined as follows: (1) Teacher-student collaborative training: This invention adopts a semi-supervised framework for collaborative training of teacher model and student model; during the training process, the input data is divided into two parts: labeled data and unlabeled data; among them, the labeled data is directly used for supervised training, while the unlabeled data is first predicted by the teacher model to generate pseudo-labels, and then used to guide the student model to learn. Specifically, for labeled data, the student model directly uses real annotations to calculate supervised loss; for unlabeled data, the teacher model first combines the candidate box optimization gating fusion mechanism in step S2 and the pseudo-label dynamic filtering module based on category distribution in step S3 to generate high-quality pseudo-labels; then, the student model performs unsupervised learning based on the generated pseudo-labels and completes parameter optimization by combining the adaptive weight loss mechanism in step S4. (2) Supervised training with labeled data: For labeled data, the student model is trained using the standard object detection loss function. The supervised loss includes classification loss and localization loss. （14） in, This indicates supervised loss on labeled data. This represents a supervised classification loss that uses binary cross-entropy loss to minimize the classification error between the predicted class and the true label. This indicates that GIoU has supervised localization loss; (3) Generation and training of pseudo-labels for unlabeled data: For unlabeled data, the teacher model first outputs the detection results in the original view and the fused view, and calculates the double reset confidence gain before and after fusion through the gating feature fusion mechanism in step S2, and selects and recalls candidate boxes in the fuzzy area in the original view; then, through the category distribution dynamic filtering module in step S3, the category-level distribution constraint is applied to the candidate pseudo-labels, thereby generating the final high-quality pseudo-label set for training. The student model calculates unsupervised loss based on the aforementioned pseudo-labels. This unsupervised loss includes unsupervised classification loss, unsupervised localization loss, and IoU consistency loss, expressed as follows: （15） in, This represents the unsupervised loss on unlabeled data. This indicates that the Focal Loss based on adaptive weights defined in step S4 is used as the unsupervised classification loss. This indicates the unsupervised localization loss of GIoU. The binary cross-entropy loss is used as the IoU consistency loss; (4) Overall joint optimization: In each training iteration, the total loss of the student model consists of the supervised loss on labeled data and the unsupervised loss on unlabeled data, and its expression is: （16） in, This represents the total training loss of the student model. Indicates monitoring losses, This indicates unsupervised loss; Through the above joint optimization method, the student model can simultaneously utilize the precise supervision information provided by a small number of real labeled samples and the potential structural information contained in a large number of unlabeled samples, thereby improving the model's adaptability to scenarios with uneven distribution of traffic sign categories and multi-scale target changes. (5) Iterative update of teacher model parameters: During training, the teacher model parameters are not directly updated through backpropagation, but are iteratively updated by the student model parameters through an exponential moving average. The expression is as follows: （17） in, Indicates the first Teacher model parameters at the next iteration This represents the teacher model parameters at the previous iteration. This represents the student model parameters in the current iteration. This represents the coefficient of the exponential moving average, and its value is 0.9996. Through the above update mechanism, the teacher model can maintain a smoother and more stable parameter evolution process than the student model, thereby generating more reliable pseudo-labels and avoiding the direct amplification and accumulation of early prediction noise in the student model.

Images