A hierarchical decoupling hybrid expert model traffic signal light recognition method

Abstract

The application discloses a traffic signal lamp recognition method of a hierarchical decoupling mixed expert model, and comprises the following steps: step one, constructing a hierarchical decoupling mixed expert model, comprising a lightweight weather perception gating network and a heterogeneous expert module; the heterogeneous expert module is a YOLOv8 detector based on multiple different weathers and independently trained, corresponding to multiple different weather scenes and multiple different weather experts; step two, respectively performing expert network independent training and gating network training; step three, collecting a road scene image through a vehicle-mounted camera, performing weather feature coding on the input image through the lightweight weather perception gating network, judging the weather feature of the current image, and selecting an optimal expert path according to the weather feature of the input image based on a Top-1 hard routing inference strategy; only a single corresponding weather expert is activated to perform detection, and finally, a traffic signal category and a bounding box are output. The traffic signal lamp can be effectively recognized in a multi-weather scene.

Description

Technical Field

[0001] This invention relates to a traffic signal recognition method based on a hierarchical decoupled hybrid expert model, belonging to the field of traffic signal recognition technology. Background Technology

[0002] Traffic light detection aims to accurately locate and identify the position and status of traffic lights from images captured by vehicle-mounted cameras. Existing technologies integrate convolutional neural networks, multi-scale feature pyramids, and attention mechanisms, driving the framework towards end-to-end real-time detection. Previous research has attempted to mitigate the impact of weather degradation on target detection performance from different perspectives. Classic end-to-end target detection methods generally follow a hierarchical "Backbone-Neck-Head" architecture, achieving target localization and classification through multi-scale feature extraction, cross-layer fusion, and attention mechanisms. However, under severe weather conditions such as fog, rain, and snow, severe image quality degradation causes domain shifts in target feature distribution, severely limiting the cross-meteorological domain generalization ability of detection models. To alleviate these problems, existing research has explored data augmentation strategies on the training side: using synthetic degradation such as fogging, rain, and snow to expand the domain coverage of training samples, enabling a single detector to implicitly learn invariant representations across multiple meteorological domains. This strategy maintains the inference architecture unchanged, but is limited by parameter capacity; feature adaptation across different meteorological domains can easily lead to performance coupling, and the domain gap between synthetic and real-world scenes restricts the upper bound of generalization. Building upon this, researchers further proposed a cascaded detection paradigm, decoupling the task into a sequential pipeline of "degradation restoration – target detection." This scheme can compensate for the information loss caused by meteorological degradation, but the cascaded restoration modules significantly increase computational overhead and inference latency, limiting the system's real-time performance and deployment feasibility. Although the above methods attempt to address meteorological degradation from different perspectives, they all face the fundamental contradiction of balancing "multi-domain adaptability" and "inference efficiency."

[0003] The YOLO series of methods introduced a single-stage detection paradigm, achieving an effective balance between detection accuracy and inference speed. These methods perform well on publicly available datasets under standard weather conditions, but their performance is prone to degradation in domain-shifted scenarios. Existing technologies have proposed traffic light detection models based on YOLOv5 to improve detection performance, further enhancing accuracy through ensemble learning and color constraints. Additionally, some studies have improved small object detection capabilities by refining the YOLO detection structure or introducing feature fusion strategies. With the increasing complexity of autonomous driving applications, researchers have also begun to focus on traffic object detection in complex environments, such as detection frameworks based on YOLOv8 and the Meta-YOLOv8 model incorporating meta-learning strategies. However, most of these methods are optimized for specific environments, and their detection performance still significantly degrades under various weather conditions.

[0004] Mixture of Experts (MoE) models deploy multiple expert networks to learn domain-specific features and dynamically select a small number of experts to participate in computation during inference using a gating mechanism, providing a new perspective on balancing multi-meteorological domain adaptation and real-time inference efficiency. Switch Transformer employs a Top-1 sparse routing strategy to achieve efficient expert selection and activation. Vision MoE improves image classification and detection performance by introducing MoE into the Visual Transformer. Adaptive Mixture of Experts models further optimize multi-task visual recognition performance by dynamically adjusting the number of expert activations. Most methods map subnetworks with the same structure to a unified feature space through isomorphic expert design, resulting in limited adaptability to input diversity. To alleviate this burden, CBDES-MoE uses a heterogeneous expert pool and a self-attention router to achieve dynamic expert selection and achieves detection performance exceeding the baseline of a single backbone network on the nuScenes dataset. Existing technologies propose a traffic light detection method based on frequency domain adaptation to alleviate the domain shift problem under adverse weather conditions, and a traffic light detection method based on logarithmic polar coordinate transformation is proposed for nighttime driving environments to improve detection stability in nighttime scenes. Building upon this foundation, researchers have begun to introduce hybrid expert models into object detection tasks. For example, researchers proposed a MoE-based 3D object detection system that improves detection efficiency in autonomous driving scenarios through an expert allocation mechanism. Furthermore, they combined the MoE structure with the YOLO detection framework, enhancing the model's robustness in complex scenarios through an adaptive expert routing mechanism. Existing technologies enable vision-based MoE to handle multi-view image inputs, exhibiting strong scene adaptability. Hybrid expert models are used to obtain multimodal trajectory prediction results, which are then refined by a planning module. The performance of the object detection model is further improved by introducing an expert calibration mechanism. These methods only achieve performance evaluation under single weather conditions. However, multi-weather traffic light detection places more stringent demands on the MoE architecture: the feature distributions of different meteorological domains are inherently heterogeneous. If a homogeneous expert design is adopted or too many parameters are shared at the backbone network level, it is difficult for each expert to fully learn domain-specific representations. Simultaneously, if the decoupling between the routing mechanism and the expert network is insufficient, feature interference will occur between experts, ultimately failing to stably balance detection accuracy and inference efficiency. Summary of the Invention

[0005] This invention designs and develops a traffic light recognition method based on a hierarchical decoupled hybrid expert model. It decouples and hierarchically processes weather domain adaptation and feature extraction at the architecture level, enabling effective recognition of traffic lights in multiple weather scenarios.

[0006] The technical solution provided by this invention is as follows:

[0007] A traffic light recognition method based on a hierarchical decoupled hybrid expert model includes:

[0008] Step 1: Construct a hierarchical, decoupled hybrid expert model, including: a lightweight weather perception gating network and heterogeneous expert modules;

[0009] The heterogeneous expert module consists of YOLOv8 detectors trained independently based on multiple different weather conditions, corresponding to multiple different weather scenarios and multiple different weather experts;

[0010] Step 2: Conduct independent training of the expert network and gated network separately;

[0011] Step 3: The road scene images captured by the vehicle-mounted camera are encoded with weather features by a lightweight weather-aware gating network. The weather features of the current image are determined. Based on the Top-1 hard routing inference strategy, the gating network selects the optimal expert path according to the weather features of the input image, activates only the corresponding single weather expert to perform detection, and outputs the final traffic signal category and bounding box.

[0012] Preferably,

[0013] The various weather conditions include: sunny, foggy, rainy, and snowy.

[0014] Set up four expert network ensembles with identical structures but independent parameters:

[0015] ;

[0016] In the formula: These represent expert models trained for sunny, foggy, rainy, and snowy weather conditions, respectively.

[0017] Based on dedicated training datasets for various weather conditions, the detection performance of each expert in its target domain is maximized through a standard YOLOv8 optimization process. The optimal expert model is:

[0018] ;

[0019] In the formula: Indicates the first The optimal model for each expert; This represents the learnable parameters of the corresponding expert network. Indicates the first Training dataset for various weather conditions, This is the standard detection loss function for YOLOv8, which includes classification loss, bounding box regression loss, and target confidence loss.

[0020] Preferably, the weather sensing gating network includes:

[0021] A lightweight feature extractor consisting of three cascaded convolutional blocks is constructed.

[0022] After three convolutional blocks, global adaptive average pooling is used to compress the spatial dimension feature information into a fixed-length feature vector.

[0023] Based on the extracted global feature vectors, they are mapped to the expert dimension space through a fully connected layer. The activation probability of each expert network is calculated using the Softmax function, and the calculation formula is as follows:

[0024] ;

[0025] In the formula, ℝ represents the set of real numbers. This represents the linear output vector of the fully connected layer, containing the unnormalized score of each expert. Indicates the first The unnormalized scores corresponding to each expert network This indicates the first step in the Softmax normalization summation process. The unnormalized scores corresponding to each expert network This represents the weight matrix of the fully connected layer. 3D feature vector mapping to Dimensional Expert Space This represents the corresponding bias vector. This indicates the number of expert networks, corresponding to four weather conditions: sunny, foggy, rainy, and snowy. , indicating the first Gating probabilities of an expert network.

[0026] Preferably, the calculation formula for the Top-1 expert selection strategy is as follows:

[0027] ;

[0028] In the formula, This indicates the expert index selected by the Top-1 selection, corresponding to the expert number with the highest gating probability. The operator that returns the index corresponding to the maximum value. Indicates the selected number The forward propagation function of an expert network, with the original image as input. , This represents the final traffic light target detection output, which includes the predicted category label and bounding box coordinates.

[0029] Preferably, the gating network training employs a multi-class cross-entropy loss function for supervised learning, the calculation formula of which is:

[0030] ;

[0031] In the formula, B represents the training loss value of the gating network, and B represents the training batch size. Indicates the first The unique thermal encoding label vector for the weather category of each sample, when the sample belongs to the... When weather conditions Otherwise, it is 0. Indicates that the gating network is for the first The predicted number of samples The probability value of weather-like conditions.

[0032] Preferably, the hierarchical decoupled hybrid expert model employs a phased training strategy, including:

[0033] During the independent training phase of the expert networks, the four expert networks were trained independently on their respective subsets of weather data.

[0034] Given a set of weather categories W = {sunny, fog, rain, snow}, the first... The training process for an expert network is as follows:

[0035] ;

[0036] In the formula, Indicates the first The expert network in the first Model parameters at the next iteration Indicates the learning rate. Indicates the parameter Operator for finding gradient Indicates the first The detection loss function of an expert network;

[0037] During the training phase of the gated network, the gated network uses weather category labels as supervision signals to learn the mapping relationship from the input image to the weather category. Let the parameters of the gated network be... The training process is as follows:

[0038] ;

[0039] in, Indicates that the gating network is in the first stage. The parameters at the next iteration This represents the learning rate of the gating network. The cross-entropy loss function of a gated network;

[0040] In the system integration and inference phase, the gating network and four expert networks are assembled into a complete inference system, given a test image x. testThe system reasoning process is as follows:

[0041] ;

[0042] ;

[0043] in, Denotes the forward propagation function of the gated network. The output of the gating network represents the first... The activation probability of an expert This indicates the expert index selected by the Top-1 team. Indicates using optimal parameters The A network of experts, This indicates the final target detection result.

[0044] The beneficial effects of this invention are as follows:

[0045] The traffic light recognition method based on a hierarchical decoupled hybrid expert model provided by this invention uses a heterogeneous expert network that is independently optimized for different weather conditions and introduces a lightweight weather perception gating mechanism and a Top-1 sparse activation strategy. During inference, the gating network selects the optimal expert path based on the weather features of the input image and activates only the corresponding single optimal weather expert to perform detection. This effectively realizes an image-level dynamic expert routing mechanism for weather perception, bridges the feature space differences between different weather domains, and alleviates the performance degradation problem caused by weather changes.

[0046] The hierarchical decoupled hybrid expert model can reliably achieve adaptive matching between the weather domain and the expert network, and achieve an effective balance between detection accuracy and computational efficiency in multiple weather scenarios.

[0047] This method possesses strong environmental adaptability. For input images under different weather conditions, the gating network can accurately identify the weather type of the current scene and provide a high confidence level, effectively addressing the interference caused by variable weather in open traffic scenarios. Simultaneously, this method achieves precise routing and feature decoupling of the expert model. After identifying the weather type, the gating network can accurately assign the input image to the corresponding weather expert network for detection, allowing traffic light features under different weather conditions to be extracted independently. This avoids the problem of feature interference between single models in mixed weather data, significantly improving the model's robustness in complex weather conditions. Furthermore, efficient inference is achieved based on the Top-1 sparse gating mechanism. During the inference phase, only one optimal expert network is activated for each input image frame, improving detection accuracy without increasing computational load. This makes it highly suitable for deployment on vehicle-mounted devices with limited computing power. Attached Figure Description

[0048] Figure 1 A comparative diagram of traffic light detection paradigms under different weather conditions.

[0049] Figure 1 (a) is a schematic diagram of a classic end-to-end target detector.

[0050] Figure 1 (b) is a schematic diagram of single-model training based on synthetic weather data augmentation.

[0051] Figure 1 (c) is a schematic diagram of the cascaded weather recovery / preprocessing and then detection method.

[0052] Figure 2 This is a schematic diagram of a multi-weather adaptive traffic light detection architecture based on a hierarchical decoupled hybrid expert model.

[0053] Figure 3(a) shows the dynamic routing results of the hierarchical decoupled hybrid expert model in a sunny day scenario.

[0054] Figure 3(b) shows the dynamic routing results of the hierarchical decoupled hybrid expert model in foggy weather.

[0055] Figure 3(c) shows the dynamic routing results of the hierarchical decoupled hybrid expert model in the rainy weather scenario.

[0056] Figure 3(d) shows the dynamic routing results of the hierarchical decoupled hybrid expert model in the snow scene. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0058] like Figure 1 As shown in Figure 3, this invention provides a traffic light recognition method based on a hierarchical decoupled hybrid expert model. Weather domain adaptation and feature extraction are decoupled and layered at the architectural level, enabling effective recognition of traffic lights in multiple weather scenarios. The method includes:

[0059] Step 1: Construct a hierarchical, decoupled hybrid expert model, including: a lightweight weather perception gating network and heterogeneous expert modules;

[0060] The heterogeneous expert module consists of YOLOv8 detectors trained independently based on multiple different weather conditions, corresponding to multiple different weather scenarios and multiple different weather experts;

[0061] Step 2: Conduct independent training of the expert network and gated network separately;

[0062] Step 3: The road scene images captured by the vehicle-mounted camera are encoded with weather features by a lightweight weather-aware gating network. The weather features of the current image are determined. Based on the Top-1 hard routing inference strategy, the gating network selects the optimal expert path according to the weather features of the input image, activates only the corresponding single weather expert to perform detection, and outputs the final traffic signal category and bounding box.

[0063] In this invention, as a preferred embodiment, the various weather conditions include: sunny days, foggy days, rainy days, and snowy days;

[0064] Establish a multi-weather adaptive traffic light detection architecture CBDES-MoE-TLD based on a hierarchical decoupled hybrid expert model, such as... Figure 2 As shown, road scene images are acquired through an onboard camera, and weather features are encoded into the input images using a lightweight weather-aware gating network. A heterogeneous expert module captures weather-specific visual patterns using four independently trained YOLOv8 detectors for different weather conditions, extracts detection features of different attributes, and outputs candidate detection results from each expert. The weather-aware gating network combines the weather feature encoding results with expert routing probabilities, performing Top-1 sparse expert selection to generate the optimal expert index. During the training and inference phases, a phased training and Top-1 hard routing inference strategy is used. During inference, the gating network selects the optimal expert path based on the weather features of the input image, activating only the corresponding single weather expert to perform detection, thereby outputting the final traffic light category and bounding box. The CBDES-MoE-TLD model effectively achieves adaptive matching between the weather domain and the expert network, achieving an effective balance between detection accuracy and computational efficiency across multiple weather scenarios, specifically including:

[0065] Establish a heterogeneous weather expert module

[0066] In this invention, as a preferred embodiment, a weather synthesis strategy based on image degradation is introduced during the expert training phase to process the original sunny day image into rain, fog, and snow, thereby constructing a representative weather-specific training dataset.

[0067] The rain-transformation process introduces rainfall noise by simulating rain streaks and brightness attenuation; the fog-transformation process degrades image contrast and sharpness based on an atmospheric scattering model; and the snow-transformation process simulates snowfall scenes through random snow particle superposition and local occlusion. This weather synthesis is used only in the offline training phase of the expert network to enhance the experts' perception of target weather degradation patterns and does not introduce additional inference overhead.

[0068] To achieve adaptive traffic light detection under various weather scenarios, this invention, as a preferred approach, introduces a heterogeneous weather expert module; following the YOLOv8 single-stage detection architecture, four expert network sets with identical structures but independent parameters are set up, and the calculation formula is as follows:

[0069] ;

[0070] In the formula: These represent expert models trained for sunny, foggy, rainy, and snowy weather conditions, respectively.

[0071] Each expert acquires weather-specific visual feature representations through end-to-end supervised learning on corresponding weather datasets. During the inference phase, they collaborate with the routing decisions of the gating network. An independent training strategy effectively avoids feature interference and performance constraints between different weather domains. Using dedicated training datasets for each weather condition, a standard YOLOv8 optimization process maximizes the detection performance of each expert within their target domain. The optimal model calculation formula is as follows:

[0072] ;

[0073] In the formula: Indicates the first The optimal model for each expert; This represents the learnable parameters of the corresponding expert network. Indicates the first Training dataset for various weather conditions This is the standard detection loss function for YOLOv8, which includes classification loss, bounding box regression loss, and target confidence loss. Represents the training dataset D i Input image samples, Represents the input image The corresponding real tags, The parameter is The An expert network analyzes the input image. The predicted output.

[0074] After training, the four expert models are integrated into a unified MOE inference framework following a parameter freezing strategy. The specialized detection capabilities are encapsulated into independent inference units that can be dynamically scheduled by the gating network using the optimal weight files of each expert.

[0075] Establish a weather-sensing gating network

[0076] In this invention, the weather-aware gating network is responsible for automatically determining the current weather conditions based on the visual features of the input image and selecting the most suitable expert network for subsequent detection tasks, thereby achieving a dynamic balance between detection accuracy and computational efficiency in multiple weather scenarios.

[0077] like Figure 2 As shown, the input image The deep features are fed into a lightweight feature extractor for hierarchical feature encoding. The extracted deep features are aggregated into a fixed-dimensional global weather feature vector through global adaptive average pooling and then input into a fully connected classifier. Leveraging the powerful local texture feature extraction capabilities of convolutional neural networks and the excellent training stability of batch normalization layers, the gating network performs a multi-classification task for weather conditions, including accurate discrimination of four typical traffic scenarios: sunny, foggy, rainy, and snowy. In this invention, as a preferred approach, a Top-1 hard-select sparse activation strategy is chosen, dynamically routing the single expert network with the highest gating probability to the object detection module, achieving efficient utilization of computational resources.

[0078] The weather perception gating network includes: establishing a lightweight feature extractor, performing global feature aggregation, a Top-1 expert selection strategy, training the gating network, and adopting a learning rate scheduling strategy.

[0079] To reduce computational overhead while maintaining classification accuracy, a lightweight feature extractor consisting of three cascaded convolutional blocks is established, given an input image. , No. The feature transformation process for each convolutional block is defined as follows:

[0080] ;

[0081] In the formula: Indicates the first Output feature maps of convolutional blocks , , They represent the first The height, width, and number of channels of the layer feature map Indicates the first 3×3 convolution operation of the layer, This indicates a batch normalization operation. Represents the linear rectified activation function. This represents a 2×2 max pooling operation.

[0082] The specific operations of convolution are as follows:

[0083] ;

[0084] In the formula, Indicates the first The weight parameters of the convolutional kernel, Indicates the corresponding bias parameter, subscript These represent the spatial location index (height, width) and channel index of the feature map, respectively. express Spatial offset of the convolution kernel This indicates the traversal index of the input channel.

[0085] After three convolutional layers, Global Adaptive Average Pooling (GAP) is used to compress the spatial feature information into a fixed-length feature vector. The formula for calculating the aggregated global weather feature vector is as follows:

[0086] ;

[0087] In the formula, This indicates a global adaptive average pooling operation. and This represents the spatial dimensions (height and width) of the feature map output by the third convolutional block. Representing spatial location in the feature map Feature vectors of all channels This represents the aggregated global weather feature vector.

[0088] Based on the extracted global feature vectors, they are mapped to the expert dimension space through a fully connected layer, and the activation probability of each expert network is calculated using the Softmax function, as follows:

[0089] ;

[0090] In the formula, ℝ represents the set of real numbers. This represents the linear output vector of the fully connected layer, containing the unnormalized score of each expert, o i Represents the first element in the linear output vector o. The component, i.e., the first component The unnormalized scores corresponding to each expert network; j Represents the first element in the linear output vector o. The nth component, i.e., the nth component in the Softmax normalization summation process. The unnormalized scores corresponding to each expert network; This represents the weight matrix of the fully connected layer, mapping the C3-dimensional feature vectors to... Dimensional Expert Space This represents the corresponding bias vector. This indicates the number of expert networks, corresponding to four weather conditions: sunny, foggy, rainy, and snowy. , indicating the first Gating probabilities of an expert network.

[0091] During the inference phase, the gating network employs a Top-1 hard selection strategy, activating only the single expert network with the highest gating probability to perform the object detection task, as shown in the following equation:

[0092] ;

[0093] In the formula, This indicates the expert index selected by the Top-1 selection, corresponding to the expert number with the highest gating probability. The operator that returns the index corresponding to the maximum value. Indicates the selected number The forward propagation function of an expert network, with the original image as input. , This represents the final traffic light target detection output, which includes the predicted category label and bounding box coordinates.

[0094] The training objective of the gating network is to accurately predict the weather category of the input image, thereby providing a reliable basis for expert selection. In this invention, as a preferred method, a multi-class cross-entropy loss function is used for supervised learning. The loss value calculation formula is shown below:

[0095] ;

[0096] In the formula, This represents the training loss value of the gating network. This represents the training batch size, which is the number of samples processed simultaneously in each iteration. Indicates the first The weather category one-hot encoded label vector of the sample, when the sample belongs to the... When weather conditions Otherwise, it is 0; Indicates that the gating network is for the first The predicted number of samples The probability value of weather-like conditions.

[0097] The cross-entropy loss function drives the gating network to learn weather discrimination features by maximizing the predicted probability of the true class, ensuring that the expert selection output is highly consistent with the actual weather conditions.

[0098] To improve training convergence performance and avoid getting trapped in local optima, this invention, as a preferred approach, employs a cosine annealing learning rate scheduler to dynamically adjust the optimizer's learning rate, as shown in the following equation:

[0099] ;

[0100] In the formula, Indicates the first The learning rate for each training epoch. This represents the initial maximum learning rate, set to 1×10. -3 , This represents the minimum learning rate, typically set to... To maintain a meager parameter update capability, 1%. This represents the total number of training rounds, and t∈[0,T] represents the index of the current training round. This represents the cosine function.

[0101] Training and reasoning strategies

[0102] The CBDES-MoE-TLD architecture established in this invention adopts a phased training strategy, sequentially completing expert network training, gating network training, and system integration, ultimately achieving end-to-end inference deployment. This phased training approach ensures that each module fully learns its corresponding task, avoiding gradient interference issues during joint training.

[0103] Independent training of expert networks is conducted. In this stage, the four expert networks are trained independently on their respective weather data subsets, allowing each expert to fully learn the target feature representation under specific weather conditions. Given a set of weather categories W={sunny, fog, rain, snow}, the... The training process for an expert network is as follows:

[0104] ;

[0105] In the formula, Indicates the first The expert network in the first Model parameters at the next iteration Indicates the learning rate. Indicates the parameter Operator for finding gradient Indicates the first The detection loss function of an expert network.

[0106] The detection loss of the expert network consists of three parts: classification loss, bounding box regression loss, and target confidence loss. The formula for calculating the detection loss function of an expert network is as follows:

[0107] ;

[0108] In the formula, Represents classification loss. This represents the bounding box regression loss. Indicates the target confidence loss. , , These represent the weighting coefficients of the corresponding loss terms.

[0109] The classification loss is calculated using the binary cross-entropy loss function, and the formula is as follows:

[0110] ;

[0111] In the formula, Indicates the number of positive samples. Indicates the total number of target categories. Indicates the first Does the nth positive sample belong to the nth positive sample? The true label of the class, The model predicts the first The sample belongs to the first The probability of a class.

[0112] Taking into account overlapping area, center point distance, and aspect ratio consistency, the bounding box regression loss adopts the CIoU loss function, the calculation formula of which is:

[0113] ;

[0114] In the formula, Indicates the first The intersection-union ratio of each predicted bounding box to the ground truth bounding box. Represents the Euclidean distance function. and These represent the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. This represents the diagonal length of the smallest bounding rectangle that can enclose both the predicted and ground truth boxes. The weighting coefficient representing the consistency of aspect ratio. This indicates a penalty for aspect ratio consistency.

[0115] The formula for calculating the aspect ratio consistency penalty is:

[0116] ;

[0117] In the formula, and These represent the width and height of the actual bounding box, respectively. and These represent the width and height of the prediction box, respectively.

[0118] The formula for calculating the target confidence loss is:

[0119] ;

[0120] In the formula, This represents the total number of all anchors or predicted locations, o a ∈{0,1} represents the first... The true target confidence label for each location (i.e., whether a target exists at that location). ∈[0,1] represents the model's prediction of the first... The confidence probability that a location contains the target.

[0121] Gated network training

[0122] The gating network uses weather category labels as supervision signals to learn the mapping relationship from the input image to the weather category. Let the gating network parameters be... The training process is as follows:

[0123] ;

[0124] In the formula, Indicates that the gating network is in the first stage. The parameters at the next iteration This represents the learning rate of the gating network. This represents the cross-entropy loss function of the gated network.

[0125] A gating network is assembled with four expert networks to form a complete inference system, given a test image x. test The system reasoning process is as follows:

[0126] ;

[0127] ;

[0128] In the formula, This represents the forward propagation function of the gated network. The output of the gating network represents the first... The activation probability of an expert This indicates the expert index selected by the Top-1 team. Indicates the use of optimal parameters The A network of experts, This represents the input test image. This indicates the final target detection result.

[0129] In this invention, as a preferred embodiment, a phased training strategy is adopted, transforming the training objective of the gated hybrid expert object detection system into a two-stage optimization problem. Simultaneously, to improve the model's generalization ability, a phased training strategy is employed. Regularization strategies and data augmentation methods include:

[0130] Phase 1: ;

[0131] Phase Two: ;

[0132] in, express The parameter set of an expert network ( (corresponding to four weather conditions) This represents the regularization coefficient of the expert network. express Regularization term, The parameters representing the gating network, Represents the regularization coefficient of the gated network. Represents the parameters of the gated network Regularization term, The regularization term (which is the sum of squares of the parameters in the corresponding model parameter matrix) is used to limit the size of the parameters, thereby preventing the model from overfitting during training.

[0133] In Phase 1, four weather expert models are trained independently to achieve full convergence in their respective weather domains. This phase focuses solely on optimizing the detection loss function, with complete decoupling of parameters between the expert models. Phase 2, with the expert model parameters frozen, trains a gating network for weather classification. This phase optimizes only the cross-entropy loss function of the gating network, enabling it to learn accurate expert routing strategies. Through these two training phases, each module of the system can independently optimize its corresponding task, avoiding gradient interference issues during joint training, ultimately achieving high-precision multi-weather adaptive target detection.

[0134] The gated network in this method can accurately identify weather conditions in input images with extremely low computational cost (approximately 0.5M parameters and inference time less than 5ms). Verification showed that the gated network achieved an average classification accuracy of 99.53% for four weather conditions on the test set: 99.68% for sunny scenes, 100% for foggy scenes, 99.37% for rainy scenes, and 99.05% for snowy scenes, validating the effectiveness of this gated network as an expert router.

[0135] Example

[0136] Dataset creation and evaluation metrics

[0137] A self-built multi-weather traffic light detection dataset, MW-TLDD (Multi-Weather Traffic Light Detection Dataset), was used to train and evaluate the CBDES-MoE-TLD network architecture constructed in this invention. MW-TLDD contains 15,352 images across multiple weather scenes, composed of three heterogeneous data sources: real-world collected data, the S2TLD dataset, and the TF6 dataset. The data is randomly partitioned into training, validation, and test sets in a ratio of approximately 8:1:1. Of these, 12,496 images were used for training, and 1,592 were used for validation. The complete training set (12,496 images) was used for joint training of the gating network to learn global weather features. Simultaneously, four subsets of 3,124 images each were divided according to weather domain from the training set, each used for independent training of a corresponding weather expert to fully exploit the discriminative features of specific weather domains. The test set contains 1,264 images, evenly covering four typical weather conditions (316 images for each weather condition). The test set includes 15 categories of fine-grained traffic light targets, covering red, green, and yellow circular traffic lights, as well as arrow traffic lights with different directional indication semantics, which can comprehensively reflect the complex distribution characteristics of traffic lights on real urban roads.

[0138] To verify the detection performance of various weather expert networks and the CBDES MoE TLD model under different weather conditions, expert networks for sunny, foggy, rainy, and snowy weather were used to detect test images in four weather scenarios, and the results were compared with those of the CBDES MoE TLD model. The test results show that each weather expert network has good detection performance in scenarios matching its training weather conditions, but under unmatched weather conditions, it is prone to decreased detection confidence, missed target detection, localization errors, or misclassification. The CBDESMoE TLD model proposed in this invention can select an expert network that matches the current weather conditions based on the weather features of the input image, thereby achieving stable detection in different weather scenarios.

[0139] Specifically, in sunny weather, all expert networks were able to detect traffic light targets well, with the CBDESMoE TLD model showing better performance in class distinction and detection result stability. In foggy weather, due to reduced visibility and decreased image contrast, the detection confidence of some non-matching expert networks decreased, while the CBDESMoE TLD model was still able to stably identify traffic light targets. In rainy weather, rain streaks, brightness changes, and background noise interfered with detection, and some non-matching expert networks were prone to misclassification, while the CBDESMoE TLD model was still able to identify traffic lights of different categories and directions. In snowy weather, snow particles, strong noise, and brightness interference further increased the detection difficulty, but the CBDESMoE TLD model still maintained relatively stable detection results, indicating that the weather-aware gating network and the Top-1 expert routing mechanism can improve the model's robustness to detection under complex weather conditions.

[0140] As shown in Figure 3(a), the traffic light detection results are displayed in a sunny scene. The CBDES MoE TLD model detected a total of 3 traffic lights, including two red circular traffic lights (red boxes in the figure) and one red left-turn arrow traffic light (yellow box in the figure). The detection confidence level of the red light located on one side of Figure 3(a) is 0.84, the detection confidence level of the left-turn arrow traffic light located on the other side of Figure 3(a) (yellow box in the figure) is 0.83, and the confidence level of the red light located on the left-turn arrow traffic light side in the figure is 0.82. The above results show that the present invention can accurately identify different types of traffic lights in a sunny scene and maintain a high detection confidence level.

[0141] As shown in Figure 3(b), the traffic light detection results are displayed in a foggy scene. The CBDES MoE TLD model detected two traffic light targets: a red circular traffic light (red box) and a green straight arrow traffic light (purple box), with corresponding detection confidence scores of 0.74 and 0.50, respectively. Despite the reduced image contrast and blurred target edges caused by fog, this invention can still complete the localization and category recognition of traffic light targets.

[0142] As shown in Figure 3(c), the traffic light detection results are displayed in a rainy weather scenario. The CBDES MoE TLD model detected a total of 4 traffic light targets, including 2 green straight arrows (purple boxes in the figure) and 2 red right-turn arrows (pink boxes in the figure). The detection confidence scores for the green straight arrows in the upper part of Figure 3(c) are 0.91, and for the red right-turn arrows, they are 0.85. The detection confidence scores for the green straight arrows and red right-turn arrows in the lower part of Figure 3(c) are both 0.91. Even under rain obstruction and complex weather interference conditions, this invention can still stably detect traffic light targets and accurately identify their categories.

[0143] As shown in Figure 3(d), the CBDES MoE TLD model detected two traffic light targets in a snowy scene: a green right-turn arrow signal light (light blue box in the figure) and a green circular signal light (green box in the figure). The corresponding detection confidence scores were 0.74 and 0.50, respectively. Despite the decrease in image contrast and blurring of target edges caused by fog, the present invention can still complete the localization and category recognition of traffic light targets.

[0144] In this embodiment, as a preferred approach, the CBDES MoE TLD model uses 0.25 as the target detection confidence threshold. When the model outputs a target detection confidence score greater than or equal to this threshold, the target is determined to be a valid detection result, and the location and category identification of the traffic light target are completed. The target detection confidence score is used to characterize the reliability of the model's prediction of the detection result. Its value ranges from 0 to 1. The higher the confidence score, the more reliable the model's prediction of the target category, and the higher the reliability of the detection result.

[0145] The confidence scores of each detected target in Figures 3(a) to (d) are all higher than the preset threshold, indicating that the CBDESMoE TLD model proposed in this method can effectively identify traffic light targets under different weather conditions.

[0146] The performance of the CBDES-MoE-TLD model proposed in this invention was compared with that of classical detection methods and single weather expert models on the MW-TLDD merged test set. The results are shown in Table 1.

[0147] Table 1. Performance comparison of classical detection methods, single weather expert models, and the method of this invention on the MW-TLDD merged test set.

[0148]

[0149] Compared to classical detection methods and single-expert model methods, the CBDES MoE TLD method (a MoE-based method based on a hybrid expert system framework) constructed in this invention achieves mAP50-95 (mean accuracy in the intersection-over-union ratio range of 0.5 to 0.95) improvements of 23.24, 4.19, and 3.80 percentage points respectively compared to SSD (Single Shot MultiBox Detector, a one-stage detection method), Faster R-CNN (Faster Region-based Convolutional Neural Network, a two-stage detection method), and YOLOv5s (a lightweight version of YOLOv5, a one-stage detection method). This indicates that the proposed method has stronger comprehensive detection capabilities in multi-weather traffic light detection tasks. Compared to single-weather expert models (expert-based methods), the advantages of CBDES MoE TLD are even more pronounced. The performance of the four single weather experts on the merged test set showed varying degrees of decline. Specifically, the mAP50 (mean accuracy at an intersection-over-union (IoU) threshold of 0.5) for the single Sunny Expert, Fog Expert, and Snow Expert were 65.55%, 64.02%, and 71.40%, respectively, indicating that single experts are prone to insufficient generalization ability when facing non-target weather domains. Compared to the best single Rain Expert, the mAP50-95 of CBDES MoE TLD improved from 52.59% to 61.80%, an improvement of 9.21 percentage points; and the mAP50 improved from 74.52% to 85.86%, an improvement of 11.34 percentage points, verifying the effectiveness of the weather-aware gating network and the Top-1 expert routing mechanism in complex multi-weather scenarios. Table 1 also shows the harmonic mean F1-score, an indicator used to comprehensively evaluate the balance between precision and recall.

[0150] From a computational complexity perspective, the four single-weather expert models, employing the same basic detection structure, all have a GFLOPs (given billions of floating-point operations, quantifying the computational complexity and hardware overhead during forward propagation inference) of 28.50. CBDES MoE TLD, however, activates only one weather expert during the inference phase and introduces an additional lightweight gating network, achieving a GFLOPs of 28.53, only 0.03 higher than the single-expert model. This demonstrates that the proposed sparse expert activation strategy avoids the linear increase in computational load caused by multi-expert parallel inference. Therefore, CBDES MoE TLD does not achieve improved accuracy by significantly increasing theoretical computational overhead, but rather achieves superior multi-weather detection performance while maintaining near-single-expert computational complexity.

[0151] Table 2. Comparison of mAP50 by category between different weather expert models and the method of this invention (CBDES MoE TLD) on the MW-TLDD merged test set.

[0152]

[0153] Table 2 compares the performance of the CBDES MoE TLD with the best-performing single weather expert in each category, using mAP50 as the evaluation metric. The results show that the CBDES MoE TLD outperforms the baseline of the best single expert in 14 out of 15 categories. Categories with significant improvements include green light pedestrians (+30.57 percentage points), green light left turns (+18.61 percentage points), traffic lights off (+18.46 percentage points), and red cross no-entry lights (+15.43 percentage points), indicating that the method of this invention has relatively stable gains across most traffic light categories. The only category that failed to surpass the best single expert is green light bicycles, which may be related to the uneven distribution of samples in this category across different weather subsets.

[0154] ablation experiment

[0155] To fully verify the effectiveness of each core design element in the hierarchical decoupled hybrid expert framework of this invention, a system ablation experiment was conducted. The performance difference between the CBDES MoE TLD hierarchical decoupled hybrid expert architecture and a single detection model was compared to verify the superiority of the heterogeneous weather expert system. Subsequently, the impact of different routing strategies on detection performance was further analyzed to demonstrate the crucial role of the weather-aware gating network in achieving accurate expert scheduling.

[0156] To verify the effectiveness of the weather expert decoupling mechanism proposed in this invention, this paradigm was compared with the Single-YOLO detection model, and the results are shown in Table 3. Table 3 introduces precision (the proportion of samples predicted as positive by the model that are actually positive) and recall (the proportion of samples that are actually positive that are correctly predicted by the model) as supplementary indicators for evaluating the model's weather expert decoupling performance.

[0157] Table 3 Performance Comparison between Hybrid Expert Model and Single YOLO Model

[0158]

[0159] As shown in Table 3, the CBDES MoE TLD paradigm exhibits significant performance gains, with mAP50 reaching 85.86% (an improvement of 1.42 percentage points), mAP50-95 reaching 61.80% (an improvement of 0.67 percentage points), and the F1 score increasing from 81.53% to 82.54%. The poor performance of the Single-YOLO paradigm is attributed to the performance interference caused by shared parameters when a single model simultaneously handles multiple heterogeneous weather domains. The proposed paradigm's advantage lies in the fact that independently trained weather experts learn specialized feature representations only for specific image degradation patterns, effectively avoiding negative transfer effects across multiple weather scenarios. By decoupling the feature learning process for each weather condition, each expert can fully optimize its detection capabilities within a specific weather domain, thereby achieving higher overall detection accuracy and robustness.

[0160] Table 4 shows the ablation experiment results comparing different routing strategies in the weather-aware gating network.

[0161] Table 4 Comparison of Routing Strategies

[0162]

[0163] As shown in Table 4, the random routing strategy performed the worst because it randomly selects an expert for testing during inference, representing the lower bound of performance without intelligent routing, with an mAP50 of only 68.87%. This indicates that the lack of an intelligent routing mechanism leads to severe performance degradation. This invention, by utilizing a Top-1 sparse activation strategy, proposes the CBDES MoE TLD, achieving an mAP50 score of 85.86%, significantly surpassing the random routing baseline. The gated routing of this invention is completely consistent with the ideal routing (representing the upper bound of the theoretical performance of the routing mechanism) across all metrics. This result strongly demonstrates the superior capabilities of lightweight gated networks in weather feature extraction and classification.

[0164] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A traffic light recognition method based on a hierarchical decoupled hybrid expert model, characterized in that, include: Step 1: Construct a hierarchical, decoupled hybrid expert model, including: a lightweight weather perception gating network and heterogeneous expert modules; The heterogeneous expert module consists of YOLOv8 detectors trained independently based on multiple different weather conditions, corresponding to multiple different weather scenarios and multiple different weather experts; Step 2: Conduct independent training of the expert network and gated network separately; Step 3: The road scene images captured by the vehicle-mounted camera are encoded with weather features by a lightweight weather-aware gating network. The weather features of the current image are determined. Based on the Top-1 hard routing inference strategy, the gating network selects the optimal expert path according to the weather features of the input image, activates only the corresponding single weather expert to perform detection, and outputs the final traffic signal category and bounding box.

2. The traffic light recognition method based on a hierarchical decoupled hybrid expert model according to claim 1, characterized in that, The various weather conditions include: sunny, foggy, rainy, and snowy. Set up four expert network ensembles with identical structures but independent parameters: ； In the formula: These represent expert models trained for sunny, foggy, rainy, and snowy weather conditions, respectively. Based on dedicated training datasets for various weather conditions, the detection performance of each expert in its target domain is maximized through a standard YOLOv8 optimization process. The optimal expert model is: ； In the formula: Let represent the optimal model for the i-th expert; This represents the learnable parameters of the corresponding expert network. This represents the training dataset for the i-th weather condition. This is the standard detection loss function for YOLOv8, which includes classification loss, bounding box regression loss, and target confidence loss.

3. The traffic light recognition method based on a hierarchical decoupled hybrid expert model according to claim 2, characterized in that, The weather-sensing gating network includes: A lightweight feature extractor consisting of three cascaded convolutional blocks is constructed. After three convolutional blocks, global adaptive average pooling is used to compress the spatial dimension feature information into a fixed-length feature vector. Based on the extracted global feature vectors, they are mapped to the expert dimension space through a fully connected layer. The activation probability of each expert network is calculated using the Softmax function, and the calculation formula is as follows: ； In the formula, ℝ represents the set of real numbers. This represents the linear output vector of the fully connected layer, containing the unnormalized score of each expert. Indicates the first The unnormalized scores corresponding to each expert network This indicates the first step in the Softmax normalization summation process. The unnormalized scores corresponding to each expert network This represents the weight matrix of the fully connected layer. 3D feature vector mapping to Dimensional Expert Space This represents the corresponding bias vector. This indicates the number of expert networks, corresponding to four weather conditions: sunny, foggy, rainy, and snowy. , indicating the first Gating probabilities of an expert network.

4. The traffic light recognition method based on a hierarchical decoupled hybrid expert model according to claim 3, characterized in that, The calculation formula for the Top-1 expert selection strategy is as follows: ； In the formula, This indicates the expert index selected by the Top-1 selection, corresponding to the expert number with the highest gating probability. The operator that returns the index corresponding to the maximum value. Indicates the selected number The forward propagation function of an expert network, with the original image as input. , This represents the final traffic light target detection output, which includes the predicted category label and bounding box coordinates.

5. The traffic light recognition method based on a hierarchical decoupled hybrid expert model according to claim 4, characterized in that, The gated network training employs a multi-class cross-entropy loss function for supervised learning, and its calculation formula is as follows: ； In the formula, This represents the training loss value of the gating network. Indicates the training batch size. Indicates the first The unique thermal encoding label vector for the weather category of each sample, when the sample belongs to the... When weather conditions Otherwise, it is 0. Indicates that the gating network is for the first The predicted number of samples The probability value of weather-like conditions.

6. The traffic light recognition method based on a hierarchical decoupled hybrid expert model according to claim 5, characterized in that, The hierarchical decoupled hybrid expert model employs a phased training strategy, including: During the independent training phase of the expert networks, the four expert networks were trained independently on their respective subsets of weather data. Given a set of weather categories W = {sunny, fog, rain, snow}, the first... The training process for an expert network is as follows: ； In the formula, Indicates the first The expert network in the first Model parameters at the next iteration Indicates the learning rate. Indicates the parameter Operator for finding gradient Indicates the first The detection loss function of an expert network; During the training phase of the gated network, the gated network uses weather category labels as supervision signals to learn the mapping relationship from the input image to the weather category. Let the parameters of the gated network be... The training process is as follows: ； in, Indicates that the gating network is in the first stage. The parameters at the next iteration This represents the learning rate of the gating network. The cross-entropy loss function of a gated network; In the system integration and inference phase, the gating network and four expert networks are assembled into a complete inference system, given a test image x. test The system reasoning process is as follows: ； ； in, Denotes the forward propagation function of the gated network. The output of the gating network represents the first... The activation probability of an expert This indicates the expert index selected by the Top-1 team. Indicates using optimal parameters The A network of experts, This indicates the final target detection result.

Images