Traffic signal identification method, apparatus and device, and storage medium

Abstract

The invention discloses a traffic signal identification method and device, equipment and a storage medium. Firstly, a current road traffic scene graph can be obtained, and convolution feature extraction is performed by using an image feature extraction module in a traffic signal recognition model to obtain a traffic scene feature graph. And then, a parallel identification module in the model is used for carrying out classification identification on the feature map, and a passing permission state is output. And the parallel identification module comprises five parallel sub-modules: a global state identification module, a turn-round state identification module, a left-turn state identification module, a straight-going state identification module and a right-turn state identification module which are respectively used for outputting passing permission states in corresponding directions. According to the invention, a unified end-to-end traffic signal recognition model is adopted, efficient and accurate recognition is realized, computing resource consumption and processing time are remarkably reduced, development and maintenance are simplified, error transmission is avoided, and a parallel state recognition module is further included, so that robustness in a complex environment and under the condition of multiple signal lamps is improved.

Description

Technical Field

[0001] This application relates to the fields of image processing and computer vision technology, specifically to a traffic signal recognition method, apparatus, device, and storage medium. Background Technology

[0002] Using a deep learning object detection model and an image classification model to jointly perform traffic light recognition is a widely used and effective multi-stage cascaded method. While this method improves accuracy, it also has some significant drawbacks. First, it has high computational cost because it requires running two models simultaneously, increasing processing time and resource consumption. Second, errors in the detection stage are propagated and amplified in the classification stage, leading to significant error accumulation, especially in complex traffic scenarios. Furthermore, this recognition method is less robust and susceptible to interference when multiple traffic lights coexist. More importantly, the training processes for the two models are independent, increasing both development complexity and maintenance costs, as each model needs to be optimized and tuned separately to ensure they work collaboratively.

[0003] Therefore, how to provide a traffic light recognition method that is computationally efficient, has low error accumulation, and is robust, in order to reduce development complexity and maintenance costs, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a traffic signal recognition method, apparatus, device, and storage medium that can reduce computational overhead and error accumulation, improve robustness in complex scenarios, and reduce development complexity and maintenance costs while ensuring the accuracy and real-time performance of traffic light recognition.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] A traffic signal recognition method, the method comprising:

[0007] Obtain the current road traffic scene image to be identified;

[0008] The current road traffic scene map is subjected to convolutional feature extraction using the image feature extraction module in the traffic signal recognition model to obtain a traffic scene feature map;

[0009] The traffic scene feature map is classified and identified using the parallel recognition module in the traffic signal recognition model to obtain the traffic permission status. The parallel recognition module includes five parallel status recognition modules: a global status recognition module, a U-turn status recognition module, a left-turn status recognition module, a straight-ahead status recognition module, and a right-turn status recognition module. The output of the global status recognition module indicates whether there is a valid traffic light in the current feature map. The output of the U-turn status recognition module indicates whether the U-turn direction is permissible in the current feature map. The output of the left-turn status recognition module indicates whether the left-turn direction is permissible in the current feature map. The output of the straight-ahead status recognition module indicates whether the straight-ahead direction is permissible in the current feature map. The output of the right-turn status recognition module indicates whether the right-turn direction is permissible in the current feature map.

[0010] Output the access permission status.

[0011] In one possible implementation, the traffic signal recognition model is obtained by end-to-end training an initial recognition model, which includes an initial feature extraction module, a first initial recognition module, a second initial recognition module, a third initial recognition module, a fourth initial recognition module, and a fifth initial recognition module.

[0012] The end-to-end training process of the traffic signal recognition model includes:

[0013] Obtain a training dataset; the training dataset includes a first training subset, a second training subset, a third training subset, a fourth training subset, and a fifth training subset. Each training subset includes multiple samples, and each sample includes a road traffic scene image and its corresponding traffic status label and feature map. The first training subset includes multiple samples with traffic light status labels and feature map labels. The second training subset includes multiple samples with U-turn status labels and feature map labels. The third training subset includes multiple samples with left turn status labels and feature map labels. The fourth training subset includes multiple samples with straight-ahead status labels and feature map labels. The fifth training subset includes multiple samples with right turn status labels and feature map labels. The traffic light status is used to indicate whether there is a valid traffic light in the corresponding road traffic scene image.

[0014] The backbone network and five classification heads are initialized to obtain the initial feature extraction module, the first initial recognition module, the second initial recognition module, the third initial recognition module, the fourth initial recognition module, and the fifth initial recognition module;

[0015] The first to fifth training subsets are used as inputs to the first to fifth initial recognition modules, respectively, and all training subsets are used as inputs to the initial feature extraction module. The initial recognition model is iteratively trained using a cosine annealing learning rate decay strategy until the training cutoff condition is reached, thereby obtaining the image feature extraction module, the global state recognition module, the U-turn state recognition module, the left turn state recognition module, the straight-ahead state recognition module, and the right turn state recognition module.

[0016] In one possible implementation, the backbone network comprises a grouped convolutional neural network Xception with the fully connected layers and Softmax layers removed.

[0017] In one possible implementation, the classification head includes two fully connected layers and a Softmax activation function.

[0018] In one possible implementation, the traffic signal recognition model is built using the PyTorch deep learning framework.

[0019] A traffic signal recognition device, the device comprising:

[0020] The first acquisition unit is used to acquire the current road traffic scene map to be identified;

[0021] The feature extraction unit is used to perform convolutional feature extraction on the current road traffic scene map using the image feature extraction module in the traffic signal recognition model to obtain a traffic scene feature map.

[0022] A traffic signal recognition unit is used to classify and recognize the traffic scene feature map using the parallel recognition modules in the traffic signal recognition model to obtain the traffic permission status. The parallel recognition modules include five parallel status recognition modules: a global status recognition module, a U-turn status recognition module, a left-turn status recognition module, a straight-ahead status recognition module, and a right-turn status recognition module. The output of the global status recognition module indicates whether there is a valid traffic light in the current feature map. The output of the U-turn status recognition module indicates whether the U-turn direction is permissible in the current feature map. The output of the left-turn status recognition module indicates whether the left-turn direction is permissible in the current feature map. The output of the straight-ahead status recognition module indicates whether the straight-ahead direction is permissible in the current feature map. The output of the right-turn status recognition module indicates whether the right-turn direction is permissible in the current feature map.

[0023] The status output unit is used to output the passage permission status.

[0024] In one possible implementation, the traffic signal recognition model is obtained by end-to-end training an initial recognition model, which includes an initial feature extraction module, a first initial recognition module, a second initial recognition module, a third initial recognition module, a fourth initial recognition module, and a fifth initial recognition module.

[0025] In one possible implementation, the device further includes:

[0026] The second acquisition unit is used to acquire a training dataset. The training dataset includes a first training subset, a second training subset, a third training subset, a fourth training subset, and a fifth training subset. Each training subset includes multiple samples. Each sample includes a road traffic scene map and its corresponding traffic status label and feature map. The first training subset includes multiple samples with traffic light status labels and feature map labels. The second training subset includes multiple samples with U-turn status labels and feature map labels. The third training subset includes multiple samples with left turn status labels and feature map labels. The fourth training subset includes multiple samples with straight-ahead status labels and feature map labels. The fifth training subset includes multiple samples with right turn status labels and feature map labels. The traffic light status is used to indicate whether there is a valid traffic light in the corresponding road traffic scene map.

[0027] An initialization unit is used to initialize the backbone network and five classification heads to obtain the initial feature extraction module, the first initial recognition module, the second initial recognition module, the third initial recognition module, the fourth initial recognition module, and the fifth initial recognition module;

[0028] The training unit is used to take the first to fifth training subsets as inputs to the first to fifth initial recognition modules, and to take all training subsets as inputs to the initial feature extraction module. The initial recognition model is iteratively trained using a cosine annealing learning rate decay strategy until the training cutoff condition is reached, and then the training stops, thus obtaining the image feature extraction module, the global state recognition module, the U-turn state recognition module, the left turn state recognition module, the straight-ahead state recognition module, and the right turn state recognition module.

[0029] In one possible implementation, the backbone network includes Xception with the fully connected layer and Softmax layer removed.

[0030] A traffic signal recognition device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the traffic signal recognition method as described above.

[0031] A computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the traffic signal recognition method as described above.

[0032] Compared with the prior art, this application has the following advantages:

[0033] This application provides a traffic signal recognition method, apparatus, device, and storage medium. Specifically, when executing the traffic signal recognition method provided in this application, the current road traffic scene map is first acquired. Then, the scene map is convolved by the image feature extraction module in the traffic signal recognition model to obtain a rich traffic scene feature map. Next, the feature map is classified and recognized using the parallel recognition module in the model to determine the traffic permission status. This parallel recognition module consists of five parallel state recognition sub-modules: global state recognition, U-turn state recognition, left turn state recognition, straight-ahead state recognition, and right turn state recognition. Among them, the global state recognition module determines whether there is a valid traffic light, and the other four modules output the traffic permission status for the corresponding direction. Finally, the overall traffic permission status is output based on these recognition results, realizing accurate traffic signal recognition for multiple directions and multiple states. This application adopts a single unified end-to-end traffic signal recognition model to achieve direct recognition of traffic signals, effectively reducing computational resource consumption and processing latency, while simplifying the development process and reducing development difficulty. Since only one traffic signal recognition model needs to be maintained, there is no need to optimize multiple independent models separately, greatly reducing debugging and maintenance costs. This traffic signal recognition model directly classifies and identifies the overall traffic scene, avoiding the error propagation and accumulation problems in the traditional two-stage "detection-classification" method. Furthermore, the traffic signal recognition model designed in this application includes five parallel state recognition modules, which respectively determine U-turns, left turns, straight ahead, right turns, and overall traffic flow. This structure enhances the system's recognition capability and stability in complex traffic environments and situations with multiple traffic lights. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart of a traffic signal recognition method provided in this application embodiment;

[0036] Figure 2This is a schematic diagram of the structure of a traffic signal recognition model provided in an embodiment of this application;

[0037] Figure 3 A flowchart illustrating a method for training a traffic signal recognition model, as provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of the structure of a traffic signal recognition device provided in an embodiment of this application. Detailed Implementation

[0039] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0040] By combining deep learning object detection models and image classification models to perform traffic light recognition, this multi-stage cascaded "detection + classification" approach is currently the most widely used and effective solution in the field of traffic light recognition. The basic process first uses an object detection model (such as the YOLO series) to locate all possible traffic lights from complete image frames captured by a camera, outputting the corresponding bounding boxes. Then, based on these bounding boxes, the traffic light region is cropped out and fed into an independent image classification model (such as Residual Network (ResNet), Mobile Network (MobileNet), Efficient Network (EfficientNet), etc.) to identify the specific state of the traffic light, such as red, green, yellow, or arrow direction.

[0041] While this method offers certain advantages in improving recognition accuracy, it also reveals several significant drawbacks in practical in-vehicle deployments. First, because the system needs to maintain and run two independent deep learning models simultaneously, not only does it significantly increase computational resource consumption and memory usage, but the serial execution of the two models also introduces significant inference latency, severely hindering real-time performance optimization for cost- and power-sensitive in-vehicle embedded platforms. Second, as a multi-stage cascaded system, errors generated in the object detection stage are losslessly amplified and propagated to the classification stage, causing error accumulation. For example, when distant or small traffic lights are not successfully located by the detection model, subsequent classification models cannot recognize them at all. Furthermore, insufficiently precise detection boxes result in a large amount of background noise (such as nearby red neon lights or billboards), easily misleading the classification model to output incorrect results, severely impacting the overall recognition performance.

[0042] Furthermore, the classification model in this method only obtains local image information after cropping, completely losing the global context of the original image. This is particularly fatal at complex intersections. In reality, when human drivers determine the status of traffic lights, they combine rich information such as lane lines, directional signs, and the dynamics of surrounding vehicles to make a comprehensive judgment. The isolated classification model cannot utilize this key information, resulting in insufficient robustness in complex traffic environments with multiple traffic lights and susceptibility to environmental interference. Finally, the independent training processes of the two models mean that different training datasets, annotation standards, model versions, and deployment processes need to be managed separately, greatly increasing the workload and cost of development and maintenance. This not only slows down the pace of technological iteration but also poses significant challenges to practical applications.

[0043] In summary, although the "detection + classification" cascade scheme performs well in the field of traffic light recognition, it has drawbacks such as high computational cost, error accumulation, insufficient robustness, and complex training and maintenance processes.

[0044] To address this issue, this application provides a traffic signal recognition method, apparatus, device, and storage medium. First, a road traffic scene image to be recognized is acquired. Then, the image feature extraction module in the traffic signal recognition model performs a convolution operation on the scene image to obtain a feature map reflecting traffic scene information. Next, the parallel recognition module in the model classifies and recognizes the feature map to obtain the traffic permit status. The parallel recognition module consists of five parallel state recognition sub-modules: a global state recognition module, a U-turn state recognition module, a left-turn state recognition module, a straight-ahead state recognition module, and a right-turn state recognition module. The global state recognition module determines whether a valid traffic light signal exists in the feature map, while the other four modules are responsible for determining the traffic permit status for U-turns, left turns, straight-ahead, and right turns, respectively. Finally, the current traffic permit status is determined and output based on the output results of each state recognition module. By employing a unified end-to-end traffic signal recognition model, this application effectively reduces computational overhead and processing time. Using a unified end-to-end traffic signal recognition model simplifies the development process and reduces debugging and maintenance costs. Simultaneously, by directly using a traffic signal recognition model to classify and identify traffic scenes, the problem of error propagation between the detection and classification stages is successfully avoided. Furthermore, this application introduces five parallel state recognition modules to identify different traffic states. This design makes the system more robust and better adaptable to complex scenarios and situations with multiple traffic lights.

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0046] See Figure 1 The figure is a flowchart of a traffic signal recognition method provided in an embodiment of this application, as shown below. Figure 1 As shown, the traffic signal recognition method may include steps S101-S104:

[0047] S101: Obtain the current road traffic scene map to be identified.

[0048] To achieve accurate traffic signal recognition, the first step is to acquire a map of the current road traffic scene. This process typically involves using cameras installed in vehicles or fixed locations to capture real-time images of the current road traffic environment. Specifically, the cameras capture images containing traffic lights, road signs, and other key information, creating one or more maps of the current road traffic scene. This image data forms the basis for subsequent recognition and processing.

[0049] The current road traffic scene image refers to an image captured in real time that reflects all visual information in the current road environment. The current road traffic scene reflects information such as traffic lights, road conditions, surrounding vehicles and pedestrians, including the status of traffic lights (red, green, yellow), lane markings, pedestrian crossing signals, and other important information in the surrounding environment. These images should have sufficient clarity and resolution to ensure that traffic signals and other key information can be accurately identified. Furthermore, the impact of external environmental changes such as lighting conditions and weather on image quality must be considered during image acquisition to ensure the accuracy and reliability of subsequent recognition steps.

[0050] In this way, the system can acquire and process information about the current road environment in real time, providing a reliable data foundation for subsequent traffic signal recognition.

[0051] S102: Use the image feature extraction module in the traffic signal recognition model to perform convolution feature extraction on the current road traffic scene map to obtain a traffic scene feature map.

[0052] To effectively capture key information in traffic scenarios, key features can be extracted from the acquired current road traffic scene map. This process is accomplished using the image feature extraction module in the traffic signal recognition model. Specifically, this module employs the Xception grouped convolutional neural network, which has been trained with fully connected layers and Softmax layers removed.

[0053] The Xception neural network is a highly efficient deep learning architecture that extracts multi-layered features from images through deep convolutional operations. In this step, we use a grouped convolutional neural network (Extremely Deep Neural Networks, Xception) model with fully connected layers and softmax layers removed, retaining its powerful convolutional feature extraction capabilities. These convolutional layers can capture local patterns and structures in road scene images, such as edges, textures, and shapes. Through a series of convolutional layers, the model can progressively extract feature information at different levels.

[0054] The result after processing by convolutional layers is called a traffic scene feature map. A traffic scene feature map is a feature representation generated after the original image has been processed by a convolutional neural network. It retains the key visual information in the original image but removes unnecessary details. Feature maps typically contain multiple channels, each representing a different feature dimension, thus comprehensively describing traffic signals, road signs, and other important elements in the image. These feature maps provide an efficient and accurate foundation for subsequent classification and recognition steps.

[0055] In this way, the system can efficiently extract key features from complex road traffic scene maps and generate traffic scene feature maps, providing clear and focused information for subsequent classification and recognition.

[0056] S103: The parallel recognition module in the traffic signal recognition model is used to classify and recognize the traffic scene feature map to obtain the passage permission status.

[0057] To obtain accurate traffic permit status, the extracted traffic scene feature map can be further classified and identified. This process is accomplished using the parallel recognition module in the traffic signal recognition model. This module includes five parallel status recognition modules: a global status recognition module, a U-turn status recognition module, a left-turn status recognition module, a straight-ahead status recognition module, and a right-turn status recognition module.

[0058] Specifically, these parallel recognition modules independently process traffic scene feature maps and output traffic permission status for different directions:

[0059] Global State Recognition Module: This module's output indicates whether a valid traffic light exists in the current feature map. This module is primarily responsible for detecting traffic lights in the image to determine if further analysis of traffic conditions in other specific directions is needed.

[0060] U-turn status recognition module: The output of this module indicates whether the U-turn direction is passable in the current feature map. If the U-turn signal light is detected to be green or a sign indicating that a U-turn is permitted is present, the corresponding output will indicate that the U-turn direction is passable.

[0061] Left-turn status recognition module: The output of this module indicates whether the left-turn direction is permissible in the current feature map. If the left-turn signal light is detected to be green or a sign indicating that a left turn is permitted, the corresponding output will indicate that the left-turn direction is permissible.

[0062] Straight-ahead Status Recognition Module: The output of this module indicates whether the straight-ahead direction is passable in the current feature map. If a green straight-ahead signal light or a sign indicating that straight-ahead is permitted is detected, the corresponding output will indicate that the straight-ahead direction is passable.

[0063] Right Turn Status Recognition Module: This module's output indicates whether a right turn is permitted in the current feature map. If a green right turn signal light or a right turn permission sign is detected, the corresponding output will indicate that a right turn is permitted.

[0064] Through these five parallel recognition modules, the system can comprehensively and accurately identify the traffic permit status in multiple directions on the current road traffic scene map and provide comprehensive traffic permit information. This parallel processing method not only improves the system's recognition efficiency but also enhances the accuracy of recognition, ensuring the reliability and timeliness of decision-making.

[0065] S104: Output the passage permission status.

[0066] After completing multi-directional classification and recognition of traffic scene feature maps, the system integrates the output results of each state recognition module to form the final traffic permission status. This output not only clearly indicates whether passage is permitted in each direction (such as U-turn, left turn, straight ahead, and right turn), but also reflects the effectiveness of traffic lights in the overall road environment. By unifying the output of this information, the system provides clear and accurate passage guidance for vehicles or traffic management equipment, which helps to achieve intelligent traffic control and safe driving, thereby improving road traffic efficiency and traffic safety levels.

[0067] Based on the content of S101-S104, the current road traffic scene image to be identified is first obtained. Then, the image feature extraction module in the traffic signal recognition model performs convolutional feature extraction on the scene image to generate a traffic scene feature map. Next, the parallel recognition module in the model classifies and identifies the feature map to obtain the traffic permission status. This parallel recognition module consists of five parallel state recognition sub-modules: a global state recognition module, a U-turn state recognition module, a left turn state recognition module, a straight-ahead state recognition module, and a right turn state recognition module. Among them, the global state recognition module is used to determine whether there are valid traffic lights in the current feature map, and the other four modules output the traffic permission status for the corresponding direction (U-turn, left turn, straight-ahead, right turn). Finally, the recognition results of each module are combined to output the overall traffic permission status. This application achieves efficient and accurate traffic signal recognition through a unified end-to-end traffic signal recognition model, significantly reducing computational resource consumption and processing time, while simplifying the development and maintenance process. This model directly classifies the entire traffic scene, avoiding the error propagation problem in traditional multi-stage methods. In addition, the built-in five parallel state recognition modules make judgments for different traffic directions, which improves the robustness and stability of the system in complex environments and under multiple traffic light conditions.

[0068] In one possible implementation, the method further includes: preprocessing the current road traffic scene map to obtain a preprocessed image.

[0069] Preprocessing includes cropping and scaling the current road traffic scene image to 1920x704 pixels.

[0070] In one possible implementation, the step of using the image feature extraction module in the traffic signal recognition model to perform convolutional feature extraction on the current road traffic scene map to obtain a traffic scene feature map includes:

[0071] The image feature extraction module in the traffic signal recognition model is used to perform convolutional feature extraction on the preprocessed image to obtain a traffic scene feature map.

[0072] In one possible implementation, this application provides a structure for a traffic signal recognition model, see [link to relevant documentation]. Figure 2 , Figure 2This is a schematic diagram of the structure of a traffic signal recognition model provided in an embodiment of this application. Specifically, the traffic signal recognition model includes an image feature extraction module and a parallel recognition module; the parallel recognition module includes five parallel state recognition modules; the five parallel state recognition modules include a global state recognition module, a U-turn state recognition module, a left turn state recognition module, a straight-ahead state recognition module, and a right turn state recognition module; the output of the image feature extraction module is connected to the input of the global state recognition module, the U-turn state recognition module, the left turn state recognition module, the straight-ahead state recognition module, and the right turn state recognition module, respectively.

[0073] In one possible implementation, the traffic signal recognition model is obtained by end-to-end training an initial recognition model, which includes an initial feature extraction module, a first initial recognition module, a second initial recognition module, a third initial recognition module, a fourth initial recognition module, and a fifth initial recognition module.

[0074] In one possible implementation, this application provides a training method for a traffic signal recognition model, see [link to relevant documentation]. Figure 3 , Figure 3 This application provides a flowchart of a method for training a traffic signal recognition model. Specifically, the end-to-end training process of the traffic signal recognition model can be implemented through steps S301-303:

[0075] S301: Obtain the training dataset.

[0076] To build an efficient and robust traffic signal recognition model, it is first necessary to obtain a training dataset covering diverse and complex road environments. This training dataset is divided into five training subsets, designated as the first to fifth subsets, each containing multiple samples. Each sample consists of a road traffic scene map, a corresponding traffic status label, and an extracted feature map.

[0077] Specifically, the first training subset includes samples with traffic light status labels and feature map labels to indicate whether there are valid traffic lights in the scene map; the second to fifth training subsets contain traffic status labels and corresponding feature map labels for U-turn, left turn, straight and right turn directions, respectively, providing specialized training data for different traffic directions.

[0078] During the data collection phase of the training dataset, the coverage was extensive, encompassing different cities to accommodate differences in traffic rules, as well as various weather conditions (sunny, rainy, snowy, foggy), various lighting environments (daytime, nighttime, dusk, backlight, tunnel entrance), and diverse and complex intersections (such as multi-way intersections, irregular intersections, temporary construction areas, etc.), to ensure the model's generalization ability and robustness in practical applications.

[0079] Furthermore, a novel, task-oriented structured annotation method is employed for the collected image data. Unlike traditional annotation based on traffic light bounding boxes, each image's label uses a five-dimensional binary classification vector that directly describes the traffic status, with the following specific dimensions:

[0080] Global State: Determines the overall traffic light status in the scene, categorizing it as "no traffic lights / all off" or "valid traffic lights exist";

[0081] U-turn permission status: Indicates whether passage is permitted in the direction of a U-turn;

[0082] Left turn permission status: Indicates the permission status for left turns;

[0083] Straight-ahead traffic status: Indicates whether passage in the straight-ahead direction is permitted;

[0084] Right turn traffic status: Indicates the traffic status for right turns.

[0085] This annotation method directly teaches the model the target of determining the final passage permission, enabling traffic signal recognition to achieve end-to-end learning and significantly improving the effectiveness of model training and the accuracy of prediction.

[0086] By subdividing the training dataset into five subsets for different traffic conditions, each containing rich and specially labeled samples, highly targeted data support was achieved. This fine-grained partitioning not only enables the model to fully learn the features of various traffic conditions but also enhances its ability to perceive global information such as the effectiveness of traffic lights, thereby improving the model's understanding and accuracy in judging complex traffic scenarios.

[0087] In one possible implementation, the traffic signal recognition model is built using the PyTorch deep learning framework, which supports end-to-end training of the backbone network and the classification head.

[0088] S302: Initialize the backbone network and 5 classification heads to obtain the initial feature extraction module, the first initial recognition module, the second initial recognition module, the third initial recognition module, the fourth initial recognition module and the fifth initial recognition module.

[0089] During the model training phase, the backbone network and five classification heads were first initialized using the PyTorch deep learning framework. Specifically, the backbone network adopted Xception weights pre-trained on the ImageNet dataset. The introduction of this pre-trained parameter effectively accelerated the model's convergence speed and improved the performance of feature extraction. Through this backbone network, an initial feature extraction module was constructed, responsible for extracting rich convolutional features from the input road traffic scene map. Simultaneously, for different dimensions of traffic permit recognition, the first to fifth classification heads were initialized, corresponding to global state recognition, U-turn state recognition, left turn state recognition, straight-ahead state recognition, and right turn state recognition, respectively, thus forming the first to fifth initial recognition modules. These modules work together to provide the infrastructure for subsequent multi-task classification, enabling independent discrimination and joint optimization of traffic states in different directions.

[0090] Furthermore, during training, the model uses a batch size of 64 to ensure a balance between computational efficiency and stability. The optimizer chosen is Adam, whose adaptive learning rate mechanism helps accelerate convergence. The initial learning rate is set to 1e-2 to provide a larger parameter update range for faster learning. In the early stages of multi-task training, the loss weights w1 to w5 for each task are all assigned the same value of 1.0 as a baseline configuration for testing and evaluation. Subsequently, these weights are dynamically adjusted based on the performance feedback from the validation set to more rationally allocate training priorities, thereby improving the overall model performance. The entire training process consists of 100 epochs to ensure the model undergoes sufficient iterations, achieves good convergence, and optimizes performance.

[0091] In one possible implementation, the backbone network uses a grouped convolutional neural network called Xception, but removes its original fully connected layers and softmax layers in order to focus on extracting more general and richer image features, providing effective input for subsequent classification tasks.

[0092] In one possible implementation, the classification head is designed to consist of two fully connected layers, with the class probability distribution output via a Softmax activation function. This structure ensures both the non-linear expressiveness of the classification module and the ability to map the model's predictions for various travel states into interpretable probability values, facilitating subsequent decision-making and analysis.

[0093] S303: The first to fifth training subsets are used as the inputs to the first to fifth initial recognition modules, respectively, and all training subsets are used as the inputs to the initial feature extraction module. The initial recognition model is iteratively trained using a cosine annealing learning rate decay strategy until the training cutoff condition is reached, and then the training stops, thus obtaining the image feature extraction module, the global state recognition module, the U-turn state recognition module, the left turn state recognition module, the straight-ahead state recognition module, and the right turn state recognition module.

[0094] During training, the first to fifth training subsets are first input into the corresponding first to fifth initial recognition modules, ensuring that each module learns specifically for its unique traffic state label. Simultaneously, all training subsets are passed as a whole to the initial feature extraction module to uniformly extract deep features from the road traffic scene image. The entire training process employs a cosine annealing learning rate decay strategy to effectively adjust the learning rate and promote stable convergence of model parameters. Through multiple rounds of iterative training, training stops when a preset training cutoff condition is met, resulting in an optimized image feature extraction module and five state recognition modules corresponding to global state, U-turn, left turn, straight ahead, and right turn. These modules together constitute a complete and efficient traffic signal recognition system.

[0095] By inputting different training subsets into the corresponding initial recognition modules, and simultaneously inputting all data into the feature extraction module, an organic combination of feature sharing and task-specific recognition is achieved. A cosine annealing learning rate decay strategy is used to effectively balance the learning rate during training, avoiding premature entry into local optima or slow convergence. Iterative training continues until the termination condition is met, ensuring sufficient model learning and good convergence, resulting in a final image feature extraction module and multiple state recognition modules with strong generalization ability and stability.

[0096] In steps S301-S303, the training dataset is subdivided into five subsets, corresponding to the traffic light status and the status labels and feature maps of the four traffic directions. This structured annotation method enables the model to learn end-to-end multi-dimensional information that directly reflects traffic conditions, avoiding the error accumulation problem caused by relying on intermediate detection results in traditional methods, and fundamentally improving recognition accuracy and stability. Simultaneously, the design of a backbone network combined with five classification head modules achieves the organic integration of feature extraction and multi-task recognition. This modular architecture not only simplifies the training process and reduces model complexity but also facilitates subsequent adjustments and upgrades for single tasks, thereby reducing the difficulty and cost of later maintenance. Furthermore, by inputting different subsets into the corresponding initial recognition modules and using all data for the feature extraction module, supplemented by a cosine annealing learning rate strategy, smooth convergence and performance optimization of the training process are achieved. The overall solution avoids error propagation caused by multi-stage pipelined training, making training simpler and more efficient, and the model exhibits good robustness in the face of complex and ever-changing traffic scenarios.

[0097] See Figure 4 , Figure 4 This is a schematic diagram of the structure of a traffic signal recognition device provided in an embodiment of this application. Figure 4 As shown, the traffic signal recognition device includes:

[0098] The first acquisition unit 401 is used to acquire the current road traffic scene map to be identified;

[0099] Feature extraction unit 402 is used to perform convolutional feature extraction on the current road traffic scene map using the image feature extraction module in the traffic signal recognition model to obtain a traffic scene feature map;

[0100] The traffic signal recognition unit 403 is used to classify and recognize the traffic scene feature map using the parallel recognition module in the traffic signal recognition model to obtain the traffic permission status. The parallel recognition module includes five parallel status recognition modules: a global status recognition module, a U-turn status recognition module, a left turn status recognition module, a straight-ahead status recognition module, and a right turn status recognition module. The output of the global status recognition module indicates whether there is a valid traffic light in the current feature map. The output of the U-turn status recognition module indicates whether the U-turn direction is permissible in the current feature map. The output of the left turn status recognition module indicates whether the left turn direction is permissible in the current feature map. The output of the straight-ahead status recognition module indicates whether the straight-ahead direction is permissible in the current feature map. The output of the right turn status recognition module indicates whether the right turn direction is permissible in the current feature map.

[0101] The status output unit 404 is used to output the passage permission status.

[0102] In one possible implementation, the traffic signal recognition model is obtained by end-to-end training an initial recognition model, which includes an initial feature extraction module, a first initial recognition module, a second initial recognition module, a third initial recognition module, a fourth initial recognition module, and a fifth initial recognition module.

[0103] In one possible implementation, the device further includes:

[0104] The second acquisition unit is used to acquire a training dataset. The training dataset includes a first training subset, a second training subset, a third training subset, a fourth training subset, and a fifth training subset. Each training subset includes multiple samples. Each sample includes a road traffic scene map and its corresponding traffic status label and feature map. The first training subset includes multiple samples with traffic light status labels and feature map labels. The second training subset includes multiple samples with U-turn status labels and feature map labels. The third training subset includes multiple samples with left turn status labels and feature map labels. The fourth training subset includes multiple samples with straight-ahead status labels and feature map labels. The fifth training subset includes multiple samples with right turn status labels and feature map labels. The traffic light status is used to indicate whether there is a valid traffic light in the corresponding road traffic scene map.

[0105] An initialization unit is used to initialize the backbone network and five classification heads to obtain the initial feature extraction module, the first initial recognition module, the second initial recognition module, the third initial recognition module, the fourth initial recognition module, and the fifth initial recognition module;

[0106] The training unit is used to take the first to fifth training subsets as inputs to the first to fifth initial recognition modules, and to take all training subsets as inputs to the initial feature extraction module. The initial recognition model is iteratively trained using a cosine annealing learning rate decay strategy until the training cutoff condition is reached, and then the training stops, thus obtaining the image feature extraction module, the global state recognition module, the U-turn state recognition module, the left turn state recognition module, the straight-ahead state recognition module, and the right turn state recognition module.

[0107] In one possible implementation, the backbone network includes Xception with the fully connected layer and Softmax layer removed.

[0108] In one possible implementation, the classification head includes two fully connected layers and a Softmax activation function.

[0109] In one possible implementation, the traffic signal recognition model is built using the PyTorch deep learning framework.

[0110] In addition, this application embodiment also provides a traffic signal recognition device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the traffic signal recognition method as described above.

[0111] In addition, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the traffic signal recognition method as described above.

[0112] This application provides a traffic signal recognition device. First, a first acquisition unit 401 acquires a current road traffic scene image to be recognized. Then, a feature extraction unit 402 uses an image feature extraction module in the traffic signal recognition model to perform convolutional feature extraction on the current road traffic scene image to obtain a traffic scene feature map. A traffic signal recognition unit 403 uses a parallel recognition module in the traffic signal recognition model to classify and recognize the traffic scene feature map to obtain the traffic permission status. The parallel recognition module includes five parallel status recognition modules: a global status recognition module, a U-turn status recognition module, a left-turn status recognition module, a straight-ahead status recognition module, and a right-turn status recognition module, each outputting the traffic permission status for its corresponding direction. Then, a status output unit 404 outputs the traffic permission status. This application employs a unified end-to-end traffic signal recognition model, which significantly reduces computational overhead and processing time. Since only one model is used for processing, the development process is simplified, development complexity is reduced, and there is no need to optimize and adjust multiple models separately, thereby reducing debugging and maintenance costs. Furthermore, this model directly classifies and identifies the entire traffic scene, avoiding error propagation between the detection and classification stages in traditional methods, thus improving the accuracy and stability of the identification. To further enhance the system's robustness, this application designs five parallel state recognition modules, each responsible for different traffic states (such as going straight, turning left, turning right, making a U-turn, and global state). This design enables the system to better handle complex traffic scenarios and situations with multiple traffic lights, improving the reliability and adaptability of the identification.

[0113] The foregoing provides a detailed description of a traffic signal recognition method, apparatus, device, and storage medium. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0114] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0115] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A traffic signal recognition method, characterized in that, The method includes: Obtain the current road traffic scene image to be identified; The current road traffic scene map is extracted by convolutional feature extraction using the image feature extraction module in the traffic signal recognition model to obtain a traffic scene feature map; The traffic scene feature map is classified and identified using the parallel recognition module in the traffic signal recognition model to obtain the traffic permission status. The parallel recognition module includes five parallel status recognition modules: a global status recognition module, a U-turn status recognition module, a left-turn status recognition module, a straight-ahead status recognition module, and a right-turn status recognition module. The output of the global status recognition module indicates whether there is a valid traffic light in the current feature map. The output of the U-turn status recognition module indicates whether the U-turn direction is permissible in the current feature map. The output of the left-turn status recognition module indicates whether the left-turn direction is permissible in the current feature map. The output of the straight-ahead status recognition module indicates whether the straight-ahead direction is permissible in the current feature map. The output of the right-turn status recognition module indicates whether the right-turn direction is permissible in the current feature map. Output the access permission status.

2. The method according to claim 1, characterized in that, The traffic signal recognition model is obtained by end-to-end training of an initial recognition model, which includes an initial feature extraction module, a first initial recognition module, a second initial recognition module, a third initial recognition module, a fourth initial recognition module, and a fifth initial recognition module. The end-to-end training process of the traffic signal recognition model includes: Obtain a training dataset; the training dataset includes a first training subset, a second training subset, a third training subset, a fourth training subset, and a fifth training subset. Each training subset includes multiple samples, and each sample includes a road traffic scene image and its corresponding traffic status label and feature map. The first training subset includes multiple samples with traffic light status labels and feature map labels. The second training subset includes multiple samples with U-turn status labels and feature map labels. The third training subset includes multiple samples with left turn status labels and feature map labels. The fourth training subset includes multiple samples with straight-ahead status labels and feature map labels. The fifth training subset includes multiple samples with right turn status labels and feature map labels. The traffic light status is used to indicate whether there is a valid traffic light in the corresponding road traffic scene image. The backbone network and five classification heads are initialized to obtain the initial feature extraction module, the first initial recognition module, the second initial recognition module, the third initial recognition module, the fourth initial recognition module, and the fifth initial recognition module; The first to fifth training subsets are used as inputs to the first to fifth initial recognition modules, respectively, and all training subsets are used as inputs to the initial feature extraction module. The initial recognition model is iteratively trained using a cosine annealing learning rate decay strategy until the training cutoff condition is reached, thereby obtaining the image feature extraction module, the global state recognition module, the U-turn state recognition module, the left turn state recognition module, the straight-ahead state recognition module, and the right turn state recognition module.

3. The method according to claim 2, characterized in that, The backbone network includes a grouped convolutional neural network, Xception, with the fully connected layers and Softmax layers removed.

4. The method according to claim 2, characterized in that, The classification head consists of two fully connected layers and a Softmax activation function.

5. The method according to any one of claims 1 or 2, characterized in that, The traffic signal recognition model is built using the PyTorch deep learning framework.

6. A traffic signal recognition device, characterized in that, The device includes: The first acquisition unit is used to acquire the current road traffic scene map to be identified; The feature extraction unit is used to perform convolutional feature extraction on the current road traffic scene map using the image feature extraction module in the traffic signal recognition model to obtain a traffic scene feature map. A traffic signal recognition unit is used to classify and recognize the traffic scene feature map using the parallel recognition modules in the traffic signal recognition model to obtain the traffic permission status. The parallel recognition modules include five parallel status recognition modules: a global status recognition module, a U-turn status recognition module, a left-turn status recognition module, a straight-ahead status recognition module, and a right-turn status recognition module. The output of the global status recognition module indicates whether there is a valid traffic light in the current feature map. The output of the U-turn status recognition module indicates whether the U-turn direction is permissible in the current feature map. The output of the left-turn status recognition module indicates whether the left-turn direction is permissible in the current feature map. The output of the straight-ahead status recognition module indicates whether the straight-ahead direction is permissible in the current feature map. The output of the right-turn status recognition module indicates whether the right-turn direction is permissible in the current feature map. The status output unit is used to output the passage permission status.

7. The apparatus according to claim 6, characterized in that, The traffic signal recognition model is obtained by end-to-end training of an initial recognition model, which includes an initial feature extraction module, a first initial recognition module, a second initial recognition module, a third initial recognition module, a fourth initial recognition module, and a fifth initial recognition module. The device further includes: The second acquisition unit is used to acquire a training dataset. The training dataset includes a first training subset, a second training subset, a third training subset, a fourth training subset, and a fifth training subset. Each training subset includes multiple samples. Each sample includes a road traffic scene map and its corresponding traffic status label and feature map. The first training subset includes multiple samples with traffic light status labels and feature map labels. The second training subset includes multiple samples with U-turn status labels and feature map labels. The third training subset includes multiple samples with left turn status labels and feature map labels. The fourth training subset includes multiple samples with straight-ahead status labels and feature map labels. The fifth training subset includes multiple samples with right turn status labels and feature map labels. The traffic light status is used to indicate whether there is a valid traffic light in the corresponding road traffic scene map. An initialization unit is used to initialize the backbone network and five classification heads to obtain the initial feature extraction module, the first initial recognition module, the second initial recognition module, the third initial recognition module, the fourth initial recognition module, and the fifth initial recognition module; The training unit is used to take the first to fifth training subsets as inputs to the first to fifth initial recognition modules, and to take all training subsets as inputs to the initial feature extraction module. The initial recognition model is iteratively trained using a cosine annealing learning rate decay strategy until the training cutoff condition is reached, and then the training stops, thus obtaining the image feature extraction module, the global state recognition module, the U-turn state recognition module, the left turn state recognition module, the straight-ahead state recognition module, and the right turn state recognition module.

8. The apparatus according to claim 7, characterized in that, The backbone network includes Xception with the fully connected layer and Softmax layer removed.

9. A traffic signal recognition device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the traffic signal recognition method as described in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the traffic signal recognition method as described in any one of claims 1-5.

Images