Multi-task traffic light recognition method, device, electronic equipment and storage medium

By constructing a multi-task traffic light recognition model based on a traffic light image sample set and using a weighted bidirectional feature pyramid network to fuse feature data, the problems of wasted computing resources and low recognition efficiency in existing technologies are solved, and efficient multi-task traffic light recognition is achieved.

CN118587681BActive Publication Date: 2026-11-17CHINA AUTOMOTIVE INNOVATION CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410791800.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-19
Publication Date
2026-11-17
Estimated Expiration
2044-06-19

AI Technical Summary

Technical Problem

Existing multi-task traffic light recognition methods waste computational resources and have low recognition efficiency.

Method used

A pre-set multi-task traffic light recognition model trained on a traffic light image sample set is adopted, including a target backbone network module, a target feature aggregation module, and multiple target task detection heads. A weighted bidirectional feature pyramid network is used to fuse multi-scale feature data, reducing the number of network layers and target task detection heads.

Benefits of technology

By using a single model to handle multiple traffic light recognition tasks, computational resources are saved, recognition efficiency is improved, the ability to detect and recognize targets of different sizes is enhanced, and the model structure is simplified.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118587681B_ABST
    Figure CN118587681B_ABST
Patent Text Reader

Abstract

This application discloses a multi-task traffic light recognition method, device, electronic device, and storage medium. This solution achieves multiple traffic light recognition tasks by processing them at once through a single preset multi-task traffic light recognition model, eliminating the need for cascading multiple models, reducing the number of network layers, saving computational resources, and improving recognition efficiency. Simultaneously, by fusing feature data of different scales through a weighted bidirectional feature pyramid network, the model's ability to detect and recognize targets of different sizes can be enhanced. Furthermore, by fusing multi-scale feature data, it is not necessary to process each scale of feature data using a separate target task detection head, thereby reducing unnecessary target task detection heads, simplifying the model structure, and further saving computational resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a multi-task traffic light recognition method, device, electronic device, and storage medium. Background Technology

[0002] With the rapid development of autonomous driving technology, target recognition in traffic scenarios has become a research hotspot. Target recognition tasks for traffic lights typically involve multiple steps, such as determining the location of the traffic light and detecting its color.

[0003] In related technologies, existing recognition schemes for multi-task traffic light recognition integrate multiple deep learning models. Specifically, different deep learning models are built for different tasks and cascaded together, and each model needs to be trained separately during construction.

[0004] However, the aforementioned multi-task traffic light recognition method not only wastes computing resources, but also reduces recognition efficiency for devices with limited computing power. Summary of the Invention

[0005] This application aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of this application proposes a multi-task traffic light recognition method, which includes:

[0006] Acquire the target traffic light image;

[0007] The target traffic light image is input into a pre-defined multi-task traffic light recognition model for calculation, generating a recognition result corresponding to the target traffic light image. The pre-defined multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The pre-defined multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel.

[0008] In one possible implementation, the process of constructing a pre-defined multi-task traffic light recognition model includes:

[0009] Obtain a traffic light image sample set; wherein, the traffic light image sample set includes a traffic light image training set and a traffic light image validation set;

[0010] The traffic light image training set is input into the initial multi-task traffic light recognition model for training, generating an intermediate multi-task traffic light recognition model; the initial multi-task traffic light recognition model includes a backbone network module, a feature aggregation module, and multiple task detection heads;

[0011] The intermediate multi-task traffic light recognition model is validated using a traffic light image validation set, and a preset multi-task traffic light recognition model is generated.

[0012] In one possible implementation, a training set of traffic light images is input into an initial multi-task traffic light recognition model for training to generate an intermediate multi-task traffic light recognition model, including:

[0013] The traffic light image training set is input into the initial multi-task traffic light recognition model. The backbone network module in the initial multi-task traffic light recognition model extracts features from the traffic light image training set to generate multi-scale feature data.

[0014] The multi-scale feature data is fused through the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features;

[0015] Based on the fusion features corresponding to each task detection head, a predicted recognition result is generated; whereby each predicted recognition result corresponds to each traffic light recognition task.

[0016] Based on the prediction and recognition results and label information, the parameters of the initial multi-task traffic light recognition model are adjusted to generate an intermediate multi-task traffic light recognition model.

[0017] In one possible implementation, the backbone network module includes multiple convolutional sub-modules, multiple convolutional layer activation function sub-modules, and a spatial pyramid pooling fast layer sub-module. It extracts features from the traffic light image training set through the backbone network module in the initial multi-task traffic light recognition model to generate multi-scale feature data, including:

[0018] By using the convolutional submodules, activation function submodules of each convolutional layer, and spatial pyramid pooling fast layer submodule in the backbone network module, feature extraction is performed on the traffic light image training set to generate multi-scale feature data.

[0019] In one possible implementation, the intermediate multi-task traffic light recognition model is validated using a traffic light image validation set to generate a preset multi-task traffic light recognition model, including:

[0020] The traffic light image validation set is input into the intermediate multi-task traffic light recognition model for validation, and the evaluation parameters and loss function values ​​of each current model are calculated; wherein, the evaluation parameters and loss function values ​​of each current model correspond to each traffic light recognition task.

[0021] If the current model evaluation parameters and the current loss function values ​​meet the preset conditions, then the intermediate multi-task traffic light recognition model will be used as the preset multi-task traffic light recognition model.

[0022] If the current model evaluation parameters and / or the current loss function values ​​do not meet the preset conditions, the steps of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training and generating an intermediate multi-task traffic light recognition model are repeated until the new current model evaluation parameters and the new current loss function values ​​meet the preset conditions, thus generating a preset multi-task traffic light recognition model.

[0023] In one possible implementation, the traffic light image training set includes a first training set, a second training set, and a third training set, each corresponding to a different traffic light recognition task. The process of inputting the traffic light image training set into an initial multi-task traffic light recognition model for training to generate an intermediate multi-task traffic light recognition model further includes:

[0024] The traffic light image training set is input into the initial multi-task traffic light recognition model, and then trained using the first training set, the second training set, and the third training set to generate an intermediate multi-task traffic light recognition model.

[0025] In one possible implementation, acquiring a traffic light image sample set includes:

[0026] Acquire multiple raw traffic light images;

[0027] For each original traffic light image, the targets in the original traffic light images are labeled to generate label information; the label information includes the bounding box and the attribute information corresponding to the target.

[0028] The original traffic light images are preprocessed to obtain multiple traffic light image samples. Based on the multiple traffic light image samples and label information, a traffic light image sample set is generated. The preprocessing operations include size transformation and pixel value normalization.

[0029] A second aspect of this application discloses a multi-task traffic light recognition device, the device comprising:

[0030] The acquisition module is used to acquire images of the target traffic lights;

[0031] The generation module is used to input the target traffic light image into a preset multi-task traffic light recognition model for calculation, and generate a recognition result corresponding to the target traffic light image. The preset multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel.

[0032] In one possible implementation, the above-described multi-task traffic light recognition device is further used for:

[0033] Obtain a traffic light image sample set; wherein, the traffic light image sample set includes a traffic light image training set and a traffic light image validation set;

[0034] The traffic light image training set is input into the initial multi-task traffic light recognition model for training, generating an intermediate multi-task traffic light recognition model; the initial multi-task traffic light recognition model includes a backbone network module, a feature aggregation module, and multiple task detection heads;

[0035] The intermediate multi-task traffic light recognition model is validated using a traffic light image validation set, and a preset multi-task traffic light recognition model is generated.

[0036] In one possible implementation, the above-described multi-task traffic light recognition device is further used for:

[0037] The traffic light image training set is input into the initial multi-task traffic light recognition model. The backbone network module in the initial multi-task traffic light recognition model extracts features from the traffic light image training set to generate multi-scale feature data.

[0038] The multi-scale feature data is fused through the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features;

[0039] Based on the fusion features corresponding to each task detection head, a predicted recognition result is generated; whereby each predicted recognition result corresponds to each traffic light recognition task.

[0040] Based on the prediction and recognition results and label information, the parameters of the initial multi-task traffic light recognition model are adjusted to generate an intermediate multi-task traffic light recognition model.

[0041] In one possible implementation, the backbone network module includes multiple convolutional sub-modules, multiple convolutional layer activation function sub-modules, and a spatial pyramid pooling fast layer sub-module. The aforementioned multi-task traffic light recognition device is further used for:

[0042] By using the convolutional submodules, activation function submodules of each convolutional layer, and spatial pyramid pooling fast layer submodule in the backbone network module, feature extraction is performed on the traffic light image training set to generate multi-scale feature data.

[0043] In one possible implementation, the above-described multi-task traffic light recognition device is further used for:

[0044] The traffic light image validation set is input into the intermediate multi-task traffic light recognition model for validation, and the evaluation parameters and loss function values ​​of each current model are calculated; wherein, the evaluation parameters and loss function values ​​of each current model correspond to each traffic light recognition task.

[0045] If the current model evaluation parameters and the current loss function values ​​meet the preset conditions, then the intermediate multi-task traffic light recognition model will be used as the preset multi-task traffic light recognition model.

[0046] If the current model evaluation parameters and / or the current loss function values ​​do not meet the preset conditions, the steps of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training and generating an intermediate multi-task traffic light recognition model are repeated until the new current model evaluation parameters and the new current loss function values ​​meet the preset conditions, thus generating a preset multi-task traffic light recognition model.

[0047] In one possible implementation, the traffic light image training set includes a first training set, a second training set, and a third training set, each corresponding to a different traffic light recognition task; the aforementioned multi-task traffic light recognition device is further used for:

[0048] The traffic light image training set is input into the initial multi-task traffic light recognition model, and then trained using the first training set, the second training set, and the third training set to generate an intermediate multi-task traffic light recognition model.

[0049] In one possible implementation, the above-described multi-task traffic light recognition device is further used for:

[0050] Acquire multiple raw traffic light images;

[0051] For each original traffic light image, the targets in the original traffic light images are labeled to generate label information; the label information includes the bounding box and the attribute information corresponding to the target.

[0052] The original traffic light images are preprocessed to obtain multiple traffic light image samples. Based on the multiple traffic light image samples and label information, a traffic light image sample set is generated. The preprocessing operations include size transformation and pixel value normalization.

[0053] A third aspect of this application provides an electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the multi-task traffic light recognition method as described in the first aspect.

[0054] The fourth aspect of this application provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the multi-task traffic light recognition method as described in the first aspect.

[0055] The embodiments of this application have the following beneficial effects:

[0056] The multi-task traffic light recognition method, apparatus, electronic device, and storage medium provided in this application embodiment include: acquiring a target traffic light image, inputting the target traffic light image into a preset multi-task traffic light recognition model for calculation, and generating a recognition result corresponding to the target traffic light image. The preset multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel. This solution achieves multiple traffic light recognition tasks by processing them in a single, pre-defined multi-task traffic light recognition model, eliminating the need for cascading multiple models, reducing network layers, saving computational resources, and improving recognition efficiency. Simultaneously, by fusing feature data at different scales through a weighted bidirectional feature pyramid network, the model's ability to detect and recognize targets of varying sizes can be enhanced. Furthermore, by fusing multi-scale feature data, it is not necessary to process each scale of feature data using a separate target task detection head, thereby reducing unnecessary target task detection heads, simplifying the model structure, and further conserving computational resources. Attached Figure Description

[0057] Figure 1 A structural block diagram of a computer device provided in an embodiment of this application;

[0058] Figure 2 A flowchart illustrating the steps of a multi-task traffic light recognition method provided in this application embodiment;

[0059] Figure 3 A flowchart illustrating the steps for constructing a preset multi-task traffic light recognition model, as provided in this application embodiment;

[0060] Figure 4 A flowchart illustrating the steps for obtaining a traffic light image sample set according to an embodiment of this application;

[0061] Figure 5 A schematic diagram of a label box provided in an embodiment of this application;

[0062] Figure 6 A flowchart illustrating the steps for generating an intermediate multi-task traffic light recognition model, as provided in this application embodiment;

[0063] Figure 7 A schematic diagram of the overall structure of a model provided in an embodiment of this application;

[0064] Figure 8 A flowchart illustrating the steps for verifying an intermediate multi-task traffic light recognition model, as provided in this application embodiment;

[0065] Figure 9 This is a structural block diagram of a multi-task traffic light recognition device provided in an embodiment of this application. Detailed Implementation

[0066] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0067] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0068] The multi-task traffic light recognition method provided in this application can be applied to computer equipment (electronic devices). The computer equipment can be a server or a terminal. The server can be a single server or a server cluster composed of multiple servers. This application does not specifically limit this. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices.

[0069] Taking a computer device as an example, Figure 1 A block diagram of a server is shown, such as Figure 1As shown, the server may include a processor and memory connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. When the computer program is executed by the processor, it implements a multi-tasking traffic light recognition method.

[0070] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the server to which the present application is applied. Optionally, the server may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0071] It should be noted that the execution subject of the embodiments of this application can be a computer device or a multi-tasking traffic light recognition device. The following method embodiments will be described with a computer device as the execution subject.

[0072] Figure 2 This is a flowchart illustrating the steps of a multi-task traffic light recognition method provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0073] Step 202: Obtain the target traffic light image.

[0074] The target recognition task for traffic lights typically involves multiple steps, such as determining the location of the traffic light and detecting its color. When performing multi-task traffic light recognition, it is necessary to first acquire an image of the target traffic light, which is an image containing the traffic light as the target object. This image can be acquired using an image acquisition device, and multiple images can be acquired. Furthermore, the target traffic light image can be an image acquired in real time.

[0075] In some optional embodiments, after the original image is acquired by the image acquisition device, preprocessing operations can be performed on the original image to obtain the target traffic light image. These preprocessing operations may include size transformation processing, pixel value normalization processing, or other types of preprocessing operations; this application embodiment does not specifically limit these operations.

[0076] Step 204: Input the target traffic light image into the preset multi-task traffic light recognition model for calculation, and generate the recognition result corresponding to the target traffic light image.

[0077] The preset multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel.

[0078] The aforementioned preset multi-task traffic light recognition model needs to be pre-built. In some optional embodiments, such as Figure 3 As shown, Figure 3 A flowchart illustrating the steps for constructing a preset multi-task traffic light recognition model, as provided in this application embodiment, includes:

[0079] Step 302: Obtain a sample set of traffic light images.

[0080] The traffic light image sample set includes a traffic light image training set and a traffic light image validation set. When obtaining the traffic light image sample set, optionally, as follows: Figure 4 As shown, Figure 4 A flowchart illustrating the steps for obtaining a traffic light image sample set according to an embodiment of this application includes:

[0081] Step 402: Obtain multiple original traffic light images.

[0082] Step 404: For each original traffic light image, the targets in the original traffic light images are labeled to generate label information.

[0083] Step 406: Perform preprocessing operations on the original traffic light images to obtain multiple traffic light image samples, and generate a traffic light image sample set based on the multiple traffic light image samples and label information.

[0084] The original traffic light images can be collected in various scenarios. For example, various scenarios can include different road conditions, different lighting conditions, different weather conditions, etc. Of course, other scenarios can also be included. This application embodiment does not specifically limit them.

[0085] After acquiring the original traffic light image, the targets in the image can be labeled to generate tag information. This tag information can include bounding boxes and corresponding attribute information for the target. The bounding box describes the target's position and size, and can be represented as (x, y, w, h), where x and y are the coordinates of the target's center point, and w and h are the target's width and height, respectively.

[0086] Different targets correspond to different attribute information. Optionally, the annotation box can include a first annotation box and a second annotation box. The first annotation box can annotate the illuminated target, and its corresponding attribute information can include red, yellow, green, left turn, straight ahead, right turn, U-turn, pedestrian, countdown, bicycle, circular light, unknown, etc. The second annotation box can annotate the lightbox of the traffic status light. Since traffic status lights can include counter lights and signal lights, their corresponding attribute information can include the front, side, and back of the counter light, and the front, side, and back of the signal light. For example... Figure 5 As shown, Figure 5 This is a schematic diagram of a labeling frame provided in an embodiment of this application. The labeling frame A for the illuminated target is the first labeling frame, and the labeling frame B for the lightbox for the traffic status light is the second labeling frame. Since the illuminated target is also located inside the lightbox, the size of the first labeling frame is larger than that of the second labeling frame.

[0087] In addition, preprocessing is required to obtain multiple traffic light image samples from the original traffic light image. Preprocessing operations include, but are not limited to, resizing and pixel value normalization. For example, during resizing, the target can be cropped from the original traffic light image, and the cropped image size can be 128*128.

[0088] Next, the traffic light image samples and their corresponding label information need to be categorized. For example, all color attribute information in the attribute information of the illuminated target and its corresponding traffic light image sample can be grouped into one category, and the type attribute information in the attribute information of the illuminated target and its corresponding traffic light image sample can be grouped into another category. The color attribute information can include red, yellow, and green, while the type attribute information can include the attribute information of the other nine illuminated targets. Finally, a traffic light image sample set can be generated based on multiple traffic light image samples and their corresponding label information.

[0089] The obtained traffic light image sample set can also be divided into a traffic light image training set and a traffic light image verification set according to a preset ratio. Usually, the amount of data in the traffic light image training set needs to be greater than the amount of data in the traffic light image verification set. For example, the preset ratio can be 9:1.

[0090] Step 304: Input the traffic light image training set into the initial multi-task traffic light recognition model for training, and generate an intermediate multi-task traffic light recognition model.

[0091] The initial multi-task traffic light recognition model can also include a backbone network module, a feature aggregation module, and multiple task detection heads. The weight parameters corresponding to these modules are also initialized. Therefore, the traffic light image training set can be input into the initial multi-task traffic light recognition model for training, generating an intermediate multi-task traffic light recognition model.

[0092] In some alternative embodiments, all traffic light image training sets can be directly input into the initial multi-task traffic light recognition model, and then the entire traffic light image training set can be used for training for each traffic light recognition task.

[0093] In other alternative embodiments, the traffic light image training set can be further divided into multiple parts based on the label information. Optionally, the traffic light image training set may include a first training set, a second training set, and a third training set, each corresponding to a different traffic light recognition task.

[0094] For example, if there are three traffic light recognition tasks: recognizing the color attribute information of the illuminated target, recognizing the type attribute information of the illuminated target, and recognizing the attribute information of the lightbox of the traffic light, the training set can be divided into three parts: a first training set train1, a second training set train2, and a third training set train3. Here, train1 can be obtained based on the color attribute information, train2 can be obtained based on the type attribute information, and train3 can be obtained based on the attribute information of the lightbox of the traffic light. Furthermore, each training set is used to train the model for one specific traffic light recognition task. Correspondingly, the traffic light image validation set can also be divided into three parts in the same way: a first validation set val1, a second validation set val2, and a third validation set val3.

[0095] Taking the example of dividing the traffic light image training set into multiple parts for model training, when the traffic light image training set is input into the initial multi-task traffic light recognition model for training and generating the intermediate multi-task traffic light recognition model, it also includes inputting the traffic light image training set into the initial multi-task traffic light recognition model and training it through the first training set, the second training set and the third training set respectively to generate the intermediate multi-task traffic light recognition model.

[0096] In some alternative embodiments, such as Figure 6 As shown, Figure 6 A flowchart illustrating the steps for generating an intermediate multi-task traffic light recognition model, as provided in this application embodiment, includes:

[0097] Step 602: Input the traffic light image training set into the initial multi-task traffic light recognition model, and extract features from the traffic light image training set through the backbone network module in the initial multi-task traffic light recognition model to generate multi-scale feature data.

[0098] Step 604: The multi-scale feature data is fused using the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features.

[0099] Step 606: Generate prediction and recognition results based on the fusion features corresponding to each task detection head using the detection heads of each task.

[0100] Step 608: Adjust the parameters of the initial multi-task traffic light recognition model based on the prediction and recognition results and label information to generate an intermediate multi-task traffic light recognition model.

[0101] The backbone network module includes multiple convolutional sub-modules, multiple convolutional layer activation function sub-modules, and a spatial pyramid pooling fast layer sub-module. For example, as shown... Figure 7 As shown, Figure 7 This is a schematic diagram of the overall structure of a model provided in an embodiment of this application. The backbone network module can include five convolutional sub-modules (Conv), four convolutional layer activation function sub-modules (C2F), and one spatial pyramid pooling-fast (SPPF) sub-module.

[0102] Therefore, feature extraction can be performed on the first, second, and third training sets of the traffic light image training set through the convolutional sub-modules, convolutional layer activation function sub-modules, and spatial pyramid pooling fast layer sub-module in the backbone network module, generating corresponding multi-scale feature data. Specifically, each convolutional sub-module in the backbone network module can use a convolutional kernel with a stride of 2 for downsampling to reduce the size of the feature map and increase the number of channels. The convolutional layer activation function sub-module C2F has fewer parameters and better feature extraction capabilities, which can be used to enhance the nonlinear expressive power of the model. The spatial pyramid pooling fast layer sub-module, while reducing the number of parameters, can convert feature maps of arbitrary size into feature vectors of fixed size. Finally, after processing the traffic light image training set by each sub-module, multi-scale feature data can be output. By adopting this backbone network module structure, the efficiency and performance of feature extraction can be improved.

[0103] Next, the multi-scale feature data can be fused using the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features. For an example, please refer to [link to example]. Figure 7The target feature aggregation module, also known as Neck, is used for feature fusion after the backbone network module extracts feature data. This module can include three branches, each corresponding to a different traffic light recognition task. Each branch can be trained using a specific training set. Figure 7 The data flow indicated by the middle arrow shows that the multi-scale feature data generated by the backbone network module flows into the three branches respectively.

[0104] In addition, each branch consists of a convolutional layer submodule Conv, an upsampling layer Upsample, a weighted bidirectional feature pyramid network (BiFPN), and a convolutional layer activation function submodule C2F.

[0105] Therefore, during feature fusion, since multi-scale feature data can be obtained in the backbone network module, the feature data output by the Spatial Pyramid Pooling Fast Layer (SPPF) submodule is sequentially processed by the Convolutional Layer (Conv) and Upsample layer in the Neck target feature aggregation module. This processed data is then fused with the feature data output by the third convolutional layer activation function (C2F) submodule in the Backbone through a weighted bidirectional feature pyramid network (BiFPN) to obtain the first intermediate fused data. This first intermediate fused data is then sequentially processed by the Convolutional Layer Activation Function (C2F), Conv, and Upsample layer in the Neck, and then fused with the feature data output by the second convolutional layer activation function (C2F) submodule in the Backbone through a weighted bidirectional feature pyramid network (BiFPN) to obtain the second intermediate fused data. Finally, this second intermediate fused data is processed by a convolutional layer activation function (C2F) submodule in the Neck to obtain the final fused features to be input into the task detection head. In this embodiment, the weighted bidirectional feature pyramid network (BiFPN) is used to fuse feature data of different scales, which can enhance the model's ability to detect and recognize targets of different sizes. In addition, by fusing multi-scale feature data, it is not necessary to use a separate task detection head for each scale of feature data, thereby reducing unnecessary task detection heads, simplifying the model structure, and saving computational resources.

[0106] It should be noted that a similar structure and processing procedure can be used for each branch, and will not be elaborated further here. Additionally, the order of the C2F submodules of the convolutional layers in the Backbone can be found in [reference needed]. Figure 7In the data flow direction indicated by the arrow, from top to bottom, they are the first convolutional layer activation function submodule C2F to the fourth convolutional layer activation function submodule C2F. The convolutional layer submodules can also be divided in the same order.

[0107] After obtaining multiple fusion features, these features can be input into each task detection head for processing. Each task detection head generates a predicted recognition result based on the fusion features corresponding to that head, with each predicted recognition result corresponding to a specific traffic light recognition task. Furthermore, different traffic light recognition tasks can correspond to different numbers of data channels within each task detection head.

[0108] For example, please continue to refer to Figure 7 The task detection head is called the Head, which can include three: DeCet1, DeCet2, and DeCet3. DeCet1 can correspond to the task of recognizing the color attribute information of the illuminated target, DeCet2 can correspond to the task of recognizing the type attribute information of the illuminated target, and DeCet3 can correspond to the task of recognizing the attribute information of the light box of the traffic status light.

[0109] Therefore, the number of output channels of DeCet1 can be 3*(4+1+3)=24. Here, the first number 3 on the left side of the equation represents three sets of anchor boxes of different scales, the second number 4 represents the center point coordinates, width and height of the target in the output prediction and recognition result, the third number 1 represents the confidence of the predicted box in the output prediction and recognition result, and the fourth number 3 represents the confidence of each predicted label information in the output prediction and recognition result. Since there are three types of color attribute information, the fourth number here is 3.

[0110] The number of output channels for Detcet2 is 3*(4+1+9)=42. The first three numbers on the left side of the equation are the same as described above. The fourth number, 9, represents the confidence level of each predicted label information in the output prediction and recognition results. Since there are nine types of type attribute information, the fourth number here is 9.

[0111] The number of output channels of Detcet3 is 3*(4+1+6)=33. Similarly, the first three numbers on the left side of the equation are the same as described above. The fourth number 6 represents the confidence level of each predicted label information in the output prediction recognition result. Since there are six types of attribute information for the light box of the traffic status light, the fourth number here is 6.

[0112] Finally, the parameters of the initial multi-task traffic light recognition model can be adjusted based on the prediction and recognition results and label information to generate an intermediate multi-task traffic light recognition model. Optionally, during model training, the model makes predictions based on the anchor boxes, thereby outputting prediction and recognition results. These prediction and recognition results include prediction boxes and prediction label information. These results are then compared with pre-labeled label information, and finally, gradient backpropagation is performed to adjust the model parameters. An intermediate multi-task traffic light recognition model is generated based on the finally adjusted parameters.

[0113] Step 306: Verify the intermediate multi-task traffic light recognition model using the traffic light image verification set, and generate a preset multi-task traffic light recognition model.

[0114] After obtaining the intermediate multi-task traffic light recognition model, the intermediate multi-task traffic light recognition model can be verified using a traffic light image verification set to generate a preset multi-task traffic light recognition model.

[0115] In some alternative embodiments, such as Figure 8 As shown, Figure 8 A flowchart illustrating the steps for verifying an intermediate multi-task traffic light recognition model, as provided in this application embodiment, includes:

[0116] Step 802: Input the traffic light image verification set into the intermediate multi-task traffic light recognition model for verification, and calculate the evaluation parameters of each current model and the value of each current loss function.

[0117] Step 804: If the current model evaluation parameters and the current loss function values ​​meet the preset conditions, then the intermediate multi-task traffic light recognition model is used as the preset multi-task traffic light recognition model.

[0118] Step 806: If the current model evaluation parameters and / or the current loss function values ​​do not meet the preset conditions, repeat the steps of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training and generating an intermediate multi-task traffic light recognition model until the new current model evaluation parameters and the new current loss function values ​​meet the preset conditions, and generate a preset multi-task traffic light recognition model.

[0119] The traffic light image validation set can include a first validation set, a second validation set, and a third validation set, with the same number of partitions as the training set. This validation set can then be input into the intermediate multi-task traffic light recognition model for validation, calculating the current model evaluation parameters and current loss function values. Each current model evaluation parameter and current loss function value corresponds to a specific traffic light recognition task; that is, each branch can calculate its own current model evaluation parameters and current loss function values. Optionally, the current model evaluation parameters can be the mean average precision (mAP). The specific calculation process for the current model evaluation parameters and current loss function values ​​can refer to existing calculation procedures and will not be elaborated here.

[0120] If the current model evaluation parameters and the current loss function values ​​meet the preset conditions, the intermediate multi-task traffic light recognition model is used as the preset multi-task traffic light recognition model. If the current model evaluation parameters and / or the current loss function values ​​do not meet the preset conditions, the steps of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training and generating an intermediate multi-task traffic light recognition model are repeated until the new current model evaluation parameters and the new current loss function values ​​meet the preset conditions, thus generating the preset multi-task traffic light recognition model. The final preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads.

[0121] The preset conditions can be that the current model evaluation parameters reach preset values ​​and the current loss function value decreases and stabilizes. If these preset conditions are met, it indicates that the model has converged, training can end, and the model parameters can be frozen. The intermediate multi-task traffic light recognition model can then be used as the preset multi-task traffic light recognition model.

[0122] If the preset conditions are not met, optionally, if none of the three branches meet the preset conditions, the entire training process needs to be re-run.

[0123] If one or two branches do not meet the preset conditions, the model parameters corresponding to the branches that meet the preset conditions need to be frozen before retraining the other branches. It should be noted that in this case, you can choose to use all traffic light image training sets as input data for retraining, or you can use a portion of the traffic light image training set as input data for retraining, that is, only use the training set required by the branches that do not meet the preset conditions as input data for retraining.

[0124] In this embodiment, by using a traffic light image validation set to validate the intermediate multi-task traffic light recognition model, the performance of the generated preset multi-task traffic light recognition model can be further guaranteed, and the accuracy of multi-task traffic light recognition can be improved. In addition, each branch can independently calculate the current model evaluation parameters and the current loss function value, which improves the efficiency of model training and validation.

[0125] The multi-task traffic light recognition method provided in this application includes: acquiring a target traffic light image, inputting the target traffic light image into a preset multi-task traffic light recognition model for calculation, and generating a recognition result corresponding to the target traffic light image. The preset multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel. This solution achieves multiple traffic light recognition tasks by processing them in a single, pre-defined multi-task traffic light recognition model, eliminating the need for cascading multiple models, reducing network layers, saving computational resources, and improving recognition efficiency. Simultaneously, by fusing feature data at different scales through a weighted bidirectional feature pyramid network, the model's ability to detect and recognize targets of varying sizes can be enhanced. Furthermore, by fusing multi-scale feature data, it is not necessary to use a separate target task detection head for each scale of feature data, thus reducing unnecessary target task detection heads, simplifying the model structure, and further saving computational resources.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0127] Figure 9 This is a structural block diagram of a multi-task traffic light recognition device provided in an embodiment of this application.

[0128] like Figure 9 As shown, the multi-task traffic light recognition device 900 includes:

[0129] The acquisition module 902 is used to acquire the target traffic light image.

[0130] The generation module 904 is used to input the target traffic light image into a preset multi-task traffic light recognition model for calculation, and generate a recognition result corresponding to the target traffic light image. The preset multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel.

[0131] Regarding the apparatus in the above embodiments, the specific methods by which each module performs its operations have been described in detail in the embodiments related to the method, and will not be elaborated upon here. Each module in the above-described multi-task traffic light recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations of each module.

[0132] In one embodiment of this application, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0133] Acquire the target traffic light image;

[0134] The target traffic light image is input into a pre-defined multi-task traffic light recognition model for calculation, generating a recognition result corresponding to the target traffic light image. The pre-defined multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The pre-defined multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel.

[0135] In one embodiment of this application, the processor further performs the following steps when executing the computer program:

[0136] Obtain a traffic light image sample set; wherein, the traffic light image sample set includes a traffic light image training set and a traffic light image validation set;

[0137] The traffic light image training set is input into the initial multi-task traffic light recognition model for training, generating an intermediate multi-task traffic light recognition model; the initial multi-task traffic light recognition model includes a backbone network module, a feature aggregation module, and multiple task detection heads;

[0138] The intermediate multi-task traffic light recognition model is validated using a traffic light image validation set, and a preset multi-task traffic light recognition model is generated.

[0139] In one embodiment of this application, the processor further performs the following steps when executing the computer program:

[0140] The traffic light image training set is input into the initial multi-task traffic light recognition model. The backbone network module in the initial multi-task traffic light recognition model extracts features from the traffic light image training set to generate multi-scale feature data.

[0141] The multi-scale feature data is fused through the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features;

[0142] Based on the fusion features corresponding to each task detection head, a predicted recognition result is generated; whereby each predicted recognition result corresponds to each traffic light recognition task.

[0143] Based on the prediction and recognition results and label information, the parameters of the initial multi-task traffic light recognition model are adjusted to generate an intermediate multi-task traffic light recognition model.

[0144] In one embodiment of this application, the backbone network module includes multiple convolutional sub-modules, multiple convolutional layer activation function sub-modules, and a spatial pyramid pooling fast layer sub-module. When the processor executes the computer program, it further implements the following steps:

[0145] By using the convolutional submodules, activation function submodules of each convolutional layer, and spatial pyramid pooling fast layer submodule in the backbone network module, feature extraction is performed on the traffic light image training set to generate multi-scale feature data.

[0146] In one embodiment of this application, an intermediate multi-task traffic light recognition model is validated using a traffic light image validation set to generate a preset multi-task traffic light recognition model, including:

[0147] The traffic light image validation set is input into the intermediate multi-task traffic light recognition model for validation, and the evaluation parameters and loss function values ​​of each current model are calculated; wherein, the evaluation parameters and loss function values ​​of each current model correspond to each traffic light recognition task.

[0148] If the current model evaluation parameters and the current loss function values ​​meet the preset conditions, then the intermediate multi-task traffic light recognition model will be used as the preset multi-task traffic light recognition model.

[0149] If the current model evaluation parameters and / or the current loss function values ​​do not meet the preset conditions, the steps of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training and generating an intermediate multi-task traffic light recognition model are repeated until the new current model evaluation parameters and the new current loss function values ​​meet the preset conditions, thus generating a preset multi-task traffic light recognition model.

[0150] In one embodiment of this application, the traffic light image training set includes a first training set, a second training set, and a third training set, each corresponding to a different traffic light recognition task; the processor, when executing the computer program, also implements the following steps:

[0151] The traffic light image training set is input into the initial multi-task traffic light recognition model, and then trained using the first training set, the second training set, and the third training set to generate an intermediate multi-task traffic light recognition model.

[0152] In one embodiment of this application, the processor further performs the following steps when executing the computer program:

[0153] Acquire multiple raw traffic light images;

[0154] For each original traffic light image, the targets in the original traffic light images are labeled to generate label information; the label information includes the bounding box and the attribute information corresponding to the target.

[0155] The original traffic light images are preprocessed to obtain multiple traffic light image samples. Based on the multiple traffic light image samples and label information, a traffic light image sample set is generated. The preprocessing operations include size transformation and pixel value normalization.

[0156] The computer device provided in the above embodiments has similar implementation principles and technical effects to the above method embodiments, and will not be described again here.

[0157] In one embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it performs the following steps:

[0158] Acquire the target traffic light image;

[0159] The target traffic light image is input into a pre-defined multi-task traffic light recognition model for calculation, generating a recognition result corresponding to the target traffic light image. The pre-defined multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The pre-defined multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel.

[0160] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps:

[0161] Obtain a traffic light image sample set; wherein, the traffic light image sample set includes a traffic light image training set and a traffic light image validation set;

[0162] The traffic light image training set is input into the initial multi-task traffic light recognition model for training, generating an intermediate multi-task traffic light recognition model; the initial multi-task traffic light recognition model includes a backbone network module, a feature aggregation module, and multiple task detection heads;

[0163] The intermediate multi-task traffic light recognition model is validated using a traffic light image validation set, and a preset multi-task traffic light recognition model is generated.

[0164] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps:

[0165] The traffic light image training set is input into the initial multi-task traffic light recognition model. The backbone network module in the initial multi-task traffic light recognition model extracts features from the traffic light image training set to generate multi-scale feature data.

[0166] The multi-scale feature data is fused through the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features;

[0167] Based on the fusion features corresponding to each task detection head, a predicted recognition result is generated; whereby each predicted recognition result corresponds to each traffic light recognition task.

[0168] Based on the prediction and recognition results and label information, the parameters of the initial multi-task traffic light recognition model are adjusted to generate an intermediate multi-task traffic light recognition model.

[0169] In one embodiment of this application, the backbone network module includes multiple convolutional sub-modules, multiple convolutional layer activation function sub-modules, and a spatial pyramid pooling fast layer sub-module. When the computer program is executed by the processor, it further implements the following steps:

[0170] By using the convolutional submodules, activation function submodules of each convolutional layer, and spatial pyramid pooling fast layer submodule in the backbone network module, feature extraction is performed on the traffic light image training set to generate multi-scale feature data.

[0171] In one embodiment of this application, an intermediate multi-task traffic light recognition model is validated using a traffic light image validation set to generate a preset multi-task traffic light recognition model, including:

[0172] The traffic light image validation set is input into the intermediate multi-task traffic light recognition model for validation, and the evaluation parameters and loss function values ​​of each current model are calculated; wherein, the evaluation parameters and loss function values ​​of each current model correspond to each traffic light recognition task.

[0173] If the current model evaluation parameters and the current loss function values ​​meet the preset conditions, then the intermediate multi-task traffic light recognition model will be used as the preset multi-task traffic light recognition model.

[0174] If the current model evaluation parameters and / or the current loss function values ​​do not meet the preset conditions, the steps of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training and generating an intermediate multi-task traffic light recognition model are repeated until the new current model evaluation parameters and the new current loss function values ​​meet the preset conditions, thus generating a preset multi-task traffic light recognition model.

[0175] In one embodiment of this application, the traffic light image training set includes a first training set, a second training set, and a third training set, each corresponding to a different traffic light recognition task; when the computer program is executed by the processor, it further implements the following steps:

[0176] The traffic light image training set is input into the initial multi-task traffic light recognition model, and then trained using the first training set, the second training set, and the third training set to generate an intermediate multi-task traffic light recognition model.

[0177] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps:

[0178] Acquire multiple raw traffic light images;

[0179] For each original traffic light image, the targets in the original traffic light images are labeled to generate label information; the label information includes the bounding box and the attribute information corresponding to the target.

[0180] The original traffic light images are preprocessed to obtain multiple traffic light image samples. Based on the multiple traffic light image samples and label information, a traffic light image sample set is generated. The preprocessing operations include size transformation and pixel value normalization.

[0181] The computer storage medium provided in the above embodiments has similar implementation principles and technical effects to the above method embodiments, and will not be described again here.

[0182] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0183] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered as the area recorded in this specification.

[0184] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of this application's patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multi-task traffic light recognition method, characterized in that, The method includes: Acquire the target traffic light image; The target traffic light image is input into a preset multi-task traffic light recognition model for calculation, generating a recognition result corresponding to the target traffic light image. The preset multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel. The construction process of the preset multi-task traffic light recognition model includes: Obtain the traffic light image sample set; wherein, the traffic light image sample set includes a traffic light image training set and a traffic light image verification set; The traffic light image training set is input into the initial multi-task traffic light recognition model for training, generating an intermediate multi-task traffic light recognition model, including: The traffic light image training set is input into the initial multi-task traffic light recognition model, and the backbone network module in the initial multi-task traffic light recognition model extracts features from the traffic light image training set to generate multi-scale feature data. The multi-scale feature data is fused using the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features; Using each of the task detection heads, a predicted recognition result is generated based on the fusion features corresponding to each of the task detection heads; wherein each of the predicted recognition results corresponds to each of the traffic light recognition tasks; Based on the predicted recognition results and the label information, the parameters of the initial multi-task traffic light recognition model are adjusted to generate the intermediate multi-task traffic light recognition model; wherein, the initial multi-task traffic light recognition model includes a backbone network module, a feature aggregation module, and multiple task detection heads; The intermediate multi-task traffic light recognition model is validated using the traffic light image validation set to generate the preset multi-task traffic light recognition model.

2. The method according to claim 1, characterized in that, The backbone network module includes multiple convolutional sub-modules, multiple convolutional layer activation function sub-modules, and a spatial pyramid pooling fast layer sub-module. The feature extraction of the traffic light image training set through the backbone network module in the initial multi-task traffic light recognition model generates multi-scale feature data, including: The traffic light image training set is used to extract features through the convolutional sub-modules, convolutional layer activation function sub-modules, and spatial pyramid pooling fast layer sub-module in the backbone network module to generate the multi-scale feature data.

3. The method according to claim 1 or 2, characterized in that, The step of validating the intermediate multi-task traffic light recognition model using the traffic light image verification set to generate the preset multi-task traffic light recognition model includes: The traffic light image verification set is input into the intermediate multi-task traffic light recognition model for verification, and the evaluation parameters of each current model and the current loss function value are calculated; wherein, each of the current model evaluation parameters and the current loss function value corresponds to each of the traffic light recognition tasks; If the current model evaluation parameters and each of the current loss function values ​​meet the preset conditions, then the intermediate multi-task traffic light recognition model is used as the preset multi-task traffic light recognition model. If the current model evaluation parameters and / or the current loss function values ​​do not meet the preset conditions, the step of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training and generating an intermediate multi-task traffic light recognition model is repeated until the new current model evaluation parameters and the new current loss function values ​​meet the preset conditions, and the preset multi-task traffic light recognition model is generated.

4. The method according to claim 1 or 2, characterized in that, The traffic light image training set includes a first training set, a second training set, and a third training set, which correspond to different traffic light recognition tasks. The step of inputting the traffic light image training set into the initial multi-task traffic light recognition model for training to generate an intermediate multi-task traffic light recognition model further includes: The traffic light image training set is input into the initial multi-task traffic light recognition model, and the model is trained using the first training set, the second training set, and the third training set to generate the intermediate multi-task traffic light recognition model.

5. The method according to claim 1 or 2, characterized in that, The step of obtaining the traffic light image sample set includes: Acquire multiple raw traffic light images; For each of the original traffic light images, the targets in the original traffic light images are labeled to generate the label information; wherein, the label information includes a bounding box and attribute information corresponding to the target; The original traffic light image is preprocessed to obtain multiple traffic light image samples, and the traffic light image sample set is generated based on the multiple traffic light image samples and the label information; wherein, the preprocessing operation includes size transformation processing and pixel value normalization processing.

6. A multi-task traffic light recognition device, characterized in that, The device includes: The acquisition module is used to acquire images of the target traffic lights; A generation module is used to input the target traffic light image into a preset multi-task traffic light recognition model for calculation, generating a recognition result corresponding to the target traffic light image. The preset multi-task traffic light recognition model is trained based on a traffic light image sample set, which includes multiple traffic light image samples and label information. The preset multi-task traffic light recognition model includes a target backbone network module, a target feature aggregation module, and multiple target task detection heads. The target feature aggregation module includes a weighted bidirectional feature pyramid network, and each target task detection head corresponds to a different traffic light recognition task and a different output channel. The construction process of the preset multi-task traffic light recognition model includes: acquiring the traffic light image sample set; wherein the traffic light image sample set includes a traffic light image training set and a traffic light image verification set; inputting the traffic light image training set into... The process of training an initial multi-task traffic light recognition model to generate an intermediate multi-task traffic light recognition model includes: inputting the traffic light image training set into the initial multi-task traffic light recognition model; extracting features from the traffic light image training set through the backbone network module of the initial multi-task traffic light recognition model to generate multi-scale feature data; fusing the multi-scale feature data through the weighted bidirectional feature pyramid network in the target feature aggregation module to generate multiple fused features; generating a predicted recognition result based on the fused features corresponding to each task detection head through each task detection head; wherein each predicted recognition result corresponds to each traffic light recognition task; and adjusting the parameters of the initial multi-task traffic light recognition model based on each predicted recognition result and the label information to generate the intermediate multi-task traffic light recognition model.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the steps of the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the steps of the method as described in any one of claims 1-5.

Citation Information

Patent Citations

  • Training method of traffic light recognition model and traffic light recognition method

    CN116310629A