End-to-end traffic light multi-task sensing method

Abstract

The invention relates to an end-to-end traffic light multi-task sensing method, and the method comprises the steps: enabling a plurality of different tasks to share a model backbone network, and enabling the model parameters to be highly reused; performing weighted summation on the loss of each task to obtain final multi-task loss, and realizing mutual supervision among multiple tasks; in the training stage, a topk10 and topk1 double-branch positive sample distribution strategy is used, and topk10 provides more positive samples for the model, so that training is easy to converge; only a topk1 branch export model is used, so that a model output result is reduced to a lower order of magnitude, complexity caused by using an NMS when multiple targets are used is avoided, and traffic light perception post-processing is accelerated. Three different tasks of traffic light target detection, attribute identification and countdown key point positioning in traffic light perception can be unified into one network model. Traffic image input and single-model reasoning output of a multi-task result are realized. And compared with a multi-model serial mode, the model parameter quantity and the calculation quantity are greatly reduced, and the method is more friendly to embedded equipment.

Description

Technical Field

[0001] The present invention relates to a target recognition technology, and in particular to an end-to-end traffic light multi-task perception method. Background Art

[0002] With the rapid development of driverless technology, higher requirements are placed on the vehicle's perception system. As a crucial signal in traffic rules, the correct recognition of traffic lights is crucial to ensure the safe driving of autonomous vehicles. The traffic light perception system not only needs to identify the state of the traffic light, but also needs to accurately locate the position of the traffic light, identify the type, recognize the color, determine the turn indication, and detect key information such as countdown.

[0003] Traditional unmanned driving systems mainly rely on high-precision map data to obtain traffic light and traffic sign information. However, such methods are heavily dependent on the real-time, accuracy and coverage of maps and cannot meet the requirements of true unmanned driving perception. In addition, in the traffic light perception task based on computer vision, due to the diversity of tasks, multiple models are often required to complete multiple tasks in series, which will lead to the consumption of equipment resources and the redundancy of information between models. Summary of the invention

[0004] Aiming at the problem of traffic light recognition in unmanned driving, an end-to-end traffic light multi-task perception method is proposed, which can unify the three different tasks of traffic light target detection, attribute recognition, and countdown key point positioning in traffic light perception into one network model. It realizes input of traffic images, single model reasoning and output of multi-task results. In the traffic light perception method of the present invention, each task shares the backbone, which greatly reduces the number of model parameters and calculations compared to the multi-model serial method, and is more friendly to embedded devices. In addition, the non-maximum suppression (NMS) process is removed, and the target result is directly output, so as to truly achieve end-to-end detection.

[0005] The technical solution of the present invention is: an end-to-end traffic light multi-task perception method, which specifically includes the following steps: 1) Dataset construction: A telephoto camera is installed in the center of the vehicle's cab, with the front lane located slightly below the center of the image. The traffic light image in front of the vehicle is captured to construct a multi-task fusion dataset of traffic lights. The data set needs to label the traffic light frame location, type attributes, color attributes, turn attributes, and countdown key point areas; 2) Build an end-to-end traffic light perception network for training: After the input image first enters the backbone network of Yolov8CSPDarknet, the image is processed by 5 levels of feature scale layers. Each level of feature layer downsamples the image features of the previous layer by 2 times; then the feature maps of feature scales 3, 4, and 5 are input to the image feature network neck layer of PAFPN for multi-scale feature fusion; Neck layer: Two training branches are set. One uses topk=10 to select the top ten targets with the largest scores to allocate positive and negative samples. The other uses topk=1 to retain only the largest scores to allocate positive and negative samples. The three scale feature maps output by the neck layer are input into the two training branches respectively. Both training branches include three task branches, namely, multi-attribute branch, detection box branch, and key point branch. Multi-task branch loss calculation is performed in the two training branches, and back propagation and iterative optimization are performed. Head layer: model reasoning and export. The feature maps of each scale first concatenate the multi-task branch results, and then concatenate the results of the three different scale feature maps. Then, they are decoupled according to the attributes, box, and points tasks of the traffic light, and the top 50 targets with the maximum value are selected based on the category scores in the attributes. The complete attribute results, box results, and points results corresponding to these 50 targets are the training output results of the end-to-end traffic light multi-task perception network. 3) Model training: Multi-task loss: Multi-attribute branch loss according to different attributes, including category loss L bce_cls , color loss L bce_color With the direction loss L bce_direction ; The detection box branch loss includes the prediction accuracy loss L CIoU and label truth loss L dfl ; The key point branch loss includes the key point coordinate loss L OKS With the key point score loss L bce_pose ; The weighted sum of the above losses is the final multi-task loss training loss; The training set data selected in the data set in step 1) is sent to the network structure in step 2), and the model training is completed by iterative optimization through calculating the multi-task loss each time; and the trained model is verified and tested through the verification set and test set selected in the data set in step 1); 4) End-to-end traffic light multi-task perception implementation: After the model training is completed, only the feature map branch corresponding to topk1 is retained, and other structures and parameters remain unchanged, forming an end-to-end traffic light multi-task perception network; real-time images are collected and sent to the network for multi-task recognition, and the recognition results are output, including traffic light target detection box, traffic light type and score, traffic light color and score, traffic light turn and score, traffic light countdown key points and scores.

[0006] Furthermore, the traffic light frame position in step 1) corresponds to the detection frame; the type attributes include motor vehicle traffic lights, non-motor vehicle traffic lights, and pedestrian traffic lights; the color attributes include red, green, yellow, and black; the steering attributes include left turn, right turn, straight go, U-turn, and no direction; the type attribute, color attribute, and steering attribute correspond to multiple attributes; and the countdown key point area corresponds to a key point.

[0007] Furthermore, each of the three task branches in step 2) is constructed using a basic convolutional block of conv+bn+relu according to the task content.

[0008] Furthermore, the positive and negative sample allocation in step 2) is performed using a task alignment learning dynamic allocation strategy. Task alignment learning selects positive samples based on a weighted score of the classification and regression quality scores, so that the sample allocation capability is continuously enhanced with training, thereby supplementing the model's demand for positive samples on different tasks.

[0009] Furthermore, the two-way training branch sets the number of topk, i.e., the maximum target of topk1 prediction fraction and the top 10 target of topk10 prediction scores, and sets two positive and negative sample allocations, corresponding to two feature map outputs; in the training stage, topk10 provides more positive samples for training to help the model converge quickly; in the export stage, only one prediction box of topk1 is assigned to a real object label to avoid NMS operation.

[0010] A shared method for end-to-end traffic light perception network model training and model export. Multiple different tasks share the model backbone network, making the model parameters highly reusable. The weighted sum of the losses of each task is used as the final multi-task loss to achieve mutual supervision between multiple tasks. In the training phase, a positive sample allocation strategy of topk10 and topk1 dual branches is used. Topk10 provides more positive samples for the model to make the training easier to converge. Only the topk1 branch is used to export the model, so that the model output result is reduced to a lower level, avoiding the complexity brought by using NMS for multiple targets and accelerating the post-processing of traffic light perception.

[0011] The beneficial effects of the present invention are as follows: the end-to-end traffic light multi-task perception method of the present invention realizes multiple perception task functions of traffic light target detection, type classification, color classification, turn classification, and countdown key point detection with a single model. Compared with the multi-model serial method: the overall model parameter amount and calculation amount are greatly reduced; the method does not include the NMS post-processing method and does not involve a complex post-processing process, so that the output is the result, which greatly reduces the post-processing time and is more friendly to embedded devices; the method integrates multiple computer vision tasks, including target detection, multi-attribute classification, and key point positioning tasks, designs new loss strategies and training methods, and can provide model integration training ideas for such multi-task problems; multi-task decoupled dual-branch positive sample allocation strategy: topk10 and topk1 positive sample allocation strategy, during training, topk10 provides more positive samples for the model, making the model training easier to converge; when the model is exported, only the topk1 branch feature map is retained, so that the model output candidate results are greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a diagram showing the structure of an end-to-end traffic light multi-task perception network of the present invention; Figure 2 Design implementation details of three task branches on a certain scale of the present invention; Figure 3 This is a detailed diagram of the decoupling of each task branch of the traffic light network of the present invention; Figure 4 The traffic light network of the present invention outputs the implementation details diagram in the type topk1; Figure 5 This is the output result diagram of the end-to-end traffic light multi-task perception of the present invention. DETAILED DESCRIPTION

[0013] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0014] An end-to-end traffic light multi-task perception method uses vehicle-mounted camera images to construct a dataset; builds an end-to-end traffic light perception network; adopts a dual-branch positive and negative sample allocation strategy for training; in the inference stage, topk1 targets are screened based on traffic light type attributes; the screening result sets direction attributes, color attributes, detection boxes, and key points are used for output; and the final post-processing is filtered according to the score threshold to obtain the output result.

[0015] The specific technical solutions are as follows: 1. Hardware equipment installation: Install the telephoto 800M camera in the center directly in front of the vehicle's cab, so that the front lane is located slightly below the center of the image.

[0016] 2. Dataset construction: The above-mentioned camera is used to shoot traffic light images in front of the car to construct a traffic light multi-task fusion dataset, and the training set, validation set, and test set are divided into 8:1:1. The data set needs to mark the traffic light frame position, type attributes (motor vehicle traffic light, non-motor vehicle traffic light, pedestrian traffic light), color attributes (red, green, yellow, black), steering attributes (left turn, right turn, straight, U-turn, no direction) and countdown key point area (if it does not exist, use -1 to occupy the place).

[0017] 3. Build an end-to-end traffic light perception network like Figure 1 As shown in the end-to-end traffic light multi-task perception network structure diagram, the input image first enters the backbone of Yolov8CSPDarknet, and then the image is processed by 5 levels of feature scale layers. Each level of feature layer downsamples the image features of the previous layer by 2 times. Then, the feature maps of feature scales 3, 4, and 5 (i.e., corresponding to 1 / 8, 1 / 16, and 1 / 32 of the original input image scale) are input to the neck layer of PAFPN (the network layer for mixing and combining image features) for multi-scale feature fusion. During training, two training branches are set. One uses topk=10 to select the top ten targets with the largest scores for positive and negative sample allocation, and the other uses topk=1 to retain only the largest score for positive and negative sample allocation. After the three scale feature maps output by the neck layer are input to the two branches respectively, the multi-task branch loss calculation is performed in the two branches, and back propagation and iterative optimization are performed. During model inference and export, the model is exported for verification during training. The feature maps of each scale are first concat-joined with the results of the multi-task branches, and then the results of the three feature maps of different scales are concat-joined. Then, according to the attributes, boxes and points of the traffic lights ( Figure 1 The kpts) task is decoupled, and the top 50 targets with the maximum value are selected based on the category score in the attribute. The complete attribute results, box results, and points results corresponding to these 50 targets are the training output results of the end-to-end traffic light multi-task perception network. After the model training is completed, other structures and parameters remain unchanged, only the feature map branch corresponding to topk1 is retained, that is, only the 3-scale feature map output results of the topk=1 branch are retained, and the topk=10 branch is no longer used for feature extraction. The model reasoning and export structure remains unchanged, and only the topk=1 branch output is reasoned and exported.

[0018] The specific structure of each part of the end-to-end traffic light perception network is as follows: 1) Basic framework: The network of the present invention uses Yolov8CSPDarknet as the backbone (backbone network) and Yolov8PAFPN as the neck (the network layer used to collect feature maps in different stages).

[0019] 2) Multi-task branch: For each traffic light task, each training branch unit of the present invention is designed with three identical task branches: multi-attribute branch, detection box branch, and key point branch. Figure 2 Design and implementation details of three task branches at a certain scale Figure 2 As shown in the figure, each branch is constructed using the basic convolutional block of conv+bn+relu. Specifically: in the detection frame branch, 2 basic convolutional blocks + 1 conv are used to build, and the parameters corresponding to the 3 conv layers are (c, 64, 3×3) (64, 64, 3×3) (64, 64, 1×1), where the output 64 is used for the subsequent detection frame coordinate decoupling calculation; in the key point branch, 2 basic convolutional blocks + 1 conv are used to build, and the parameters corresponding to the 3 conv layers are (c, 16, 3×3) (16, 16, 3×3) (16, 12 ,1×1), where 12 are the four corner points (x, y, score) of the countdown area; in the multi-attribute branch, 4 basic convolution blocks + 1 conv are used, and the parameters corresponding to the 5 conv layers are (c, 256, 3×3) (256, 64, 1×1) (64, 64, 3×3) (64, 64, 1×1) (64, n, 1×1), where c is the dimension of the input feature map, and n is the total number of categories in the attribute category (n=3+4+5=12).

[0020] 3) Multi-scale feature map detection: The present invention sets up three scale detection branches (80×80), (40×40), and (20×20). The corresponding three task branches on the three scale feature maps are concat (connected) respectively, and then the three scales are concat (connected) to finally obtain an output result of dimension (bs, m, 8400), where bs is the model batch size, m=64 (detection box dimension) + 12 (key point dimension) + n (number of types 3 + number of colors 4 + number of turns 5).

[0021] 4) Branch decoupling: Figure 3The decoupling implementation details of each task branch of the traffic light network are shown in the figure. In the above m dimension, the split operation is performed according to the number required by each task to obtain the decoupled branches of each task. The attribute branch (type + color + direction) directly performs the sigmoid operation; for the detection frame branch, the DFL decoupling operation is first performed to reduce the feature result from 64 to 4, and the corresponding 4 dimensions are decoupled into the coordinates of the upper left point (x1, y1) and the lower right corner (x2, y2) of the rectangular box using the center point of the anchor (in target detection, the anchor point is each element on the feature map, which corresponds to a certain position in the original image); for the key point branch, the 12 dimensions are directly decoupled into the (x, y, score) of the four corner points of the countdown area, namely the upper left, upper right, lower right, and lower left.

[0022] 5) Aggregate output results by type: Figure 4 The traffic light network is shown in the implementation details of the topk1 output of the category. The category attribute in the attribute branch is used to select the top 50 target dimensions with the largest scores for topk1. The selected results are used together with the remaining color attributes, steering attributes, detection boxes, and key point output results for the gather operation. The final output dimension is (bs, 50, m2), m2 = 4 + 12 + n (3 categories + 4 colors + 5 steering). This operation reduces the number of targets from the original output of 80 × 80 + 40 × 40 + 20 × 20 = 8400 to 50 targets, which can save the non-maximum suppression (NMS) operation in post-processing.

[0023] 4. Model training (1) Positive and negative sample allocation strategy The task alignment learning (TAL) dynamic allocation strategy is used to allocate positive and negative samples. Task alignment learning selects positive samples based on the weighted scores of classification and regression quality scores, so that the sample allocation ability can be continuously enhanced with training, supplementing the model's demand for positive samples on different tasks. The prediction quality of the anchor is described by the weighted product of classification quality and regression quality: (1) The anchor-level alignment degree is calculated for each label box instance using formula (1), where s and u are the classification score and CIoU value respectively, and α and β are weight hyperparameters (default values ​​α=0.5, β=6.0). Through training, the prediction quality t can guide the network to dynamically focus on high-quality positive samples. For each label box instance, k anchors with the largest t value are selected as positive samples, and the remaining anchors are selected as negative samples.

[0024] CIoU calculation method: (2) (3) (4) Among them, α is the weight parameter; v is used to measure the consistency of aspect ratio; w is the width; h is the height; gt is the real box; (b,b gt ) represent the center points of the predicted box and the true box respectively; ρ is the Euclidean distance between the two center points, and c is the diagonal distance of the minimum closed area containing the predicted box and the true box.

[0025] The specific process of the positive and negative sample allocation of the present invention is as follows: ① Preprocessing: Process the categories, bounding boxes, and anchor center points of feature maps. The categories are processed with sigmoid. The detection boxes are restored to the network input scale. The anchor center is restored to the network input scale. The label data is processed in the same way.

[0026] ② Initial screening: All anchors whose center points are in the label box are taken as preliminary positive samples.

[0027] ③ Fine screening: Calculate the alignment score t of the anchor in the previous step, where the category score is calculated based on the type attribute of the traffic light, and select the topk candidate boxes with the highest scores as positive samples.

[0028] ④ Remove duplicate assignments: Find the anchors assigned to multiple label boxes by summing the mask matrix and the anchors, leaving only the anchors with large CIoU values.

[0029] ⑤ Obtain the training label of the sample: obtain the positive and negative samples of the anchor, the positive and negative samples corresponding to the label box, and the positive sample anchor index corresponding to the label box through the mask, and obtain the filtered positive sample training information.

[0030] In the training phase, by setting the number of topk (i.e., the maximum target of topk1 prediction fraction and the top 10 targets of topk10 prediction scores), two positive and negative sample matching processes are set, corresponding to two feature map outputs. In the training phase, topk10 provides more positive samples for training to help the model converge quickly; in the export phase, only one prediction box is assigned to a real object label using topk1 to avoid NMS operation. The topk problem is to find the top K largest (of course, it can also be the top K smallest) numbers in a pile of data.

[0031] Multi-task loss loss Multi-attribute branch loss loss adopts L bce , according to different attributes, including types L bce_cls 、Color L bce_color With direction L bce_direction , (5) Where N is the number of samples, i is the i-th sample, and y i is the label of sample i, p i is the probability that sample i is predicted to be a positive class.

[0032] The detection box branch loss adopts the prediction accuracy loss L CIoU and label truth loss L dfl Two types of loss.

[0033] (6) (7) Among them, S i is the predicted value, S i+1 is the nearest predicted value, y is the true value of the label, and y i is the label integral value, y i+1 Proximity tag score value.

[0034] The key point branch loss adopts L OKS With L bce_pose , where L bce_pose Used to calculate the key point score, L OKS Calculate keypoint coordinates.

[0035] (8) Among them, i represents the key point id, d i 2 represents the Euclidean distance between the label and the predicted key point, and s represents the scale factor (equal to the square root of the area occupied by the target in the label, i.e. ), k i represents the normalization factor of the ith point (obtained by calculating the standard deviation of all labels in the data set, reflecting the standard deviation of the current key point annotation. The larger the value, the more difficult the point is to annotate). i It represents the visibility of the i-th key point (visible is 1, invisible is 0), and δ is the function used to select the visible points for calculation.

[0036] The weighted sum of each loss is the final multi-task loss training loss, where s 1~7 The summed weights corresponding to each loss are:

[0037] 5. Model export and result output In the model export stage, only the feature map branch corresponding to topk1 is retained. The model input dimension (bs, 3, 640, 640) is output as (1, 50, m2) by the above model. The corresponding post-processing is to select targets greater than the threshold according to the topk1 score threshold of the traffic light type attribute. The output target detection box is the final detection box, and the key points are filtered and output according to the key point threshold. The color attribute and steering attribute of the corresponding target are selected according to their respective category scores. The category corresponding to the maximum score is output as the final attribute result. The output after the above post-processing is the final detection result of the model. In this way, non-maximum suppression (NMS) operation is not required in post-processing.

[0038] The model of the present invention can finally output traffic light target detection frame, traffic light type and score, traffic light color and score, traffic light turn and score, traffic light countdown key point and score, so that one model can solve multiple traffic light detection tasks. Each task shares a backbone and neck, which can realize the effective use of network parameters. Compared with the multi-model serial method, the number of model parameters and the amount of calculation are greatly reduced, and the embedded deployment is more friendly.

[0039] The present invention proposes an end-to-end traffic light multi-task perception method, in which a single model can realize target detection (traffic light positioning), attribute recognition (color, steering, type classification), and key point detection (countdown area task) in traffic light perception tasks. Compared with the existing multiple model serial solution method, it has the characteristics of low parameter amount and low computational amount, and is more friendly to embedded device deployment; in addition, the multi-model serial method has the dependency and logical complexity of scheduling between models, and the present invention can effectively avoid such problems during deployment, thereby improving the simplicity and robustness of the system; the technical solution of the present invention involves multi-task learning optimization problems, and multiple different tasks share the backbone network (backbone), so that the model parameters are highly reused to avoid parameter redundancy; at the same time, multiple tasks supervise each other to jointly improve the detection performance; in the training stage, the positive sample allocation strategy of topk10 and topk1 double branches is used, topk10 provides more positive samples for the model to make the training easier to converge, and the topk1 export model reduces the model output results to a lower level, thereby avoiding the complexity brought by the use of NMS for multiple targets, making the post-processing process of traffic light perception more streamlined and rapid.

[0040] The above-mentioned embodiments only express the specific implementation of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. An end-to-end traffic light multi-task perception method, characterized in that: The specific steps include: 1) Dataset construction: A telephoto camera is installed in the center of the vehicle's cab, with the front lane located slightly below the center of the image. The traffic light image in front of the vehicle is captured to construct a multi-task fusion dataset of traffic lights. The data set needs to label the traffic light frame location, type attributes, color attributes, turn attributes, and countdown key point areas; 2) Build an end-to-end traffic light perception network for training: After the input image first enters the backbone network of Yolov8CSPDarknet, the image is processed by 5 levels of feature scale layers. Each level of feature layer downsamples the image features of the previous layer by 2 times; then the feature maps of feature scales 3, 4, and 5 are input to the neck layer of the image feature network of PAFPN for multi-scale feature fusion; Neck layer: Two training branches are set. One uses topk=10 to select the top ten targets with the largest scores for positive and negative sample allocation, and the other uses topk=1 to retain only the largest scores for positive and negative sample allocation. The three scale feature maps output by the neck layer are input into the two training branches respectively. Both training branches include three task branches, namely, multi-attribute branch, detection box branch, and key point branch. Multi-task branch loss calculation is performed in the two training branches, and back propagation and iterative optimization are performed. Head layer: model reasoning and export. The feature maps of each scale first concatenate the multi-task branch results, and then concatenate the results of the three different scale feature maps. Then, they are decoupled according to the attributes, box, and points tasks of the traffic light, and the top 50 targets with the maximum value are selected based on the category scores in the attributes. The complete attribute results, box results, and points results corresponding to these 50 targets are the training output results of the end-to-end traffic light multi-task perception network. 3) Model training: Multi-task loss: Multi-attribute branch loss according to different attributes, including category loss L bce_cls , color loss L bce_color With the direction loss L bce_direction ; The detection box branch loss includes the prediction accuracy loss L CIoU and label truth loss L dfl ; The key point branch loss includes the key point coordinate loss L OKS With the key point score loss L bce_pose ; The weighted sum of the above losses is the final multi-task loss training loss; The training set data selected in the data set in step 1) is sent to the network structure in step 2), and the model training is completed by iterative optimization through calculating the multi-task loss each time; and the trained model is verified and tested through the verification set and test set selected in the data set in step 1); 4) End-to-end traffic light multi-task perception implementation: After the model training is completed, only the feature map branch corresponding to topk1 is retained, and other structures and parameters remain unchanged, forming an end-to-end traffic light multi-task perception network; real-time images are collected and sent to the network for multi-task recognition, and the recognition results are output, including traffic light target detection box, traffic light type and score, traffic light color and score, traffic light turn and score, traffic light countdown key points and scores.

2. The end-to-end traffic light multi-task perception method according to claim 1 is characterized in that: The traffic light frame position in step 1) corresponds to the detection frame; the type attributes include motor vehicle traffic lights, non-motor vehicle traffic lights, and pedestrian traffic lights; the color attributes include red, green, yellow, and black; the steering attributes include left turn, right turn, straight go, U-turn, and no direction; the type attribute, color attribute, and steering attribute correspond to multiple attributes; the countdown key point area corresponds to the key point.

3. The end-to-end traffic light multi-task perception method according to claim 2 is characterized in that: Each of the three task branches in step 2) is constructed using the basic convolutional block of conv+bn+relu according to the task content.

4. The end-to-end traffic light multi-task perception method according to claim 3 is characterized in that: The positive and negative sample allocation in step 2) is performed using a task alignment learning dynamic allocation strategy. Task alignment learning selects positive samples based on a weighted score of the quality scores of classification and regression, so that the sample allocation capability is continuously enhanced with training, thereby supplementing the model's demand for positive samples on different tasks.

5. The end-to-end traffic light multi-task perception method according to claim 4, characterized in that: The two-way training branch sets the number of topk, that is, the maximum target of topk1 prediction fraction and the top 10 target of topk10 prediction scores, sets two positive and negative sample allocations, corresponding to two feature map outputs; in the training stage, topk10 provides more positive samples for training to help the model converge quickly; in the export stage, only one prediction box of topk1 is assigned to a real object label to avoid NMS operation.

6. A common method for end-to-end traffic light perception network model training and model export, characterized in that: The model backbone network is shared by multiple different tasks, making the model parameters highly reusable. The weighted sum of the losses of each task is used as the final multi-task loss to achieve mutual supervision between multiple tasks. In the training phase, a positive sample allocation strategy of topk10 and topk1 branches is used. Topk10 provides more positive samples for the model, making it easier for training to converge. Only the topk1 branch is used to derive the model, reducing the model output results to a lower level, avoiding the complexity brought by using NMS for multiple targets, and accelerating the post-processing of traffic light perception.

Images