A method and system for improving classification accuracy of traffic scene target detection

Abstract

This invention discloses a method and system for improving the accuracy of target detection and classification in traffic scenes, comprising the following steps: S1: acquiring traffic video; S2: converting the traffic video into traffic images; S3: recognizing the traffic images based on a convolutional neural network to obtain the classification results of the traffic images; S4: issuing warnings based on the classification results of the traffic images. The beneficial effects of this invention are: it can improve the convergence speed of the model, accelerate the training of the model, and improve the recognition accuracy of the model.

Description

Technical Field

[0001] This invention relates to the field of traffic detection technology, and in particular to a method and system for improving the accuracy of target detection and classification in traffic scenes. Background Technology

[0002] In recent years, deep learning-based object detection technology has been increasingly widely used in traffic scenarios. Initially, people focused more on the location information of traffic targets, but as technology and life become more closely integrated, the demand for more refined categories of traffic targets has also increased.

[0003] In existing technologies, finer category recognition is mainly achieved by adding attention modules, which refine the intermediate layer features of the network to further improve category accuracy. However, adding attention modules increases the model's complexity and inference time, hindering its deployment and use, especially in traffic scenarios where high inference speed is required. When deploying the model on devices like Jetson, lightweight design must be ensured. This results in a relatively long model recognition time.

[0004] For example, a method for pedestrian traffic light recognition based on traffic light geometric attributes, disclosed in Chinese patent literature, with publication number CN113011251A and application date of February 3, 2021, identifies the dynamic state of traffic lights by using traffic light frame images and the shape of the identified traffic lights, in order to provide more accurate guidance for visually impaired people. However, it suffers from problems such as long model recognition time and low accuracy. Summary of the Invention

[0005] To address the shortcomings of existing technologies, such as long recognition time and low accuracy, this invention proposes a method and system for improving the accuracy of target detection and classification in traffic scenes. This method can improve the convergence speed of the model, accelerate model training, and enhance the model's recognition accuracy.

[0006] The following is the technical solution of the present invention: a method for improving the accuracy of target detection and classification in traffic scenes, comprising the following steps:

[0007] S1: Obtain traffic video;

[0008] S2: Convert traffic video into traffic images;

[0009] S3: Traffic images are identified based on convolutional neural networks to obtain the classification results of the traffic images;

[0010] S4: Issue early warnings based on the classification results of traffic images.

[0011] In this solution, traffic videos are acquired through a video acquisition module and converted into traffic images. These images are then processed by a convolutional neural network to classify and identify the traffic data. Warnings are then issued based on the classification results. This approach, which derives classification information from traffic videos, improves the model's convergence speed and recognition accuracy.

[0012] Preferably, S3 includes the following steps:

[0013] S31: Input traffic image, calculate anchor boxes and data augmentation;

[0014] S32: Use the CSP structure to downsample the traffic image five times;

[0015] S33: The feature maps obtained from the third, fourth and fifth downsampling are subjected to feature extraction and fusion respectively to obtain feature maps T3, T4 and T5 respectively;

[0016] S34: T3, T4 and T5 are respectively connected to the GSConv structure, and the generated feature maps are denoted as P3, P4 and P5 respectively;

[0017] S35: P5 is upsampled by 2x and then superimposed with P4 through channels. After passing through the C3 structure, the fusion features are further refined. Then, after being upsampled by 2x, it is superimposed with P3 through channels. After passing through the C3 structure, the fusion features are further refined to obtain feature M3. M3 is downsampled by 2x and then superimposed with P4 through channels. After passing through the C3 structure, the fusion features are further refined to obtain feature M4.

[0018] After M4 is downsampled by 2x, it is superimposed with P5 through channels, and then further refined and fused through the C3 structure to obtain feature M5.

[0019] S36: M3, M4 and M5 are respectively obtained by a conv+BN+SiLU convolutional block and are denoted as Q3, Q4 and Q5;

[0020] S37: Input Q3, Q4 and Q5 into the detection head respectively;

[0021] S38: Make predictions for Q3, Q4, and Q5, generate bounding boxes, and predict the classification of traffic images.

[0022] In this scheme, the traffic image is downsampled five times, and the feature map obtained by downsampling is processed by the GSConv structure. The feature map is then channel-stacked according to a specified rule to obtain a refined fusion feature map, which is then subjected to standard convolution to obtain a new feature map. The new feature map is then input into the detection head for prediction to obtain the classification of the traffic image, which can improve the convergence speed and recognition accuracy of the model.

[0023] Preferably, in step S31, the traffic image is scaled proportionally to the image size of the convolutional neural network model.

[0024] In this scheme, traffic images are scaled proportionally to the image size preset by the convolutional neural network model, which allows the traffic image content to be entered into the convolutional neural network model for recognition without distortion, thereby improving the model's recognition accuracy.

[0025] As a preferred option, in step S33, 1x1 convolution is used for feature extraction and feature fusion.

[0026] In this scheme, the convolution process of the 1x1 convolution kernel is equivalent to the computation process of a fully connected network. By adding a non-linear activation function, the non-linearity of the network can be increased, enabling the network to express more complex features. In model design, this can play a role in model optimization and reducing the number of parameters, thereby improving the model's recognition accuracy.

[0027] As a preferred embodiment, in step S34, the GSConv structure is as follows: input feature map C1, C1 passes through a conv+BN+SiLU convolutional block to obtain feature map C21, feature map C21 passes through another conv+BN+SiLU convolutional block to obtain feature map C22, C21 and C22 are channel-overlapped, and after channel clipping by adding shuffle, feature map C2 is output.

[0028] In this scheme, the feature map is processed using the GSConv structure, resulting in more complex features and higher accuracy in recognition and classification.

[0029] As a preferred approach, the sample allocation strategy in a convolutional neural network model includes the following steps:

[0030] S301: Match anchors and ground truth to determine the positive sample anchors of the current feature map;

[0031] S302: Assign the positive samples of the current feature map to the corresponding grid;

[0032] S303: Calculate the regression and classification loss for each positive sample with respect to each ground truth (GT), and obtain the cost matrix and IoU matrix;

[0033] S304: Based on the IoU matrix, select the top ten candidate boxes by sorting.

[0034] S305: Add the IoU of these ten candidate boxes and round down to get the number of candidate boxes k;

[0035] S306: Select the top k candidate boxes based on the cost matrix and remove duplicate candidate boxes.

[0036] In this approach, the model's accuracy is not improved by balancing positive and negative samples through increasing the dataset.

[0037] Preferably, in step S4, if the classification result matches the warning classification, a warning is issued for the classification result through text, sound, and light.

[0038] In this solution, traffic images requiring warnings are categorized as warning categories. Each warning category includes one or more classification results. Different warning schemes are set for different classification results, including warning text, warning sound, and warning lights. In other words, warnings are issued based on the classification results through text, sound, and light. This allows for warnings to be issued for specified classification results, improving the system's applicability.

[0039] Preferably, in step S2, the traffic video is converted into a traffic image every 5 to 20 frames.

[0040] In this solution, since a traffic video contains too many frames, it is not necessary to convert all frames into traffic images. Therefore, the traffic images are converted every 10 frames, which can improve the recognition speed of the model.

[0041] A system for improving the accuracy of target detection and classification in traffic scenes includes: a video acquisition module, which is connected to an operation and maintenance transmission module, which is connected to a video storage module, which is connected to a video processing module and a display module, which is connected to a target detection module, which is connected to a target storage module and a target early warning module, and which is connected to a display module.

[0042] In this solution, the video acquisition module is used to acquire traffic videos; the operation and maintenance transmission module is used to transmit traffic videos to the database; the video storage module is used to store traffic videos; the video processing module is used to extract image frames from the traffic videos; the target detection module is used to detect, identify, and classify traffic images; the target storage module is used to store the classification results of traffic images; the target warning module is used to issue warnings based on the classification results of traffic images; and the display module is used to display traffic videos and classification results.

[0043] Preferably, the video acquisition module is installed on the support pole where the traffic lights are located or on the monitoring poles on both sides of the road.

[0044] In this solution, it is convenient for the video acquisition module to collect traffic videos on the lanes.

[0045] The beneficial effects of this invention are: improving the recognition accuracy of the model with minimal time cost; improving the accuracy of the model without increasing the dataset to balance positive and negative samples; and improving the convergence speed of the model and accelerating the training of the model. Attached Figure Description

[0046] Figure 1This invention provides a schematic diagram of a system for improving the accuracy of target detection and classification in traffic scenes.

[0047] Figure 2 The present invention provides a flowchart of a method for improving the accuracy of target detection and classification in traffic scenes.

[0048] Figure 3 This invention relates to a model data transfer diagram for a method to improve the accuracy of target detection and classification in traffic scenes.

[0049] Figure 4 The GSConv structure diagram of the method for improving the accuracy of target detection and classification in traffic scenes according to the present invention.

[0050] In the diagram: 1. Video acquisition module; 2. Operation and maintenance transmission module; 3. Video storage module; 4. Video processing module; 5. Target detection module; 6. Target storage module; 7. Target early warning module; 8. Display module. Detailed Implementation

[0051] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0052] Example: Figure 1 As shown, a system for improving the accuracy of target detection and classification in traffic scenes includes:

[0053] Video acquisition module 1 is used to acquire traffic videos;

[0054] Operation and maintenance transmission module 2 is used to transmit traffic videos to the database and is connected to video acquisition module 1;

[0055] Video storage module 3 is used to store traffic videos and is connected to maintenance and transmission module 2;

[0056] Video processing module 4 is used to extract image frames from video traffic videos and is connected to video storage module 3;

[0057] Target detection module 5 is used to detect, identify and classify traffic images, and is connected to video processing module 4;

[0058] The target storage module 6 is used to store the classification results of traffic images and is connected to the target detection module 5;

[0059] Target warning module 7 is used to issue warnings based on traffic image classification results and is connected to target detection module 5;

[0060] Display module 8 is used to display traffic videos and classification results, and connects video storage module 3 and target storage module 6.

[0061] Video acquisition module 1 is used to acquire traffic video and connects to maintenance and transmission module 2. Video acquisition module 1 can be a camera or a monitor, typically mounted on the support pole of the traffic lights or on monitoring poles along the roadside. Video acquisition module 1 monitors vehicles traveling on the road, facilitating traffic analysis based on the video footage.

[0062] The maintenance and transmission module 2 is used to transmit traffic video to the database and connects the video storage module 3 and the video acquisition module 1. The maintenance and transmission module 2 and the video storage module 3 are wirelessly connected.

[0063] Video storage module 3 is used to store traffic videos and connects to operation and maintenance transmission module 2 and video processing module 4. Video storage module 3 can be a database. When storing traffic videos, video storage module 3 stores traffic videos from different video acquisition modules 1 in different data tables, or stores traffic videos from different video acquisition modules 1 in the same data table, distinguishing the source of the traffic videos based on the video acquisition module 1's serial number.

[0064] The video processing module 4 is used to extract image frames from the traffic video and connects the video storage module 3 and the target detection module 5. The video processing module 4 extracts the traffic video from the video storage module 3, converts the traffic video into traffic images, saves the traffic images to the corresponding folder, and uses the converted traffic images for detection and recognition by the target detection module 5.

[0065] The target detection module 5 is used to detect, identify, and classify traffic images, and is connected to the video processing module 4. The target detection module 5 uses a YOLOv5 convolutional neural network to identify and classify traffic images: the input traffic image enters the convolutional neural network, adaptively calculates anchor boxes, and performs data augmentation processing on the traffic image; the feature extraction backbone network uses a fully convolutional network with a CSP structure, performing five downsampling operations on the traffic image; the feature maps obtained from the third, fourth, and fifth downsampling operations are respectively subjected to 1x1 convolution for feature extraction and feature fusion, resulting in feature maps T3, T4, and T5, respectively; feature maps T3, T4, and T5 are then connected to a GSConv structure, generating feature maps P3, P4, and P5, respectively; feature map P5 is upsampled by two times and then superimposed with feature map P4 through channels, further refined and fused after passing through a C3 structure, and then... After a 2x upsampling, the feature map is overlaid with feature map P3 via channels. After passing through a C3 structure, the fused features are further refined, resulting in feature M3. Feature M3 is then downsampled with feature map P4 via channels, and after passing through a C3 structure, the fused features are further refined, resulting in feature M4. Feature M4 is then downsampled with feature map P5 via channels, and after passing through a C3 structure, the fused features are further refined, resulting in feature M5. Features M3, M4, and M5 are then processed through a conv+BN+SiLU convolutional block to obtain feature maps Q3, Q4, and Q5, respectively. Feature maps Q3, Q4, and Q5 are then input into the detection head. Prediction is performed on feature maps Q3, Q4, and Q5 to generate bounding boxes and predict the classification of the traffic image. This improves the model's recognition accuracy with minimal time cost.

[0066] The target storage module 6 stores the classification results of traffic images predicted by the convolutional neural network and is connected to the target detection module 5. The target storage module 6 stores the classification results of traffic images, facilitating statistical analysis of the traffic image classification.

[0067] The target warning module 7 is used to issue warnings based on the traffic image classification results of the target detection module 5, and is connected to the target detection module 5. It sets the traffic image classification requiring warning as a warning category, which includes one or more classification results. For different classification results, the same or different warning schemes are set. Warning schemes include warning text, warning sound, and warning lights, i.e., warnings are issued based on the classification results through text, sound, and light. When the classification result of the target detection module 5 is a warning category, the target warning module 7 executes the warning scheme.

[0068] Display module 8 is used to display traffic videos and classification results, and connects video storage module 3 and target storage module 6. Display module 8 retrieves and displays traffic videos from video storage module 3, and retrieves and displays classification results from target storage module 6, facilitating accurate viewing of traffic conditions.

[0069] like Figure 2 As shown, a method for improving the accuracy of target detection and classification in traffic scenes includes the following steps:

[0070] S1: Obtain traffic video;

[0071] S2: Convert traffic video into traffic images;

[0072] S3: Traffic images are identified based on convolutional neural networks to obtain the classification results of the traffic images;

[0073] S4: Issue early warnings based on the classification results of traffic images.

[0074] Traffic videos are acquired through video acquisition module 1 and converted into traffic images. These images are then used by a convolutional neural network to identify and classify them, resulting in a classification result. Warnings are then issued based on this classification result.

[0075] S1: Obtain traffic video.

[0076] Specifically, cameras or monitors are installed on the support poles of traffic lights or on monitoring poles along both sides of the road to collect traffic videos. The collected traffic videos are then transmitted to the database via the maintenance and transmission module 2, which is wirelessly connected to the video storage module 3.

[0077] S2: Convert traffic video into traffic images.

[0078] Specifically, video processing module 4 extracts traffic video from video storage module 3, converts the traffic video into traffic images, and saves the traffic images to the corresponding folder. These converted traffic images are then used by target detection module 5 for detection and recognition. The traffic images are 1920*1080 pixels in size; larger traffic images improve the detection of small targets and enhance feature extraction. Since a single traffic video contains too many frames, it is not necessary to convert all frames into traffic images; therefore, the conversion is performed every 10 frames.

[0079] S3: Convolutional neural networks identify traffic images and classify them.

[0080] Specifically, YOLOv5 convolutional neural networks are used for image recognition and classification, such as... Figure 3 and Figure 4As shown, the data transfer in the convolutional neural network model improves the model's recognition accuracy with minimal time cost, including the following steps:

[0081] S31: The input traffic image is fed into a convolutional neural network, which adaptively calculates anchor boxes and performs data augmentation on the traffic image. The input traffic image is scaled according to the input image size ratio.

[0082] S32: The feature extraction backbone network adopts a fully convolutional network with a CSP structure, and performs five downsampling operations on the traffic image.

[0083] S33: The feature maps obtained from the third, fourth and fifth downsampling are subjected to 1x1 convolution for feature extraction and feature fusion, and the resulting feature maps are denoted as feature map T3, feature map T4 and feature map T5 respectively.

[0084] S34: Feature maps T3, T4, and T5 are then connected to the GSConv structure, and the generated feature maps are denoted as feature map P3, feature map P4, and feature map P5, respectively.

[0085] The GSConv structure is as follows: Input feature map C1, the feature map is passed through a conv+BN+SiLU convolution block to obtain a hidden feature map C21, feature map C21 is passed through another conv+BN+SiLU convolution block to obtain a hidden feature map C22, feature maps C21 and C22 are channel-stacked, and after adding shuffle to perform channel clipping, feature map C2 is output.

[0086] S35: Feature map P5 is upsampled by 2 times and then superimposed with feature map P4 through channels. After passing through the C3 structure, the features are further refined and fused. Then, after being upsampled by 2 times again, it is superimposed with feature map P3 through channels. After passing through the C3 structure, the features are further refined and fused to obtain the feature denoted as feature M3.

[0087] Feature M3 is downsampled by 2 times and then superimposed on feature map P4. After passing through structure C3, the feature is further refined and fused to obtain feature M4.

[0088] Feature M4 is downsampled by 2 times and then superimposed on feature map P5. After passing through structure C3, the feature is further refined and fused to obtain feature M5.

[0089] S36: Features M3, M4 and M5 are respectively processed by a conv+BN+SiLU convolutional block to obtain feature maps Q3, Q4 and Q5;

[0090] S37: Input feature map Q3, feature map Q4 and feature map Q5 into the detection head respectively;

[0091] S38: Predict feature maps Q3, Q4, and Q5, generate bounding boxes, and predict the classification of traffic images.

[0092] The sample allocation strategy in convolutional neural network models, which balances positive and negative samples and improves model accuracy without increasing the dataset, includes the following steps:

[0093] S301: Match the prior bounding boxes (anchors) and ground truth (GT) to determine the positive sample anchors of the current feature map;

[0094] S302: Assign positive samples from the current feature map to the corresponding grid.

[0095] S303: Calculate the regression and classification loss of each positive sample with respect to each ground truth, and obtain the cost matrix and the IoU (Intersection over Union) matrix;

[0096] S304: Based on the IoU matrix, select the top ten candidate boxes by sorting.

[0097] S305: Add the IoU of these ten candidate boxes and round down to get the number of candidate box objects k assigned to the target box;

[0098] S306: Select the top k candidate boxes based on the cost matrix and remove duplicate candidate boxes.

[0099] S4: Issue early warnings based on the classification results of traffic images.

[0100] Specifically, traffic images requiring warnings are categorized as warning categories. A warning category includes one or more classification results. Different warning schemes are set for different classification results, including warning text, warning sound, and warning lights. In other words, warnings are issued based on the classification results through text, sound, and light. When the classification result of the target detection module 5 is a warning category, the target warning module 7 executes the warning scheme.

[0101] Improve the model's recognition accuracy with minimal time cost; improve the model's accuracy without increasing the dataset to balance positive and negative samples; improve the model's convergence speed and accelerate model training.

Claims

1. A method for improving the accuracy of target detection and classification in traffic scenes, characterized in that, Includes the following steps: S1: Obtain traffic video; S2: Convert traffic video into traffic images; S3: Traffic image recognition based on convolutional neural network; input traffic image, calculate anchor boxes and data augmentation; The traffic image was downsampled five times using the CSP structure; The feature maps obtained from the third, fourth, and fifth downsampling are subjected to feature extraction and fusion respectively to obtain feature maps T3, T4, and T5; After T3, T4 and T5, respectively, they are connected to the GSConv structure to generate feature maps P3, P4 and P5; After being upsampled by 2x, P5 is superimposed with P4 through the C3 structure, then upsampled by 2x again and superimposed with P3 through the C3 structure to obtain feature M3. After M3 is downsampled by 2 times, it is superimposed with P4 channels, and then passed through the C3 structure to obtain feature M4; After M4 is downsampled by 2x, it is superimposed with P5 channels, and then passed through the C3 structure to obtain feature M5; M3, M4 and M5 are obtained by passing a conv+BN+SiLU convolutional block to obtain feature maps Q3, Q4 and Q5 respectively; Q3, Q4, and Q5 are input into the detection head respectively; Predict Q3, Q4, and Q5, generate bounding boxes, and predict the classification results of the traffic image; S4: Issue early warnings based on the classification results of traffic images.

2. The method for improving the accuracy of target detection and classification in traffic scenes according to claim 1, characterized in that, Traffic images are scaled proportionally to the image size of the convolutional neural network model.

3. A method for improving the accuracy of target detection and classification in traffic scenes according to claim 1 or 2, characterized in that, The process of extracting and fusing features from the feature maps obtained by the third, fourth, and fifth downsampling steps includes using 1x1 convolution for feature extraction and feature fusion.

4. A method for improving the accuracy of target detection and classification in traffic scenes according to claim 1 or 2, characterized in that, The GSConv structure is as follows: input feature map C1, C1 passes through a conv+BN+SiLU convolution block to obtain feature map C21, feature map C21 passes through another conv+BN+SiLU convolution block to obtain feature map C22, C21 and C22 are stacked by channels, and after adding shuffle to perform channel clipping, feature map C2 is output.

5. The method for improving the accuracy of target detection and classification in traffic scenes according to claim 1, characterized in that, The sample allocation strategy in a convolutional neural network model includes the following steps: S301: Match anchors and ground truth to determine the positive sample anchors of the current feature map; S302: Assign the positive samples of the current feature map to the corresponding grid; S303: Calculate the regression and classification loss for each positive sample with respect to each ground truth (GT), and obtain the cost matrix and IoU matrix; S304: Based on the IoU matrix, select the top ten candidate boxes by sorting. S305: Add the IoU of these ten candidate boxes and round down to get the number of candidate boxes k; S306: Select the top k candidate boxes based on the cost matrix and remove duplicate candidate boxes.

6. The method for improving the accuracy of target detection and classification in traffic scenes according to claim 1, characterized in that, In step S4, if the classification result matches the warning classification, a warning is issued for the classification result through text, sound, and light.

7. The method for improving the accuracy of target detection and classification in traffic scenes according to claim 1, characterized in that, In step S2, the traffic video is converted into a traffic image every 5 to 20 frames.

8. A system for improving the accuracy of target detection and classification in traffic scenes, applicable to the method for improving the accuracy of target detection and classification in traffic scenes as described in any one of claims 1-7, characterized in that, include: The video acquisition module connects to the operation and maintenance transmission module, which in turn connects to the video storage module. The video storage module connects to the video processing module and the display module. The video processing module connects to the target detection module, which in turn connects to the target storage module and the target early warning module. Finally, the target storage module connects to the display module.

9. The system for improving the accuracy of target detection and classification in traffic scenes according to claim 8, characterized in that, The video acquisition module is installed on the support pole where the traffic lights are located or on the monitoring poles on both sides of the road.

Images