Multi-scene tunnel accident discrimination method and device based on YOLOv5, and readable medium

Abstract

The invention discloses a YOLOv5-based multi-scene tunnel accident discrimination method and device and a readable medium, and the method comprises the steps: constructing a multi-scene tunnel accident data set, and marking the data set which comprises a fire hazard type, a vehicle collision type and a pedestrian road occupation type; and carrying out subdivision by considering the development stage and severity of the accident, carrying out model training on the marked data set by using a YOLOv5 target detection algorithm to obtain a model capable of identifying the accident types of fire, vehicle collision and pedestrian road occupation, and developing a PyQt5-based tunnel accident judgment software platform. The tunnel accident discrimination software platform integrates functions of picture detection, video detection and camera real-time detection, and when an accident is detected, the software platform automatically gives an alarm and the like, and the tunnel accident discrimination software platform can automatically identify a monitoring video in a tunnel, quickly discover and judge a traffic accident, automatically give alarm information, and improve the safety of the tunnel. Therefore, the automation level of tunnel operation management is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation systems, and in particular to a multi-scenario tunnel accident identification method, device and readable medium based on YOLOv5. Background Art

[0002] With the rapid development of my country's transportation infrastructure, highway tunnels, as important transportation hubs, are receiving increasing attention for their safe operation and management. However, the special geographical location, unique structure and particularity of traffic flow in tunnels make them face many challenges in safety management. Tunnels are complex in structure, with limited space, long depth, few entrances and exits, and high closure. Once an accident occurs, it often causes serious consequences, including traffic interruption, secondary accidents, and even major economic losses and social impacts.

[0003] Therefore, the importance of tunnel safety operation and management is becoming increasingly prominent. At present, tunnel accident inspections mainly rely on manual video inspections, which have the problems of low automation and poor timeliness. In addition, existing tunnel accident detection research focuses on fire accidents, lacks comprehensive analysis and identification of multi-scenario accidents, and existing research rarely considers the severity and development stage of accidents, which limits the generalization ability and practicality of the model. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] Therefore, the purpose of the present invention is to provide a multi-scenario tunnel accident identification method, device and readable medium based on YOLOv5, which can automatically identify the surveillance video in the tunnel, quickly discover and judge traffic accidents, and automatically issue alarm information, thereby improving the automation level of tunnel operation management, reducing labor costs, improving accident response speed, and effectively preventing the occurrence of secondary accidents.

[0006] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0007] A multi-scenario tunnel accident identification method based on YOLOv5, the steps are as follows:

[0008] S1. Construct a multi-scenario tunnel accident dataset and annotate the dataset. The dataset includes fire, vehicle collision, and pedestrian occupation, and is segmented by considering the development stage and severity of the accident.

[0009] S2. Use the YOLOv5 target detection algorithm to train the model on the labeled data set to obtain a model that can identify accident types such as fire, vehicle collision, and pedestrians occupying the road;

[0010] S3. Develop a tunnel accident identification software platform based on PyQt5, which integrates image detection, video detection and camera real-time detection functions, and when an accident is detected, the software platform automatically issues an alarm;

[0011] S4. Deploy the trained model on the edge computing device Jetson Orin Nano and perform performance testing. After the test is completed, run the deployed model in the tunnel monitoring environment to detect accidents in real-time video streams. When an accident is detected, the system automatically issues an alarm and provides detailed information about the accident to the monitoring personnel.

[0012] As a preferred solution of the multi-scenario tunnel accident identification method based on YOLOv5 described in the present invention, in step S1, constructing a multi-scenario tunnel accident dataset includes the following steps:

[0013] Determine all the types of accidents that are expected to be identified, including fire, vehicle collision, and pedestrians occupying the road, and subdivide them according to the development stage and severity of the accident;

[0014] Collect original image materials, write crawler scripts using Python, search and download original image sets corresponding to accident categories, and perform data screening to remove images that do not meet the requirements;

[0015] Perform data augmentation on images to improve the generalization ability of the model;

[0016] Use the LabelImg tool to label the data set, create a class.txt file, and input all accident label categories;

[0017] Review the annotated images and corresponding labels to ensure the accuracy and consistency of the annotations, ensure that all image materials have corresponding annotation files, and keep the file names consistent.

[0018] As a preferred solution of the multi-scenario tunnel accident identification method based on YOLOv5 described in the present invention, the specific steps of determining all the accident types expected to be identified are as follows:

[0019] Fire accidents are divided into three levels according to the development stage and the severity of the combustion, including smoke in the initial stage, open flames in the development and spread stage, and severe fires with intense combustion and thick black smoke;

[0020] The vehicle collision accident is divided into two scenarios: minor collision and severe collision according to the severity of the collision;

[0021] Pedestrians occupying the road accident is defined as an unexpected incident in which pedestrians appear on the motor vehicle lane in a tunnel.

[0022] As a preferred solution of the multi-scenario tunnel accident identification method based on YOLOv5 described in the present invention, the specific steps of collecting original image materials and screening data are as follows:

[0023] By writing a Python crawler script, we searched with the keywords "tunnel accident", "tunnel fire", "highway fire" and "highway collision" and downloaded the original picture sets of the corresponding accident categories;

[0024] Classify the downloaded image sets and check whether the image quality meets the production requirements of the data set, and eliminate blurred, overexposed, underexposed images and images that do not meet the accident type requirements.

[0025] As a preferred solution of the multi-scenario tunnel accident identification method based on YOLOv5 described in the present invention, the specific steps of calibrating the data set are as follows:

[0026] Use the LabelImg tool to label the YOLO format data set, create a class.txt file, and input all accident label categories;

[0027] Anchor boxes are annotated for each image in the image set to ensure that the anchor boxes fit closely to the edges of the target objects to improve the accuracy of detection.

[0028] Different fire level features that may exist simultaneously in fire accident pictures are selected and calibrated one by one.

[0029] As a preferred solution of the multi-scenario tunnel accident identification method based on YOLOv5 described in the present invention, in step S2, the specific parameters of the model training are set as follows:

[0030] The input image size is 640*640, the weight file is YOLOv5s.pt, the batch size is 8, the learning rate is 0.01, and the training cycle is 100.

[0031] As a preferred solution of the multi-scenario tunnel accident identification method based on YOLOv5 described in the present invention, the following evaluation indicators are used to evaluate the performance of the accident type model obtained by training:

[0032] Loss function L, including confidence loss, classification loss and positioning loss;

[0033] The intersection over union (IoU) is used to measure the similarity between the predicted anchor box and the actual anchor box of the target;

[0034] Precision and recall are used to quantify the detection accuracy of the model;

[0035] The corresponding formulas are as follows:

[0036]

[0037] Among them, TP is the number of true positive examples, FP is the number of false positive examples, and FN is the number of false negative examples. In the performance evaluation, the Precision-Recall curve is used to select the optimal IoU threshold to achieve a balance between demand and performance;

[0038] The average precision (AP) and mean average precision (mAP) are used to evaluate the performance of the model under different classification thresholds.

[0039] A multi-scenario tunnel accident identification device based on YOLOv5, comprising:

[0040] Deep learning target detection model based on YOLOv5;

[0041] A processor, configured to implement a multi-scenario tunnel accident identification method when executing the computer program;

[0042] Memory, used to store computer programs.

[0043] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a multi-scenario tunnel accident identification method is implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects: the present invention constructs a multi-scenario tunnel accident data set and uses the YOLOv5 target detection algorithm for training to identify accident types such as fire, vehicle collision and pedestrians occupying the road. The device of the present invention includes a GUI interface developed based on PyQt5, which integrates image detection, video detection and camera real-time detection functions, and can automatically issue an alarm when an accident is detected. In addition, the present invention also considers the deployment of the model on the edge computing device Jetson Orin Nano. By deploying the model on the Jetson Orin Nano device, the present invention not only realizes low-latency accident detection, but also ensures effective accident identification in the absence of network connection, so as to achieve practicality in a low-latency and offline working environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0046] Figure 1 A schematic diagram of a multi-scenario tunnel accident identification method based on YOLOv5 in the present invention;

[0047] Figure 2 A classification diagram of accident types and data set labels in the embodiment provided by the present invention;

[0048] Figure 3 This is an example diagram of Labelimg annotation in the embodiment provided by the present invention;

[0049] Figure 4 This is an example diagram of fire accident data classification in the embodiment provided by the present invention;

[0050] Figure 5 An example diagram of vehicle collision accident data classification in an embodiment provided by the present invention;

[0051] Figure 6 An example diagram of multi-level accident classification and labeling data in the embodiment provided by the present invention;

[0052] Figure 7 A statistical diagram of the data set annotation information in the embodiment provided by the present invention;

[0053] Figure 8 A detailed diagram of the network structure of the YOLOv5 model in the embodiment provided by the present invention;

[0054] Fig. 9 A loss curve diagram of the model training process in the embodiment provided by the present invention;

[0055] Fig.10 A model reasoning verification diagram in the embodiment provided by the present invention;

[0056] Fig.11 This is a detection effect diagram of the model in the embodiment provided by the present invention under actual tunnel monitoring video;

[0057] Fig.12 A confusion matrix diagram of the model in the embodiment provided by the present invention;

[0058] Fig.13 A chart of model performance evaluation indicators in the embodiments provided by the present invention;

[0059] Fig.14A logic diagram of the software platform operation in the embodiment provided by the present invention;

[0060] Fig.15 This is a diagram of the video detection function interface of the software platform in the embodiment provided by the present invention;

[0061] Fig.16 This is a diagram of the real-time alarm function interface of the software platform in the embodiment provided by the present invention;

[0062] Fig.17 A diagram of an edge device deployment environment in an embodiment of the present invention;

[0063] Fig.18 This is a diagram showing the effect of accident detection on an edge device in an embodiment provided by the present invention. DETAILED DESCRIPTION

[0064] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0065] Secondly, the present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0066] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0067] A multi-scenario tunnel accident detection method based on YOLOv5, such as Figure 1 As shown, the following steps are included:

[0068] S1. Data collection and classification

[0069] In this step, a large amount of image and video data related to tunnel accidents was collected, covering various types of accidents, including fires, vehicle collisions, and pedestrians occupying the road. The data was collected from multiple sources, including videos shot on site, existing surveillance videos, and public accident video databases.

[0070] S2. Data preprocessing

[0071] The collected data needs to be preprocessed to improve the data quality. This step includes removing invalid or low-quality data and standardizing the data, such as adjusting the image size and enhancing the image contrast, to meet the needs of model training.

[0072] S3. Data Annotation

[0073] Use the professional labeling tool LabelImg to perform detailed labeling on the preprocessed image set. Labeling includes drawing a bounding box for each accident image and assigning the corresponding category label. This step is key to ensure that the model can accurately learn accident features.

[0074] S4. Model training

[0075] The deep learning algorithm YOLOv5 is used to train the enhanced dataset. During the training process, the model parameters and learning rate are adjusted, and the changes in the loss function are monitored to ensure that the model can effectively learn and identify different types of tunnel accidents.

[0076] S5. Model Validation

[0077] The performance of the model was tested on an independent validation set, and the accuracy and generalization ability of the model were evaluated by calculating indicators such as precision, recall, and average precision.

[0078] S6. Software platform development

[0079] A user-friendly software platform was developed based on PyQt5, which integrated the model and provided image and video detection capabilities.

[0080] S7. Model deployment

[0081] Deploy the trained model on the edge computing device Jetson Orin Nano. This step includes configuring the device environment, installing necessary libraries and dependencies, and optimizing the model to adapt to the computing power of the edge device.

[0082] S8, Real-time detection and alarm

[0083] The deployed model is run in the actual tunnel monitoring environment to detect accidents on real-time video streams. When an accident is detected, the system automatically issues an alarm and provides detailed information about the accident to the monitoring personnel.

[0084] Through the above steps, the present invention realizes the rapid detection and identification of accidents in tunnels, effectively improves the automation level of highway tunnel operation management, provides timely early warning information for management personnel, and thus reduces the losses caused by accidents.

[0085] The tunnel accident types and data set label classification involved in the present invention are as follows Figure 2 As shown, the following steps are included:

[0086] S11. Determination of accident categories and labels. It is necessary to determine all the types of accidents that are expected to be identified. After research and discussion on tunnel accidents, three major types of accidents were determined: fire, vehicle collision, and pedestrians occupying the road. Taking into account the development stage and severity of the accident, the major types of accidents are further subdivided into subcategories of different levels. Fires are divided into three levels according to the development stage and the severity of the combustion, and collision accidents are divided into two levels according to the severity of the collision.

[0087] S12. Use Python to write a crawler script to search Baidu images with keywords such as "tunnel accident", "tunnel fire", "highway fire", and "highway collision" and download the original picture sets of the corresponding accident categories, and save them into the corresponding images folders according to the three categories.

[0088] S21. After the accident pictures are classified, check whether the picture quality meets the requirements of the data set production, and remove blurry, overexposed, underexposed and pictures that do not meet the accident type requirements. In order to improve the generalization ability of the model, some pictures are subjected to data enhancement operations such as scaling, color change and rotation.

[0089] S31. After the image set is prepared, the data set needs to be calibrated. Select LabelImg as the labeling tool, which provides a simple and intuitive operation interface and can label YOLO format data sets. After completing the relevant environment configuration in Anaconda and running the program, the labeling interface is as follows Figure 3 As shown. During the labeling process, you first need to create a class.txt file and enter all the accident label categories. After that, use the Open Dir command to open the folder "images" where the image set is stored, and use the Chang Save Dir command to set the folder "labels" where the image labels are saved. After opening a picture, use the Create RectBox command to label the anchor box. Click and drag the mouse on the image to draw the box so that the box fits closely to the edge of the target object to improve the accuracy of detection. After labeling, use the Save command to save the label file. The YOLO format label file is stored in .txt format. The label file contains five groups of numbers in each line, representing the label sequence, the horizontal relative coordinates of the left and right boundaries of the anchor box, and the vertical relative coordinates of the upper and lower boundaries of the anchor box.

[0090] S32. Fire is divided into three levels according to the development stage and the severity of burning. The examples are as follows Figure 4As shown in the figure. Fire level 1 is the identification of smoke produced by fire burning, which usually occurs when a traffic accident has just occurred and is the initial stage of the fire; level 2 is the identification of open flames, and the fire is in the development and spread stage; level 3 is the identification of intense flames accompanied by black burning smoke, which can be regarded as a more serious fire. At the same time, in the development process of a fire, the fire level of the latter stage usually contains the fire characteristics of the previous stage.

[0091] S33. Collision accidents are divided into two scenarios: minor collision and severe collision according to the severity of the collision. The examples are as follows Figure 5 A minor collision is a minor collision, including a vehicle scrape, a light rear-end collision, etc. A serious collision is a serious collision, including a collision that causes serious deformation of the vehicle, a rollover, etc.

[0092] S34: Pedestrians occupying the road accident refers to unexpected incidents where pedestrians appear on the motor vehicle lane in the tunnel, including scenes such as crossing the road, leaving the vehicle and being stranded on the road due to an accident, etc.

[0093] S35. In an accident picture, there may be accident features of different degrees in one picture at the same time, so it is necessary to select and calibrate the features of each accident and its level one by one. Figure 6 As shown, two levels of fire were calibrated separately.

[0094] S36. After labeling the image set with LabelImg, in order to construct a multi-scenario tunnel accident detection dataset, the datasets need to be merged so that the model can simultaneously perform recognition training for multiple categories of targets. According to the principle analysis of the dataset labels, each label's .txt format file stores the labels in the relative order of the categories. Therefore, when merging datasets, the operation required is to change the first digit of each row of data so that it corresponds to the label category of the merged dataset to display the correct labeling information. Write a Python program, use the os module to traverse all label files, and use the re module to match the data stored in the label category in the label file through regular expressions and modify it, then write the modified data back to the label file and save it. After performing corresponding operations on all six categories of dataset labels, the merged multi-scenario tunnel accident discrimination dataset is obtained. After merging, the statistical information of the obtained multi-scenario tunnel accident discrimination dataset is as follows: Figure 7 As shown in the figure, this part of the work obtained a data set suitable for tunnel accident identification models in multiple scenarios, which was used for training and verification of subsequent models.

[0095] S41. After the data set is prepared, use YOLOv5 to train the model. To run YOLOv5, you first need to configure the relevant environment. Create a virtual environment in Anaconda and install the environment configuration required for running YOLOv5, and run the YOLOv5 code in the virtual environment in PyCharm software.

[0096] S411 and YOLO are one-stage detection algorithms. One-stage detection attempts to directly classify each region of interest as background or target object, that is, only one stage is needed to directly give the object category probability and position coordinate value. This type of method does not require the extraction of candidate areas in advance but directly predicts and outputs the target object category probability and position, which greatly reduces the consumption of computing resources and speeds up the detection speed of the model.

[0097] YOLOv5s has a simple network structure and occupies a small deployment space. It meets the requirements of tunnel accident target detection while ensuring the real-time monitoring video stream detection speed. Therefore, the present invention uses the YOLOv5s model as the pre-training model of the tunnel accident discrimination model. Its network mainly consists of four parts: input end, Backbone, Neck and Prediction. Its network structure is as follows Figure 8 shown.

[0098] S42. Data set configuration parameter adjustment: Before training, the parameters must first be adjusted to the parameters of the multi-scenario tunnel accident discrimination data set.

[0099] S421. Division of training set and validation set: In deep learning model training, the training set is used for the actual training process of the model's multi-scenario tunnel accident identification. The model identifies patterns and features by learning from the data on the training set. The validation set is used to evaluate the generalization ability of the model in accident classification and grade identification. The model may overfit on the training set, that is, it performs well on the training data but poorly on unseen data. By testing on the validation set, the model's ability to adapt to new data can be evaluated. For this data set, the training set and validation set were divided at a ratio of 0.8 and 0.2, and arranged in the training format.

[0100] S422, configuration of the .yaml file in the data directory: parameter "Train" is set to the training set path. Parameter "Val" is set to the validation set path. Parameter "nc" is set to the number of label categories, which is set to 6 in this dataset. In "names", enter the label names one by one according to the label order in the class.txt file when the dataset was created.

[0101] S423. Configuration of the .yaml file in the model directory: Modify the value of the parameter "nc" to make it consistent with the parameter "nc" in the data directory.

[0102] S424. Setting of training parameters in the train.py file: The parameter settings in the training program train.py are shown in Table 2 below.

[0103] Table 2 Training parameter settings

[0104]

[0105] S43. After all parameters are adjusted, run the train.py file to start training the model using the multi-scenario tunnel accident dataset. The training iteration round epoch of the multi-scenario tunnel accident discrimination dataset set in this paper is 100. Fig. 9 This is the loss-iteration curve in model training. After 70-80 rounds of training, the curve tends to be stable, the change in loss value is very small with the increase in rounds, and the performance change is small, reaching convergence.

[0106] S51. After training the multi-scenario tunnel accident discrimination model, perform model inference verification to detect the actual effect of the model.

[0107] S511. Use the detect.py file to perform inference test on the model. Set the weight parameter representing the model weight in the detect.py file to the path of the best model best.pt obtained through training, and call the model to perform image detection. The detection results are as follows: Fig.10 shown.

[0108] S512. Considering the actual application scenario, a video of a real vehicle fire accident in a tunnel was intercepted and video detection was performed. The detection results are as follows Fig.11 In the detection of this video, the model can correctly identify the fire accident that is happening and output the video file marked with the detection frame, which verifies the practical function of the model.

[0109] S52, such as Fig.12As shown in the figure, in the training confusion matrix of the model, for the multi-class classification problem of multi-scenario tunnel accident identification, the elements on the diagonal represent the percentage of accident types and levels in the tunnel scenario that are correctly classified. For example, the diagonal element of "serious collision" is 0.90, indicating that 90% of serious collision accident pictures are correctly classified. The elements on the non-diagonal lines represent the percentage of accident types and levels in the tunnel scenario that are incorrectly classified. For example, the misclassification value between "firelevel2" and "firelevel3" is 0.04, indicating that 4% of "firelevel2" targets are mistakenly identified as "firelevel3". According to the confusion matrix, the values ​​of TP, FP and FN of each category can be obtained, and the accuracy, recall and precision of the model prediction can be further calculated.

[0110] S53. In order to evaluate the detection performance of the trained tunnel accident discrimination model, it is necessary to introduce a series of evaluation index parameters.

[0111] S531. Loss function. YOLO's composite loss function, including confidence loss, classification loss, and positioning loss, covers several necessary loss function modules in target detection:

[0112]

[0113] Represents the target positioning loss of the model. The positioning loss function represents the degree of deviation between the target bounding box predicted by the model and the actual bounding box marked on the image. The deviation between the predicted box and the real box is calculated using the mean square error or absolute error. Represents the confidence loss of the model's target detection. The confidence loss function reflects the difference between the confidence predicted by the model and the actual annotation situation; Represents the model's predicted target classification loss, which is used to measure the accuracy of category prediction.

[0114] In deep learning, the loss function is an indicator to measure the accuracy of model predictions. It quantifies the error between the model's predicted value and the actual value. The smaller the value of the loss function, the smaller the difference between the model's predicted value and the actual value, so it can be considered that the performance of the model is better.

[0115] S532, IoU, or intersection over union, is used to measure the similarity between the predicted anchor box and the actual anchor box of the target.

[0116] Its value is the ratio of the intersection area of ​​the two frames to the union area.

[0117]

[0118] Among them, B is the predicted anchor box area, B gt is the actual anchor box area of ​​the target. In YOLOv5, IoU is used to determine the best matching predicted anchor box. The higher the IoU, the lower the positioning loss of the model.

[0119] S533, precision and recall. Precision is used to quantify the ratio of actual positive examples in the case of predicted positive examples. Recall is used to quantify the proportion of samples that are accurately predicted as positive examples in the actual positive examples. The corresponding formulas for the two are as follows:

[0120]

[0121] Among them, TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives. In performance evaluation, the Precision-Recall curve can be used to select the optimal IoU threshold to achieve a balance between demand and performance.

[0122] S534, AP and mAP. Average precision AP (Average Precision) is the average value of the Precision of each point on the Precision-Recall curve under different thresholds. Average precision is an indicator used to evaluate the performance of the classifier under different classification thresholds. Mean average precision mAP (mean Average Precision) is the average value of the AP values ​​corresponding to all target categories detected by the model. In the detection of multiple target categories, mAP is an important performance indicator.

[0123] S535. After obtaining the data of TP, FP, and FN, the precision and recall rates can be calculated, and the average precision curve and the precision-recall curve can be obtained. The precision, recall, and mAP values ​​during the training process can be output, and the performance changes of the model during the training process can be intuitively seen, such as Fig.13 shown.

[0124] In terms of average precision, the comprehensive performance of all categories reaches 1.00 when the confidence threshold is 0.929. At this threshold, all samples detected as positive are correct. There are also differences in recognition accuracy between different types of accidents at the same confidence threshold. The average precision of collision accidents and pedestrian recognition is high, while the average precision of primary and secondary fire recognition is relatively low. The precision-recall curve shows the relationship between the precision and recall of the model at different confidence thresholds. When the confidence is high, the model has a high precision for the detection of the fire category, but as the recall rate increases, the precision decreases rapidly, indicating that at low confidence thresholds, the model may generate more false positive examples of fire recognition. For the "minor collision" minor collision accident, the model shows a high precision, and the precision decreases less with the increase of the recall rate, which shows that the model has good detection performance for this category. The two categories of "serious collision" and "person" show high precision and relatively stable performance, and can maintain high precision even at high recall rates, and have good recognition performance. For the mean average precision (mAP) value, the model has the highest mAP value of 0.866 on the "minor collision" category, and the best overall detection performance. mAP@0.5 represents the average precision at a confidence threshold of 0.5. After 60-80 rounds of training, the mAP@0.5 value of this model is around 0.7, which shows that the multi-scenario tunnel accident discrimination model has a good average performance at a confidence threshold of 0.5.

[0125] S54. In terms of image processing speed, taking the model deployed on a device equipped with an NVIDIA GeForce GTX 1650Ti 4096MiB graphics card as an example, the processing speed is about 8-14ms per image. For 25-30 frames of highway surveillance camera video stream input, it can meet the requirements of real-time target monitoring.

[0126] S61. After obtaining the multi-scenario tunnel accident discrimination model, in order to improve the versatility of the model and enhance the convenience of user operation, the development of the detection platform GUI operation interface and the deployment test of related hardware were carried out. After studying and judging the actual usage scenarios of the tunnel accident detection platform, the overall operation logic of the tunnel accident detection platform was determined as follows: Fig.14 shown.

[0127] S62. Based on the overall operation logic of the platform, the main interface of the tunnel accident detection platform GUI developed using PyQt5 is as follows Fig.15 shown.

[0128] S63. Run the detection platform, select the multi-scenario tunnel accident discrimination model for the detection model, select the camera detection for the detection mode, and use a section of the tunnel car accident fire route for testing. In the continuous detection of video images, after the accident type is detected, the accident will be marked in the image using an anchor box, and the accident category will be marked above, and an alarm will be issued to the management personnel, such as Fig.16 Considering that the detection platform will perform a detection on each frame of the image, a timing parameter is introduced in the alarm program to set the shortest alarm interval to avoid the mutual influence of continuous alarms.

[0129] S71. After the software GUI interface is developed, it can be deployed to the hardware device for practical testing. The application scenario of the multi-scenario tunnel accident identification model studied in this paper is tunnel highway monitoring equipment. For the scenario where the computer equipment in the monitoring room of the tunnel operation and management unit manages the camera input video stream, the feasibility of the model deployment on the Windows system has been verified during the development process. When deployed on the computer equipment in the monitoring room, the model can perform real-time target detection and early warning on the input stream of the monitoring equipment in the tunnel. For field detection equipment on the side of the tunnel highway, it is necessary to consider the deployment of the model on the edge of the non-Windows system.

[0130] Jetson Orin Nano is a small, low-power development kit for edge AI applications launched by NVIDIA. It is based on the ARM architecture and integrates NVIDIA's GPU core, which is suitable for machine learning and deep learning tasks including object detection. First, complete the assembly of the motherboard and equipment, and install the necessary dependencies for YOLOv5 in the Linux system, including Python, OpenCV, PyTorch, etc., to complete the deployment of the multi-scenario tunnel accident identification system, such as Fig.17 shown.

[0131] S81. After the environment configuration is completed, the model and its related programs are deployed on the Jetson Orin Nano device. After inputting the test image file, the GPU is used for accelerated calculation. The model has achieved good results in accident detection performance. The model deployed on the edge device is used to analyze the real-time video stream and identify accident signs in the image. The model output includes bounding boxes, category labels, and confidence scores. Related deployment and test results are as follows: Fig.18 As shown in Figure 1. When a potential accident is detected, the edge device can immediately issue an alarm. The alarm can be implemented through sound and light signals, message push, or automatic notification to the control center. All detected accident events and system response actions are recorded on the edge device for subsequent review and analysis. This data can be used to improve accident detection models and optimize safety management strategies.

[0132] The advantages of deploying the model on edge devices include: low latency, local data processing, reduced transmission time, and real-time processing in situations where a quick response is required; offline work, normal detection can be performed even without a network connection; information protection, detection data does not need to be transmitted to the cloud, protecting relevant information.

[0133] The patent of this invention proposes an efficient method for distinguishing tunnel accidents by innovatively applying deep learning technology, which greatly improves the automation level of tunnel operation and management. This method constructs a multi-scenario tunnel accident data set, covering various accident types including fire, vehicle collision and pedestrians occupying the road, adopts the advanced YOLOv5 algorithm, and combines customized network structure and training strategy to achieve fast and accurate detection of accidents in tunnels. In addition, the developed software platform not only integrates model reasoning and result display functions, but also has real-time alarm triggering capabilities, making the accident response process more intuitive and convenient. By deploying the model on edge computing devices, the present invention achieves low-latency accident detection, improves response speed, and reduces dependence on centralized processing. The combination of these technologies not only improves the safety of tunnel traffic, but also provides a new direction for the development of intelligent transportation systems, demonstrating the great potential of deep learning technology in practical applications. The patent of this invention is not only innovative in technology, but also has important significance in practical applications. It has brought substantial progress to the field of tunnel safety management, helped reduce the losses caused by accidents, and improved the safety of road users.

[0134] It should be noted that the multi-scenario tunnel accident detection method based on YOLOv5 provided in the above embodiment can essentially exist in the form of a program processing flow. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the method provided in the above embodiment, which includes a memory and a processor;

[0135] The memory is used to store computer programs;

[0136] The processor is used to implement the multi-scenario tunnel accident detection method based on YOLOv5 as described above when executing the computer program.

[0137] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the method provided in the above embodiment, and a computer program is stored on the storage medium. When the computer program is executed by the processor, a multi-scenario tunnel accident detection method based on YOLOv5 as mentioned above is implemented.

[0138] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A multi-scenario tunnel accident identification method based on YOLOv5, characterized in that: Here are the steps: S1. Construct a multi-scenario tunnel accident dataset and annotate the dataset. The dataset includes fire, vehicle collision, and pedestrian occupation, and is segmented by considering the development stage and severity of the accident. S2. Use the YOLOv5 target detection algorithm to train the model on the labeled data set to obtain a model that can identify accident types such as fire, vehicle collision, and pedestrians occupying the road; S3. Develop a tunnel accident identification software platform based on PyQt5, which integrates image detection, video detection and camera real-time detection functions, and when an accident is detected, the software platform automatically issues an alarm; S4. Deploy the trained model on the edge computing device Jetson Orin Nano and perform performance testing. After the test is completed, run the deployed model in the tunnel monitoring environment to detect accidents in real-time video streams. When an accident is detected, the system automatically issues an alarm and provides detailed information about the accident to the monitoring personnel.

2. According to the multi-scenario tunnel accident identification method based on YOLOv5 in claim 1, it is characterized in that: In step S1, constructing a multi-scenario tunnel accident dataset includes the following steps: Determine all the types of accidents that are expected to be identified, including fire, vehicle collision, and pedestrians occupying the road, and subdivide them according to the development stage and severity of the accident; Collect original image materials, write crawler scripts using Python, search and download original image sets corresponding to accident categories, and perform data screening to remove images that do not meet the requirements; Perform data augmentation on images to improve the generalization ability of the model; Use the LabelImg tool to label the data set, create a class.txt file, and input all accident label categories; Review the annotated images and corresponding labels to ensure the accuracy and consistency of the annotations, ensure that all image materials have corresponding annotation files, and keep the file names consistent.

3. According to the multi-scenario tunnel accident identification method based on YOLOv5 in claim 2, it is characterized in that: The specific steps to determine all the types of accidents that are expected to be identified are as follows: Fire accidents are divided into three levels according to the development stage and the severity of the combustion, including smoke in the initial stage, open flames in the development and spread stage, and severe fires with intense combustion and thick black smoke; The vehicle collision accident is divided into two scenarios: minor collision and severe collision according to the severity of the collision; Pedestrians occupying the road accident is defined as an unexpected incident in which pedestrians appear on the motor vehicle lane in a tunnel.

4. According to a multi-scenario tunnel accident identification method based on YOLOv5 according to claim 2, it is characterized in that: The specific steps for collecting original image materials and screening data are as follows: By writing a Python crawler script, we searched for the keywords "tunnel accident", "tunnel fire", "highway fire", and "highway collision" and downloaded the original image sets of the corresponding accident categories; Classify the downloaded image sets and check whether the image quality meets the production requirements of the data set, and eliminate blurred, overexposed, underexposed images and images that do not meet the accident type requirements.

5. According to a multi-scenario tunnel accident identification method based on YOLOv5 according to claim 1, it is characterized in that: The specific steps for calibrating the dataset are as follows: Use the LabelImg tool to label the YOLO format data set, create a class.txt file, and input all accident label categories; Anchor boxes are annotated for each image in the image set to ensure that the anchor boxes fit closely to the edges of the target objects to improve the accuracy of detection. Different fire level features that may exist simultaneously in fire accident pictures are selected and calibrated one by one.

6. According to a multi-scenario tunnel accident identification method based on YOLOv5 according to claim 1, it is characterized in that: In step S2, the specific parameters of model training are set as follows: The input image size is 640*640, the weight file is YOLOv5s.pt, the batch size is 8, the learning rate is 0.01, and the training cycle is 100.

7. The multi-scenario tunnel accident identification method based on YOLOv5 according to claim 1 is characterized in that: The following evaluation indicators are used to evaluate the performance of the accident type model obtained through training: Loss function L, including confidence loss, classification loss and positioning loss; The intersection over union (IoU) is used to measure the similarity between the predicted anchor box and the actual anchor box of the target; Precision and recall are used to quantify the detection accuracy of the model; The corresponding formulas are as follows: Among them, TP is the number of true positive examples, FP is the number of false positive examples, and FN is the number of false negative examples. In the performance evaluation, the Precision-Recall curve is used to select the optimal IoU threshold to achieve a balance between demand and performance; The average precision (AP) and mean average precision (mAP) are used to evaluate the performance of the model under different classification thresholds.

8. According to a multi-scenario tunnel accident identification method based on YOLOv5 according to claim 1, it is characterized in that: The specific steps to deploy the trained model on the edge computing device Jetson Orin Nano and perform performance testing are as follows: Environment configuration: Configure the necessary operating environment on Jetson Orin Nano, including installing Python, OpenCV, and PyTorch libraries to ensure that the model can run normally; Model deployment: Deploy the trained YOLOv5 model and related programs to the device to ensure that the model can process the video stream from the surveillance camera in real time; Performance testing: Test the performance of the model on edge devices to evaluate its actual application effect in low-latency and offline working environments.

9. A multi-scenario tunnel accident identification device based on YOLOv5, characterized in that: include: Deep learning target detection model based on YOLOv5; A processor, configured to implement the multi-scenario tunnel accident identification method according to any one of claims 1 to 8 when executing the computer program; Memory, used to store computer programs.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the multi-scenario tunnel accident identification method as described in any one of claims 1-8 is implemented.

Images