Electric two-wheeled vehicle safety monitoring method and system based on deep learning

Abstract

The invention discloses an electric two-wheeled vehicle safety monitoring method and system based on deep learning, and the method carries out the target classification detection and multi-target tracking of an electric two-wheeled vehicle through the deep learning technology, achieves the recognition of an electric vehicle and passengers, effectively improves the traffic safety level, and improves the safety of the electric vehicle. According to the system, the driving condition of the electric vehicle can be monitored in real time, an illegal behavior can be warned in time, and a management department can more effectively cope with potential risks, optimize urban traffic management, improve the overall traffic efficiency and ensure the safety of pedestrians and vehicles by collecting and analyzing related data; through the target detection technology based on the RT-DETR model, the key safety information of the speed, the number of riders and the helmet wearing condition of the electric two-wheeled vehicle can be accurately detected in real time, and accurate traffic monitoring and safety early warning are provided, so that the safety supervision level of the electric two-wheeled vehicle is effectively improved; innovation is provided for urban traffic management and construction of an intelligent traffic system.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more specifically, to a safety monitoring method and system for an electric two-wheeled vehicle based on deep learning. Background Art

[0002] The core of the target detection task is to accurately identify and locate multiple objects in complex scenes and assign accurate category labels to each object. This task occupies a vital position in the field of computer vision and has promoted the development of many cutting-edge applications, such as autonomous driving, intelligent video surveillance, public safety, intelligent manufacturing, and medical image analysis. With the advancement of science and technology, while electric two-wheelers provide people with a fast and convenient way of travel, their safety issues have also attracted great attention from society and public security departments. This phenomenon has prompted researchers to think deeply about how to build a smart transportation system that is deeply integrated with the mobile Internet, so as to accurately detect electric two-wheelers, helmet wearing conditions, and whether they are overloaded in complex traffic environments. This has an important role in promoting the development of industries such as smart security, safe cities, smart transportation, smart cities, and unmanned driving, and therefore has significant scientific research and practical application value.

[0003] At present, the intelligent traffic detection systems in most cities mainly focus on the road driving conditions of motor vehicles. However, with the advancement of technology and the improvement of environmental awareness, more and more people choose electric two-wheeled vehicles as their daily means of transportation, which also leads to the increasing frequency of traffic accidents involving electric two-wheeled vehicles. Therefore, the establishment of a mature electric two-wheeled vehicle safety detection system has become an indispensable part of the modern intelligent transportation system.

[0004] At present, with the popularity of electric two-wheeled vehicles, traffic accidents involving electric two-wheeled vehicles are increasing, especially in urban traffic environments. Existing traffic monitoring systems mainly focus on the safety supervision of motor vehicles, but not enough attention is paid to the monitoring of electric two-wheeled vehicles, and there is a lack of safety detection technology for electric two-wheeled vehicles. In the existing technology, although some intelligent traffic monitoring systems have introduced target detection technology, due to the dynamic characteristics of electric two-wheeled vehicles and riders, these systems perform poorly in complex urban traffic environments and cannot accurately detect the real-time status of electric two-wheeled vehicles (such as vehicle speed, helmet wearing status, whether it is overloaded, etc.), resulting in loopholes in traffic management and safety monitoring.

[0005] Therefore, in response to the above technical problems, it is necessary to provide an electric two-wheeled vehicle safety monitoring method and system based on deep learning. Summary of the invention

[0006] The purpose of the present invention is to provide an electric two-wheeled vehicle safety monitoring method and system based on deep learning to solve the above-mentioned problems.

[0007] In order to achieve the above purpose, the technical solution provided by an embodiment of the present invention is as follows:

[0008] A method and system for safety monitoring of an electric two-wheeled vehicle based on deep learning, comprising the following steps:

[0009] S1, video image acquisition, through the front-end integrated camera or video file input, use the video decoder to decode the acquired video stream, and convert the video frame into a three-channel image in RGB format;

[0010] S2, using the manual labeling method, using the open source labeling tool Labeling to carefully label the three-channel images obtained in S1, accurately marking the key categories of electric two-wheelers, people on the vehicle, and helmets in the image to ensure that each target is accurately identified and classified;

[0011] S3, send the labeled sample set in S2 to the target detection model based on RT-DETR for training, use the deep learning framework, combine the ResNet classic convolutional network as the backbone network for feature extraction, and then learn the target information in the image through the encoder and decoder of the Transformer architecture;

[0012] S4. Deploy the trained electric two-wheeled vehicle detection model to the front-end interface, perform target detection on the images in the real-time video stream, identify the electric two-wheeled vehicle, the person on the vehicle, and the helmet target, and use the classifier to further distinguish the electric two-wheeled vehicle, whether the driver is wearing a helmet, and the number of people on the vehicle based on the location information of the target;

[0013] S5. On the front-end interface, the original image and the detection result image, as well as the original video and the detection result video are displayed simultaneously, providing users with intuitive detection result feedback and allowing users to adjust the confidence score on the interface to optimize the accuracy of the detection results. The system also supports exporting the detection results into image and video formats.

[0014] As a further improvement of the present invention, in step S1, the video is captured using 8 professional shooting devices in different environments, and the video resolution is 1920*1080.

[0015] As a further improvement of the present invention, the data in step S2 is annotated using the open source annotation tool labelme, and 8 professionals are hired to annotate. 10,000 original images are screened and double-checked, and finally 6,000 complete, high-quality pixel-level annotated data are obtained.

[0016] As a further improvement of the present invention, the specific training steps of the target detection model of RT-DETR in step S3 are as follows:

[0017] S301. Select the classic ResNet as the backbone and use only the outputs of the last three stages.

[0018] S302. In terms of feature extraction, RT-DETR uses a layer of Transformer encoder, which only processes the S5 feature layer output by the backbone network, namely the AIFI module;

[0019] S303. In the decoding stage, RT-DETR uses a Transformer-based decoder to perform the final detection and positioning of the target;

[0020] S304. According to the actual training situation, it is necessary to adjust the learning rate, batch size and Epochs hyperparameters to optimize the training effect and convergence speed of the model.

[0021] As a further improvement of the present invention, during the training process of the target detection model of RT-DETR, a set of person bounding boxes P = {p1, p2, p3, ...} is obtained, where p i =(x c ,y c ,w,h) respectively represent the coordinates of the center of the person frame in the original image and the width and height. In addition, the electric two-wheeled vehicle bounding box set M = {m1,m2,m3,...} is obtained for each m i ∈M where m i =(x′ c ,y′ c ,w′,h′), each electric two-wheeled vehicle frame m i For each electric two-wheeled vehicle frame m i , there exists a person box p j Make the area of ​​the person frame larger than the area of ​​the electric two-wheeled vehicle frame by a threshold a, and then j Included in the electric two-wheeled vehicle frame m i The corresponding cumulative number of personnel.

[0022] As a further improvement of the present invention, the formulas obtained from the statistics of people on the vehicle during the training process of the target detection model of RT-DETR include:

[0023] The area of ​​the personnel frame is: Area(p j )=w j *h j ;

[0024] The area of ​​the electric two-wheeled vehicle frame is: Area (m i )=w i ′*h i ′;

[0025] For each electric two-wheeled vehicle frame m i , calculate each person box p j The area ratio of the following formula is expressed, where the personnel box statistics that meet the conditions, if Ratio(p j ,m i )>a, then the personnel frame p j Considered to be in the electric two-wheeled vehicle frame i superior

[0026] The cumulative number of people on the electric two-wheeled vehicle frame for each m i , calculate the number of people boxes N(m i ),Right now:

[0027] As a further improvement of the present invention, the speed detection in the target detection model training process of the RT-DETR includes:

[0028] In each frame t and t+1, the RT-DETR model detects the bounding box of the electric two-wheeler, and each bounding box is represented by m t =(x c ,y c ,w,h), where and x c ,y c are the center coordinates of the bounding box, w and h are the width and height of the box;

[0029] For two image frames, frame t and frame t+1, the bounding box of the electric two-wheeler output by RT-DETR is m t and m t+1 , where t and t+1 represent the time of the current frame and the next frame respectively;

[0030] Calculate the coordinate change of the center of the bounding box: For the bounding boxes of the electric two-wheeled vehicle in the t-th frame and the t+1-th frame, calculate the change of their center coordinates as follows:

[0031] Calculate the moving distance: The moving distance d of the electric two-wheeled vehicle is the distance between the center points in the two-dimensional space. According to the Euclidean distance formula, the moving distance of the electric two-wheeled vehicle between two frames is calculated:

[0032] Calculate the speed. The time interval between frames is Δt. The speed v of the electric two-wheeler is the distance moved per unit time, which can be calculated by the following formula:

[0033] As a further improvement of the present invention, in step S4, the trained model is deployed to the front end, and the RT-DETR model is converted through the TensorFlow framework and directly deployed to the front end.

[0034] As a further improvement of the present invention, in step S5, a modern PyQt5 framework is selected to construct the front-end interface.

[0035] A deep learning-based electric two-wheeled vehicle safety monitoring system comprises the deep learning-based electric two-wheeled vehicle safety monitoring method.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] (1) This solution uses deep learning technology to perform target classification detection and multi-target tracking on electric two-wheelers, comprehensively solving the problem of identifying electric vehicles and their occupants, including monitoring the number of people on the vehicle and whether it is driving in the wrong direction. It effectively improves the level of traffic safety, can monitor the driving conditions of electric vehicles in real time, and issue timely warnings for violations. By collecting and analyzing relevant data, management departments can more effectively respond to potential risks, optimize urban traffic management, improve overall traffic efficiency, and ensure the safety of pedestrians and vehicles.

[0038] (2) This solution uses target detection technology based on the RT-DETR model to accurately detect key safety information such as the speed, number of riders, and helmet wearing status of electric two-wheelers in real time, and provide precise traffic monitoring and safety warnings, effectively improving the safety supervision level of electric two-wheelers and providing innovation for urban traffic management and the construction of intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a process of an embodiment of the present invention;

[0040] Figure 2 Prepare a schematic diagram for the implementation of the data set of the present invention;

[0041] Figure 3 A schematic diagram of a model training process in an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of the front-end interface in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments, and all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0044] Embodiment 1:

[0045] See also Figure 1 , a safety monitoring method and system for an electric two-wheeled vehicle based on deep learning, comprising the following steps:

[0046] S1, video image acquisition, through the front-end integrated camera or video file input, use the video decoder to decode the acquired video stream, convert the video frame into a three-channel image in RGB format, and provide raw data for subsequent target detection;

[0047] S2, using the manual labeling method, using the open source labeling tool Labeling to carefully label the three-channel images obtained in S1, accurately marking the key categories of electric two-wheeled vehicles, people on the vehicle, and helmets in the images, ensuring that each target is accurately identified and classified, thereby constructing a high-quality electric two-wheeled vehicle sample set and providing the necessary labeled data for model training;

[0048] S3, send the labeled sample set in S2 to the target detection model based on RT-DETR for training, use the deep learning framework, combine the ResNet classic convolutional network as the backbone network for feature extraction, and then learn the target information in the image through the encoder and decoder of the Transformer architecture, and finally train a detection model that can accurately identify electric two-wheeled vehicles, the number of people and helmets on the vehicle, and the speed;

[0049] S4. Deploy the trained electric two-wheeled vehicle detection model to the front-end interface, perform target detection on the images in the real-time video stream, identify the electric two-wheeled vehicle, the person on the vehicle, and the helmet target, and use the classifier to further distinguish the electric two-wheeled vehicle, whether the driver is wearing a helmet, and the number of people on the vehicle based on the location information of the target, so as to provide real-time data analysis for traffic management and safety monitoring;

[0050] S5. On the front-end interface, the original image and the detection result image, as well as the original video and the detection result video are displayed simultaneously, providing users with intuitive detection result feedback and allowing users to adjust the confidence score on the interface to optimize the accuracy of the detection results. The system also supports exporting the detection results into image and video formats, which is convenient for users to conduct subsequent data analysis and recording.

[0051] In step S1, 8 professional cameras are used to capture videos in different environments, and the video resolution is 1920*1080. In order to improve the generalization ability of the model surface in different environments, 8 professional cameras are used to capture videos in different environments, and the video resolution is 1920*1080. Then, a video decoder is used to decode the acquired video stream, and the video frame is converted into a three-channel image in RGB format to provide raw data for subsequent target detection.

[0052] See also Figure 2In step S2, the data is annotated using the open source annotation tool labelme. Eight professionals are hired to annotate. 10,000 original images are screened and checked twice, and finally 6,000 complete, high-quality pixel-level annotated data are obtained. The three-channel images obtained in S1 are carefully annotated using the open source annotation tool Labelimg by using the manual labeling method. Key categories such as electric two-wheeled vehicles, people on the vehicle, and helmets are accurately marked in the image to ensure that each target is accurately identified and classified, thereby constructing a high-quality electric two-wheeled vehicle sample set to provide the necessary annotation data for model training. The TDETW dataset contains different perspectives and multiple scenes, with a total of 6,000 pixel-level annotated images of categories such as electric two-wheeled vehicles, drivers, and helmets. The data is annotated using the open source annotation tool labelme. Eight professionals are hired to annotate. 10,000 original images are screened and checked twice, and finally 6,000 complete, high-quality pixel-level annotated data are obtained.

[0053] See also Figure 3 ,The specific training steps of the RT-DETR target detection model in step S3 are as follows:

[0054] S301. Select the classic ResNet as the backbone and use only the outputs of the last three stages.

[0055] S302. In terms of feature extraction, RT-DETR uses a layer of Transformer encoder, which only processes the S5 feature layer output by the backbone network, namely the AIFI module;

[0056] S303. In the decoding stage, RT-DETR uses a Transformer-based decoder to perform the final detection and positioning of the target;

[0057] S304. According to the actual training situation, it is necessary to adjust the learning rate, batch size and Epochs hyperparameters to optimize the training effect and convergence speed of the model;

[0058] The classic ResNet is selected as the Backbone, and only the output of its last three stages is used. This is because the later layers of ResNet can effectively extract high-level semantic features while maintaining a high spatial resolution, which is suitable for accurately detecting targets such as electric two-wheelers, riders and helmets. Compared with using the output of the entire network, focusing on the features of the last three stages not only improves the model's recognition ability for complex scenes, but also significantly reduces the amount of calculation and improves the detection speed. In terms of feature extraction, RT-DETR uses a layer of Transformer encoder to process only the S5 feature layer output by the backbone network, namely the AIFI module. The core idea of ​​this design is to use a single-scale Transformer encoder to perform self-attention interactions within the scale only on the S5 feature layer, thereby maintaining computational efficiency while improving the model's ability to capture high-level semantic information. As the last layer of the backbone network, the S5 feature already contains relatively rich semantic information and contextual relationships. Therefore, Transformer encoding at this layer can effectively capture the spatial relationship and conceptual connection between targets, which is especially important for the positioning and recognition of electric two-wheelers, riders and helmets. In this way, the RT-DETR model can reduce computational costs while still being able to This design not only improves the efficiency of the model, but also enables subsequent modules to detect and classify targets more accurately, especially in complex scenarios, and can better handle the interactions and relationships between different targets. In the decoding stage, RT-DETR uses a Transformer-based decoder to perform final detection and positioning of the target. The input of the decoder is the feature representation processed by the Transformer encoder. The decoder locates targets such as electric two-wheelers, riders, and helmets by interacting with the query vector. Each query vector represents a potential target. By performing attention calculations with the encoded features, the decoder can effectively extract the precise location and category of each target from them. This process does not rely on multi-scale features, but focuses on deep semantic reasoning at a single scale (S5 layer features), further reducing the computational cost. Depending on the actual training situation, we may need to adjust hyperparameters such as learning rate, batch size, and Epochs to optimize the training effect and convergence speed of the model. Through training, we finally obtain a detection model that can accurately identify electric two-wheelers, people on board, and helmets. This model can process multi-scale features, effectively capture target information at different scales, and reduce computational complexity.

[0059] During the training process of the target detection model of RT-DETR, the set of person bounding boxes P = {p1, p2, p3, ...} is obtained, where p i =(x c ,yc ,w,h) respectively represent the coordinates of the center of the person frame in the original image and the width and height. In addition, the electric two-wheeled vehicle bounding box set M = {m1,m2,m3,...} is obtained for each m i ∈M where m i =(x′ c ,y′ c ,w′,h′), each electric two-wheeled vehicle frame m i For each electric two-wheeled vehicle frame m i , there exists a person box p j Make the area of ​​the person frame larger than the area of ​​the electric two-wheeled vehicle frame by a threshold a, and then j Included in the electric two-wheeled vehicle frame m i The corresponding cumulative number of personnel.

[0060] The formulas obtained for the statistics of people on board during the training of the RT-DETR target detection model include:

[0061] The area of ​​the personnel frame is: Area(p j )=w j *h j ;

[0062] The area of ​​the electric two-wheeled vehicle frame is: Area (m i )=w i ′*h i ′;

[0063] For each electric two-wheeled vehicle frame m i , calculate each person box p j The area ratio of the following formula is expressed, where the personnel box statistics that meet the conditions, if Ratio(p j ,m i )>a, then the personnel frame p j Considered to be in the electric two-wheeled vehicle frame i superior

[0064] The cumulative number of people on the electric two-wheeled vehicle frame for each m i , calculate the number of people boxes N(m i ),Right now:

[0065] During the training process of the RT-DETR target detection model, speed detection includes:

[0066] In each frame t and t+1, the RT-DETR model detects the bounding box of the electric two-wheeler, and each bounding box is represented by m t =(x c ,y c,w,h), where and x c ,y c are the center coordinates of the bounding box, w and h are the width and height of the box;

[0067] For two image frames, frame t and frame t+1, the bounding box of the electric two-wheeler output by RT-DETR is m t and m t+1 , where t and t+1 represent the time of the current frame and the next frame respectively;

[0068] Calculate the coordinate change of the center of the bounding box: For the bounding boxes of the electric two-wheeled vehicle in the t-th frame and the t+1-th frame, calculate the change of their center coordinates as follows:

[0069] Calculate the moving distance: The moving distance d of the electric two-wheeled vehicle is the distance between the center points in the two-dimensional space. According to the Euclidean distance formula, the moving distance of the electric two-wheeled vehicle between two frames is calculated:

[0070] Calculate the speed. The time interval between frames is Δt. The speed v of the electric two-wheeler is the distance moved per unit time, which can be calculated by the following formula:

[0071] The speed of the electric two-wheeled vehicle is calculated by the position change of the bounding box of the electric two-wheeled vehicle in two consecutive frames, and the calculation of the speed depends on the position change and the time interval.

[0072] In step S4, the trained model is deployed to the front end, and the RT-DETR model is converted and directly deployed to the front end through the TensorFlow framework; the trained model is deployed to the front end. The key to integrating model deployment and front-end is to ensure that the RT-DETR target detection model can run efficiently on the browser side, focusing on real-time detection of electric two-wheeled vehicles, people on the vehicle, and helmet wearing conditions, and interacting seamlessly with the front-end interface. In order to eliminate dependence on back-end server computing and reduce latency, we convert the RT-DETR model through the TensorFlow framework and deploy it directly to the front end; in the real-time monitoring scenario of electric two-wheeled vehicles, real-time processing of image and video data is crucial. The goal of system design is to ensure that when a user shoots an electric two-wheeled vehicle on the road through a camera, the model can quickly and accurately identify the electric two-wheeled vehicle, the rider, and the helmet wearing status, and mark the detection in the image. The system first uses a deep learning model to analyze the input image or video stream in real time, and uses the features learned during the training process to accurately identify targets such as electric two-wheeled vehicles, people on the vehicle, and helmets. The success of this step directly depends on the accuracy and inference speed of the model, which will affect the accuracy of subsequent analysis and the response time of the system. In addition, the system also has the function of determining whether the driver is wearing a helmet. This function is achieved by analyzing the features of the person's head area. The model can distinguish between the state of wearing a helmet and the state of not wearing a helmet, which is of great significance for improving road safety. At the same time, the system can track the position changes of the electric two-wheeled vehicle in continuous video frames, thereby calculating the speed of the electric two-wheeled vehicle. This function is not only of great value for real-time traffic monitoring, but also provides data support for traffic management and planning.

[0073] See also Figure 4In step S5, we select the modern PyQt5 framework to build the front-end interface. In terms of front-end interface development, we choose the modern PyQt5 framework to build an interface that is both user-friendly and responsive. PyQt5 is known for its powerful componentization capabilities and efficient data binding mechanism, which enables the interface to maintain good adaptability on different devices and screen sizes while ensuring a smooth interactive experience. To achieve this goal, we use the rich components provided by PyQt5, such as QLabel, QGraphicsView, etc. to build the detection result display, video playback and user interaction parts. The core function of the interface design is real-time video stream processing and dynamic display of target detection results. In the video stream processing part, we use PyQt5 modules such as QCamera and QMediaPlayer. OpenCV is used for video capture and image processing to ensure that the video data stream from the camera can be acquired and processed in real time. The target detection results are drawn to the video frame through PyQt5's QPainter, marking the position and confidence of the electric two-wheeled vehicle, the people on the vehicle, and the wearing of the helmet. In order to optimize the user experience, we designed a simple and intuitive interface. Users can control the confidence threshold by adjusting a slider. This operation is implemented through PyQt5's QSlider control, allowing users to adjust the detection sensitivity in real time according to actual needs. The adjusted threshold will directly affect the target detection result, ensuring that the best detection accuracy can be obtained in different monitoring environments. The threshold adjustment process is closely integrated with the TensorFlow model reasoning part to ensure that the system updates the detection results in real time according to user settings.

[0074] A deep learning-based electric two-wheeled vehicle safety monitoring system includes a deep learning-based electric two-wheeled vehicle safety monitoring method.

[0075] It can be seen from the above technical solutions that the present invention has the following beneficial effects:

[0076] The present invention uses deep learning technology to perform target classification detection and multi-target tracking on electric two-wheeled vehicles, which comprehensively solves the problem of identifying electric vehicles and their occupants, including monitoring the number of people on the vehicle and whether it is traveling in the wrong direction, effectively improving the level of traffic safety, and being able to monitor the driving conditions of electric vehicles in real time, and promptly warn of violations. By collecting and analyzing relevant data, management departments can more effectively respond to potential risks, optimize urban traffic management, improve overall traffic efficiency, and ensure the safety of pedestrians and vehicles. Through target detection technology based on the RT-DETR model, key safety information such as the speed, number of riders, and helmet wearing conditions of electric two-wheeled vehicles can be accurately detected in real time, and accurate traffic monitoring and safety warnings can be provided, effectively improving the safety supervision level of electric two-wheeled vehicles, and providing innovation for urban traffic management and the construction of intelligent transportation systems.

[0077] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0078] In addition, it should be understood that although the present specification is described according to embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that those skilled in the art can understand.

Claims

1. A safety monitoring method for electric two-wheeled vehicles based on deep learning, characterized in that: The following steps are involved: S1, video image acquisition, through the front-end integrated camera or video file input, use the video decoder to decode the acquired video stream, and convert the video frame into a three-channel image in RGB format; S2, using the manual labeling method, using the open source labeling tool Labeling to carefully label the three-channel images obtained in S1, accurately marking the key categories of electric two-wheelers, people on the vehicle, and helmets in the image to ensure that each target is accurately identified and classified; S3, send the labeled sample set in S2 to the target detection model based on RT-DETR for training, use the deep learning framework, combine the ResNet classic convolutional network as the backbone network for feature extraction, and then learn the target information in the image through the encoder and decoder of the Transformer architecture; S4. Deploy the trained electric two-wheeled vehicle detection model to the front-end interface, perform target detection on the images in the real-time video stream, identify the electric two-wheeled vehicle, the person on the vehicle, and the helmet target, and use the classifier to further distinguish the electric two-wheeled vehicle, whether the driver is wearing a helmet, and the number of people on the vehicle based on the location information of the target; S5. On the front-end interface, the original image and the detection result image, as well as the original video and the detection result video are displayed simultaneously, providing users with intuitive detection result feedback and allowing users to adjust the confidence score on the interface to optimize the accuracy of the detection results. The system also supports exporting the detection results into image and video formats.

2. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 1, characterized in that: In step S1, the video is captured using 8 professional shooting devices in different environments, and the video resolution is 1920*1080.

3. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 1, characterized in that: The data in step S2 is annotated using the open source annotation tool labelme, and 8 professionals are hired to annotate. 10,000 original images are screened and double-checked, and finally 6,000 complete, high-quality pixel-level annotated data are obtained.

4. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 1, characterized in that: The specific training steps of the target detection model of RT-DETR in step S3 are as follows: S301. Select the classic ResNet as the backbone and use only the outputs of the last three stages. S302. In terms of feature extraction, RT-DETR uses a layer of Transformer encoder, which only processes the S5 feature layer output by the backbone network, namely the AIFI module; S303. In the decoding stage, RT-DETR uses a Transformer-based decoder to perform the final detection and positioning of the target; S304. According to the actual training situation, it is necessary to adjust the learning rate, batch size and Epochs hyperparameters to optimize the training effect and convergence speed of the model.

5. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 4 is characterized in that: The target detection model training process of RT-DETR obtains a set of person bounding boxes P = {p1, p2, p3, ...}, where p i =(x c ,y c ,w,h) respectively represent the coordinates of the center of the person frame in the original image and the width and height. In addition, the electric two-wheeled vehicle bounding box set M = {m1,m2,m3,...} is obtained for each m i ∈M where m i =(x′ c ,y′ c ,w′,h′), each electric two-wheeled vehicle frame m i For each electric two-wheeled vehicle frame m i , there exists a person box p j Make the area of ​​the person frame larger than the area of ​​the electric two-wheeled vehicle frame by a threshold a, and then j Included in the electric two-wheeled vehicle frame m i The corresponding cumulative number of personnel.

6. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 5, characterized in that: The formulas obtained for the statistics of people on the vehicle during the training of the RT-DETR target detection model include: The area of ​​the personnel frame is: Area(p j )=w j *h j ; The area of ​​the electric two-wheeled vehicle frame is: Area (m i )=w i ′*h i ′; For each electric two-wheeled vehicle frame m i , calculate each person box p j The area ratio of the following formula is expressed, where the personnel box statistics that meet the conditions, if Ratio(p j ,m i )>a, then the personnel frame p j Considered to be in the electric two-wheeled vehicle frame i superior The cumulative number of people on the electric two-wheeled vehicle frame for each m i , calculate the number of people boxes N(m i ),Right now:

7. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 6, characterized in that: The speed detection process of the RT-DETR target detection model training process includes: In each frame t and t+1, the RT-DETR model detects the bounding box of the electric two-wheeler, and each bounding box is represented by m t =(x c ,y c ,w,h), where and x c ,y c are the center coordinates of the bounding box, w and h are the width and height of the box; Two frames of images, frame t and frame t+1, the bounding box of the electric two-wheeler output by RT-DETR is m t and m t+1 , where t and t+1 represent the time of the current frame and the next frame respectively; Calculate the coordinate change of the center of the bounding box: For the bounding boxes of the electric two-wheeled vehicle in the t-th frame and the t+1-th frame, calculate the change of their center coordinates as follows: Calculate the moving distance: The moving distance d of the electric two-wheeled vehicle is the distance between the center points in the two-dimensional space. According to the Euclidean distance formula, the moving distance of the electric two-wheeled vehicle between two frames is calculated: Calculate the speed. The time interval between frames is Δt. The speed v of the electric two-wheeler is the distance moved per unit time, which can be calculated by the following formula:

8. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 1, characterized in that: In step S4, the trained model is deployed to the front end, and the RT-DETR model is converted through the TensorFlow framework and directly deployed to the front end.

9. The electric two-wheeled vehicle safety monitoring method based on deep learning according to claim 1, characterized in that: In step S5, the modern PyQt5 framework is selected in the front-end interface for construction.

10. An electric two-wheeled vehicle safety monitoring system based on deep learning, characterized in that: It includes a video image acquisition module, a deep learning algorithm module and a data processing module, which are used to implement the electric two-wheeled vehicle safety monitoring method based on deep learning as described in claims 1-9.

Images