Electric vehicle overload behavior detection method based on YOLOv5 and Deepsort

Abstract

The invention relates to an electric vehicle overload behavior detection method based on YOLOv5 and Deepsort. The method solves the problems in the prior art that the judgment of the overload of the electric vehicle by a visual identification algorithm is easily influenced by traffic conditions, weather conditions and quantity, and the identification accuracy is poor. The method comprises the following steps: S1, establishing a YOLOv5 algorithm database; s2, a Deepsort algorithm database is established; s3, overload detection of the electric vehicle; and S4, obtaining an overload electric vehicle behavior face. The system has the advantages that video images shot by road monitoring can be detected, the overload behavior of the electric vehicle can be automatically found, the time and place for finding the overload behavior can be recorded, the identity information of a driver can be obtained through shot face images, education and punishment can be carried out, and therefore a large amount of manpower resources of traffic police can be saved; safety driving awareness of the public is improved, and major accidents caused by overloading of the electric vehicle are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of electric vehicle passenger detection, and particularly relates to a method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort. Background Technique

[0002] When an electric vehicle is overloaded with passengers, the inertia of the vehicle will increase, and the braking distance will be greatly increased. Once an emergency occurs, the vehicle cannot stop within a safe distance, which is likely to lead to traffic accidents. By using road cameras to detect the overloading behavior of electric vehicles, the driving situation of people on electric vehicles can be monitored in real time, and then the information system can be used to remind people of their safety driving awareness and reduce accidents during safe travel.

[0003] Traditional detection of overloading of electric vehicles basically relies on traffic police to conduct on-site inspections at intersections. However, relying on manual inspections will consume a large amount of precious traffic police resources, and when overloaded passengers find that there are traffic police conducting inspections from a long distance, they may get off the vehicle in advance to avoid punishment. With the continuous development of artificial intelligence, its application in life and production is becoming more and more extensive. Now, some detection technologies for illegal behaviors of electric vehicles based on various deep learning algorithms have been introduced on the market. Most of them rely on road cameras as front-end devices to obtain intersection images, then use algorithms in computer vision to preprocess the images, and finally detect illegal behaviors of electric vehicles in the images. To protect the privacy rights of the public, the monitoring cameras on the road are controlled by the traffic management department and cannot be installed randomly, resulting in difficulty in obtaining a large number of electric vehicle images. Moreover, the actual road environment is very complex and is easily affected by lighting conditions, weather changes, vehicle occlusion, etc. When the amount of data used for training is insufficient, the model will not be able to accurately identify. In addition to electric vehicles, bicycles, tricycles, wheelchairs, baby carriages, etc. may also appear in the images, causing misidentification by the algorithm. It can be seen that conventional visual recognition algorithms are prone to be affected by traffic conditions, weather conditions and quantity when judging the overloading of electric vehicles, and the recognition accuracy is not good.

[0004] In order to solve the deficiencies of the existing technology, people have conducted long-term explorations and proposed various solutions. For example, a Chinese patent document discloses a method and device for detecting overloading of electric bicycles and an electric bicycle [CN202111197444.5], which includes that a plurality of distance sensors are arranged around the seat of the electric bicycle. The method includes: obtaining a first signal reflecting the load of the electric bicycle; obtaining a second signal output by the distance sensor, and the second signal represents the detection distance between the target object around the seat and the distance sensor; according to the first signal and the second signal, detecting whether an overloading event occurs, where the overloading event is an event that the number of passengers on the electric bicycle exceeds the set upper limit of the number of passengers; in the case of detecting that an overloading event occurs, performing a set operation in response to the overloading event.

[0005] The above solution solves to some extent the problem of poor safety of electric vehicles carrying passengers in the prior art. However, there are still many deficiencies in this solution. For example, the judgment of electric vehicle overloading is affected by traffic conditions, weather conditions, and quantity, and the recognition accuracy is not good. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort in view of the above problems.

[0007] To achieve the above object, the present invention adopts the following technical solutions: A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort includes the following steps:

[0008] S1. Establish a YOLOv5 algorithm database;

[0009] S2. Establish a Deepsort algorithm database;

[0010] S3. Detect overloading of electric vehicles;

[0011] S4. Obtain the faces of the occupants of overloaded electric vehicles.

[0012] In the above method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort, step S1 includes the following steps:

[0013] S11. Use the YOLOv5 algorithm to detect the types of objects appearing in the road image,

[0014] S12. Obtain electric vehicle images from different road 4K high-definition cameras at different time points;

[0015] S13. Label the image data by combining manual and automatic annotation;

[0016] S14. Train the YOLOv5 model using data augmentation;

[0017] S15. Train to obtain the YOLOv5 model as the YOLOv5 algorithm database to detect various object types in the road image;

[0018] S16. Record the detection results for subsequent determination of electric vehicle overloading.

[0019] In step S11, the object types include object types and helmet wearing status; among them, the object types include motor vehicles, bicycles, cars, pedestrians, wheelchairs, and baby carriages, and the helmet wearing status includes wearing a helmet and not wearing a helmet; in step S12, the electric vehicle images include images of electric vehicles in the unloaded and overloaded states and two state images of the riders wearing helmets.

[0020] In the above method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort, step S2 includes the following steps:

[0021] S21. At different time points, obtain images with complexity and quantity from different road 4K high-definition cameras;

[0022] S22. Label the images by combining manual and automatic annotation methods;

[0023] S23. Use the labeled data to train the Deepsort model.

[0024] In steps S13 and S22, when annotating image data, the electric vehicle and its riders are labeled as a whole and defined as EBicycleCyclist, the human head is defined as the Head class, and the human head wearing a helmet is defined as the Helmet class. After annotation, according to the ratio, they are randomly assigned to the training set, test set, and validation set for model training.

[0025] In the above method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort, step S14 specifically includes the following steps:

[0026] S141. Perform operations such as rotation, scaling, splicing, and cropping on the images to increase the data volume for training the initial YOLOv5 model;

[0027] S142. Use the initial YOLOv5 model to detect the remaining images and output the coordinate positions of the detected objects;

[0028] S143. Manually adjust the coordinate positions, add objects that are missed in annotation, and delete and correct mis-annotated objects;

[0029] S144. Use all the data to set different hyperparameters, perform several trainings of the YOLOv5 model, and select the model with the optimal accuracy;

[0030] Step S23 specifically includes the following steps:

[0031] S231. Sample the video by taking one frame of image every 1 second to obtain a series of images containing the object motion trajectories;

[0032] S232. Annotate the data; S233. Assign different ID numbers to the objects;

[0033] S234. Use whether the data is selected from the same video as the discrimination condition, divide it into the training and test sets, and perform Deepsort model training.

[0034] In the above-mentioned electric vehicle overload behavior detection method based on YOLOv5 and Deepsort, step S3 specifically includes the following steps:

[0035] S31, obtaining a real-time video stream from a road monitoring camera;

[0036] S32, using the YOLOv5 algorithm to detect each frame of the image, identify the required target categories, and obtain the position information of the target object to be detected;

[0037] S33, using the Deepsort target tracking algorithm to assign an ID to each detected target;

[0038] S34, judging the positional relationship between the various targets, and if the overloading rule is met and the ID of the offending subject does not appear repeatedly, it is determined that the electric vehicle is overloaded;

[0039] The current frame is set as the start of the alarm, the alarm start time is recorded and the video is recorded from the rtsp video stream, and the alarm occurrence location is recorded according to the camera position;

[0040] When the electric vehicle with this ID disappears from the image or is no longer judged to be overloaded, the previous frame image is used as the alarm end image, the alarm end time is recorded, and the video recording is ended to generate an MP4 file.

[0041] In step S31, a video stream is obtained from a 4K high-definition camera on the road according to the rstp protocol, and images are extracted from the video stream at a set time interval for overload behavior detection; the time interval can be set in the following two ways:

[0042] ① Use server-deployed algorithms for detection: The server is configured with a multi-core high-performance GPU and CPU to run the image preprocessing and algorithm detection processes on the GPU;

[0043] ②: Use embedded devices as carriers to deploy algorithms: embedded devices and cameras are directly installed in the equipment cabinet at the intersection;

[0044] In the above-mentioned electric vehicle overload behavior detection method based on YOLOv5 and Deepsort, step S4 includes the following steps:

[0045] S41. When the alarm starts, a face detection algorithm is used to capture the face image with the highest confidence level from the overloaded electric vehicle area;

[0046] S42. In the frames subsequently determined to be overloaded, faces are also captured, the confidence levels are compared between faces, and the best face is selected for subsequent identification of the person who drove the overloaded electric vehicle and for educational punishment.

[0047] In the above-mentioned electric vehicle overloading behavior detection method based on YOLOv5 and Deepsort, steps S41 - S42 specifically include the following steps:

[0048] S430. Obtain the current frame image, use the YOLOv5 algorithm for detection, and then directly use the Deepsort target tracking algorithm. According to the object information in the previous frame and the object information in the current frame, assign corresponding IDs to the old targets and new IDs to the new targets.

[0049] S431. Based on the ID information of the overloaded electric vehicle recorded at the start of this alarm, find the corresponding target electric vehicle in the current frame. In the area of the target electric vehicle, judge whether the electric vehicle still has an overloading behavior according to the number of people's heads and helmets.

[0050] S432. If there is still an overloading behavior, perform face detection in the area of the overloaded electric vehicle, compare the confidence of the detected face with that in the previous frame, and retain the face with higher confidence, so as to obtain a face cutout with better quality.

[0051] S433. Repeat steps S430 - S432 until the Deepsort target tracking algorithm can no longer track the overloaded electric vehicle recorded at the start of this alarm, or the target electric vehicle cannot be determined to be overloaded for multiple consecutive frames. At this time, record the alarm end time, save the previous frame of the current frame as the alarm end image locally, and end the writing of the MP4 video to generate an MP4 file, and save the face image with the maximum confidence collected during the period from the start to the end of this alarm. Thus, a complete alarm message is obtained.

[0052] S434. Upload the face image to the cloud, compare it with the local face database, and obtain the identity information of the actor of the overloaded electric vehicle behavior for warning and punishment.

[0053] Compared with the prior art, the advantages of the present invention are as follows: It can detect the video images captured by road monitoring, automatically discover the overloading behavior of electric vehicles and record the time and location of the discovery of the overloading behavior. It can obtain the driver's identity information through the captured face images for education and punishment, thereby saving a large amount of traffic police human resources, improving the public's awareness of safe driving of electric vehicles, and reducing major accidents caused by overloading of electric vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is the flowchart of the method of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0055] The following further describes the present invention in detail with reference to the drawings and specific embodiments.

[0056] AsFigure 1 As shown in the figure, a method for detecting the overloading behavior of electric vehicles based on YOLOv5 and Deepsort includes the following steps:

[0057] S1. Establish a YOLOv5 algorithm database;

[0058] S2. Establish a Deepsort algorithm database;

[0059] S3. Detect the overloading of electric vehicles;

[0060] S4. Obtain the faces of the overloading electric vehicle behaviors.

[0061] Step S1 includes the following steps:

[0062] S11. Use the YOLOv5 algorithm to detect the types of objects appearing in the road images,

[0063] S12. At different time points, obtain electric vehicle images from different road 4K high-definition cameras;

[0064] S13. Label the image data by combining manual and automatic labeling;

[0065] S14. Use data augmentation to train the YOLOv5 model;

[0066] S15. Train to obtain the YOLOv5 model as the YOLOv5 algorithm database to detect various object types in the road images;

[0067] S16. Record the detection results for subsequent determination of electric vehicle overloading.

[0068] In step S11, the object types include the object type and the helmet wearing status; among them, the object types include electric vehicles, bicycles, cars, pedestrians, wheelchairs, and baby carriages, and the helmet wearing status includes wearing a helmet and not wearing a helmet; in step S12, the electric vehicle images include electric vehicle images in the un-overloaded and overloaded states and two state images of the riders wearing helmets.

[0069] The number of images in multiple states needs to be balanced to ensure that the training results can meet the expected requirements. Images at different times are collected to provide data under different lighting conditions. The lighting intensity and angle are different in the morning, daytime, evening, and night. During the day, due to the good lighting conditions, the camera can basically capture good images. In the morning and evening, because the sun is at a low position, the backlit areas will be hidden in large shadow areas. At night, without sunlight, relying on streetlights for supplementary lighting or without supplementary lighting may lead to underexposure, resulting in lost details, increased noise, and overly dark images. In addition, rainy and snowy weather will also affect the lighting conditions, and since the riders wear rain gear, etc., it will block a large part of the visible part of the electric vehicle, affecting subsequent recognition. Selecting images from different locations can provide images with different angles and background situations, thereby increasing the versatility of the model after training.

[0070] Step S2 includes the following steps:

[0071] S21. At different time points, obtain images with complexity and quantity from different road 4K high-definition cameras;

[0072] S22. Label the images by combining manual and automatic annotation methods;

[0073] S23. Use the labeled data for Deepsort model training.

[0074] In steps S13 and S22, when annotating image data, the main object to be detected is the electric vehicle. To distinguish it from the electric vehicles parked by the roadside, the electric vehicle and its rider are labeled as a whole and defined as EBicycleCyclist. By labeling in this way, it can also provide an appropriate-sized area of the electric vehicle for subsequent judgment of whether there is overloading behavior within this area. Define the human head as the Head class and the human head wearing a helmet as the Helmet class. These two classes are used to determine the specific number of people carried by the electric vehicle. In addition to the above three classes mainly used, other label classes such as the car as the Vehicle class, the pedestrian as the Person class, the wheelchair as the Wheelchair class, and the baby carriage as the BabyCar class are also defined, so as to reduce the misrecognition of objects with similar characteristics to the electric vehicle. After annotation, according to the proportion, it is randomly assigned to the training set, test set, and validation set for model training.

[0075] Step S14 specifically includes the following steps:

[0076] S141. Perform operations such as rotating, scaling, splicing, and cropping on the images to increase the data volume for training the initial version of the YOLOv5 model;

[0077] S142, use the initial version of the YOLOv5 model to detect the remaining images and output the coordinate positions of the detected objects;

[0078] S143, manually adjusting the coordinate positions, adding objects that were not marked, and deleting and correcting objects that were incorrectly marked;

[0079] S144, using all the data to set different hyperparameters, perform YOLOv5 model training several times, and select the model with the best accuracy;

[0080] Step S23 specifically includes the following steps:

[0081] S231, sampling the video in a manner of taking a frame of image every 1 second, thereby obtaining a series of images containing the motion trajectory of the object;

[0082] S232, labeling the data; the labeling rules are different from those of the YOLOv5 algorithm. In addition to distinguishing objects of different classes, different objects of the same class also need to be distinguished;

[0083] S233, assigning different ID numbers to objects;

[0084] S234. Whether the data is selected from the same video is used as a distinguishing condition to divide the data into training and test sets, and perform Deepsort model training.

[0085] The amount of data and the quality of annotations will affect the accuracy of model detection to a large extent, so data marking will be an ongoing process. As new images are collected in the future, the database will continue to increase and replace the images with poor quality in the original database. In addition, special models can be trained according to the specific requirements of different environments.

[0086] Use a high-definition 4K camera as the front-end device to provide images. The camera should be installed at a certain height according to the surrounding conditions of the road so that it can clearly capture the non-motorized vehicle lane. When debugging the camera, adjust the camera focal length and angle so that the main traffic flow on the non-motorized vehicle lane will pass through the center of the picture, and the electric vehicle can account for 1 / 2 of the image at the center, so that a clearer facial image of the driver and passenger of the overloaded electric vehicle can be captured for subsequent identity determination. In addition to ensuring that there are no trees, road signs, etc. that block non-motorized vehicles, when there is no street lighting on the non-motorized vehicle lane at night or the brightness of the street lighting cannot meet the requirements, it is necessary to install a fill light to ensure that clear and detailed images can be captured at night.

[0087] Step S3 specifically includes the following steps:

[0088] S31, obtaining a real-time video stream from a road monitoring camera;

[0089] S32. Use the YOLOv5 algorithm to detect each frame of the image, identify each required target category, and obtain the position information of the target objects to be detected.

[0090] S33. Use the Deepsort object tracking algorithm to assign an ID to each detected target.

[0091] S34. Judge the positional relationship between each target. When the overload rule is met and the ID of the illegal subject does not appear repeatedly, it is determined that the electric vehicle is overloaded.

[0092] Set the current frame as the start of this alarm, record the alarm start time and start recording the video from the rtsp video stream, and record the alarm occurrence location according to the camera position.

[0093] When the electric vehicle with this ID disappears from the image or is no longer determined to be overloaded, the previous frame image is used as the alarm end image, record the alarm end time, and end the video recording to generate an MP4 file.

[0094] In step S31, obtain the video stream from the road 4K high-definition camera according to the rstp protocol, and extract images from the video stream at a set time interval for overloading behavior detection. Affected by the device performance of the deployed algorithm, network bandwidth resources, and the number of cameras for simultaneous detection by the algorithm on the same device, the setting of the time interval includes the following two types:

[0095] ① Use the server to deploy the algorithm for detection: The server configures a multi-core high-performance GPU and CPU, and runs the image preprocessing and algorithm detection processes on the GPU for calculation.

[0096] This can greatly improve the detection speed and save CPU consumption. However, using the server to deploy the algorithm requires high network conditions. Most importantly, it requires a high bandwidth. For a 4K high-definition camera, each frame of the image it captures has 66,000,000 bits, and the default frame rate of most cameras on the market is 25 frames. When the server simultaneously detects dozens of cameras, the bandwidth requirement will be extremely high. At the same time, to ensure the accuracy of recognition, the packet loss rate and latency need to be controlled at a low level, otherwise there will be a large number of mosaic situations in the obtained images, affecting recognition.

[0097] ② Use an embedded device as a carrier to deploy the algorithm: The embedded device and the camera are directly installed in the intersection equipment cabinet together.

[0098] Since the cameras and other devices are located in the same local area network and the distance between the devices is relatively close, the requirements for the network will be greatly reduced. The detection calculation of the image will be in the embedded device at the intersection, and usually only the detection results will be uploaded to the central cloud, which reduces a large amount of image data transmission, effectively reducing the network burden and bandwidth consumption. In addition, the privacy and security of the data are improved, offline operation is supported, and it can also be deployed at intersections without good network conditions. However, due to the limited performance of embedded devices, it is impossible to detect a large number of cameras. Although some chips similar to GPU architecture are now available on the market that can be used to accelerate algorithms on embedded devices, the performance of a single embedded device cannot be compared with that of a high-configuration server. Therefore, when a large number of cameras are installed at an intersection, multiple embedded devices will need to be deployed at the same time. As the number of devices increases, the cost of equipment maintenance will increase.

[0099] The image obtained from the camera is detected by the YOLOv5 algorithm. First, find out whether there is an electric vehicle in the detected information. When there is an electric vehicle, select the electric vehicle with the highest confidence. Secondly, traverse the detected heads and helmets to determine whether they are in the electric vehicle area with the highest confidence. In order to prevent the heads and helmets of non-target vehicles nearby from mistakenly entering the electric vehicle area, the detection area is set to the upper half of the electric vehicle area, which prevents the heads of the back seat from leaving the detection area and ensures that when there is an overloaded person in front of the driver, his head is in the area. When it is determined that there is overloading in the image, the position information of all objects detected by YOLOv5 and the overall image are input into the Deepsort target tracking algorithm, and a temporary ID is given to all detected objects. At this time, the ID of the overloaded electric vehicle is recorded. At the same time, the face detection algorithm is used to detect faces in the overloaded electric vehicle area, and the face image with the highest confidence is intercepted and recorded together with the ID of the overloaded electric vehicle. At this moment, the overload alarm begins. The alarm start time, camera position and other information are recorded. A box is added to the image to indicate the position of the overloaded electric vehicle. The image is saved locally, and the real-time video stream is captured from the camera and written into an MP4 file.

[0100] Step S4 includes the following steps:

[0101] S41. When the alarm starts, a face detection algorithm is used to capture the face image with the highest confidence level from the overloaded electric vehicle area;

[0102] S42. In the frames subsequently determined to be overloaded, faces are also captured, the confidence levels are compared between faces, and the best face is selected for subsequent identification of the person who drove the overloaded electric vehicle and for educational punishment.

[0103] Step S41-step S42 specifically includes the following steps:

[0104] S430, obtaining the current frame image, using the YOLOv5 algorithm for detection, and then directly using the Deepsort target tracking algorithm to assign a corresponding ID to the old target and a new ID to the new target based on the object information in the previous frame and the object information in the current frame;

[0105] S431, according to the ID information of the overloaded electric vehicle recorded when the alarm starts, find the corresponding target electric vehicle in the current frame, and determine whether the electric vehicle is still overloaded according to the number of heads and helmets in the target electric vehicle area;

[0106] S432, if the overloading behavior still exists, face detection is performed in the overloaded electric vehicle area, and the confidence of the detected face is compared with that in the previous frame, and the face with higher confidence is retained, so as to obtain a face cutout with better quality; the face with higher confidence is usually larger in size, richer in details, more positive in posture, and less blocked;

[0107] S433, repeating steps S430-S432 until the Deepsort target tracking algorithm can no longer track the overloaded electric vehicle recorded when the alarm starts, or the target electric vehicle cannot be determined as overloaded for multiple consecutive frames. This is recorded as the alarm end time, and the previous frame of the current frame is saved locally as the alarm end image. The writing of the MP4 video is terminated, and an MP4 file is generated to save the face image with the maximum confidence collected from the start to the end of the alarm, thereby obtaining a complete alarm information;

[0108] The complete alarm information includes the start and end time, the location of the alarm, the start and end images, a face image with the best quality and a video of the overloaded electric vehicle.

[0109] S434. Upload the facial image to the cloud and compare it with the local facial database to obtain the identity information of the person who overloaded the electric vehicle for warning and punishment. The images and videos saved locally can be uploaded when needed as supporting evidence.

[0110] In summary, the principle of this embodiment is as follows: By collecting a large number of electric vehicle images in different road environments, after annotation, the YOLOv5 model and the Deepsort model are trained. The trained YOLOv5 model is used in conjunction with the Deepsort model to identify non-compliant electric passenger vehicles and perform face detection, record overloading behaviors and select whether to alarm; it has high detection accuracy and versatility. According to the performance of the operating equipment, the detection speed is adjusted by modifying the complexity of the model network structure. Compared with other Deepsort object tracking algorithms, the algorithm of the Deepsort model runs faster and is more suitable for cooperating with the YOLOv5 object detection algorithm. Since both the YOLOv5 and Deepsort algorithms are relatively mature, they can be deployed to the central server relatively conveniently and quickly; at the same time, after modifying the model, it can be relatively simply deployed and applied to embedded devices. For the transportation field, using embedded devices to achieve edge computing can better save network resource consumption and save equipment and facility costs.

[0111] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Those skilled in the art of the present invention can make various modifications or supplements to the described specific embodiments or use similar methods to replace them, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort, characterized in that, It includes the following steps: S1. Establish a YOLOv5 algorithm database; S2. Establish a Deepsort algorithm database; S3. Detect overloading of electric vehicles; S4. Obtain the faces of the riders of overloaded electric vehicles.

2. The method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 1, characterized in that, Step S1 includes the following steps: S11. Use the YOLOv5 algorithm to detect the types of objects appearing in the road images, S12. At different time points, obtain electric vehicle images from different road 4K high-definition cameras; S13. Label the image data by combining manual and automatic annotation methods; S14. Use data augmentation methods to train the YOLOv5 model; S15. Train to obtain the YOLOv5 model as the YOLOv5 algorithm database to detect various object types in the road images; S16. Record the detection results for subsequent determination of electric vehicle overloading.

3. The method for detecting the overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 2, wherein, In step S11, the object types include object categories and helmet-wearing status; among them, the object categories include motor vehicles, bicycles, cars, pedestrians, wheelchairs, and baby carriages, and the helmet-wearing status includes wearing a helmet and not wearing a helmet; in step S12, the electric vehicle images include images of electric vehicles in non-overloaded and overloaded states and two state images of the riders wearing helmets.

4. A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 3, characterized in that, Step S2 includes the following steps: S21. At different time points, obtain images with complexity and quantity from different road 4K high-definition cameras; S22. Label the images by combining manual and automatic annotation methods; S23. Use the labeled data to train the Deepsort model.

5. A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 4, characterized in that, In steps S13 and S22, when annotating the image data, label the electric vehicle and its rider as a whole and define it as EBicycleCyclist, define the human head as the Head class and the human head wearing a helmet as the Helmet class. After annotation, randomly allocate them into the training set, test set, and validation set according to a ratio for model training.

6. The method for detecting the overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 4, wherein Step S14 specifically includes the following steps: S141. Perform operations such as rotating, scaling, splicing, and cropping on the images to increase the data volume for training the initial version of the YOLOv5 model; S142. Use the initial version of the YOLOv5 model to detect the remaining images and output the coordinate positions of the detected objects; S143. Manually adjust the coordinate positions, add objects that are missed in annotation, and delete and correct mis-annotated objects; S144. Use all the data to set different hyperparameters and perform several trainings of the YOLOv5 model, and select the model with the optimal accuracy; Step S23 specifically includes the following steps: S231. Sample the video at a rate of taking one frame every 1 second to obtain a series of images containing the object movement trajectories; S232. Annotate the data; S233. Assign different ID numbers to the objects; S234. Use whether the data is selected from the same video as the discrimination condition, divide it into the training and test sets, and perform Deepsort model training.

7. A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31. Obtain the real-time video stream from the road monitoring camera; S32, using the YOLOv5 algorithm to detect each frame of the image, identify the required target categories, and obtain the position information of the target object to be detected; S33, using the Deepsort target tracking algorithm to assign an ID to each detected target; S34, judging the positional relationship between the various targets, and if the overloading rule is met and the ID of the offending subject does not appear repeatedly, it is determined that the electric vehicle is overloaded; The current frame is set as the start of the alarm, the alarm start time is recorded and the video is recorded from the rtsp video stream, and the alarm occurrence location is recorded according to the camera position; When the electric vehicle with this ID disappears from the image or is no longer judged to be overloaded, the previous frame image is used as the alarm end image, the alarm end time is recorded, and the video recording is ended to generate an MP4 file.

8. A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 7, characterized in that, In step S31, a video stream is obtained from a 4K high-definition camera on the road according to the rstp protocol, and images are extracted from the video stream at a set time interval for overload behavior detection; the time interval can be set in the following two ways: ① Use server-deployed algorithms for detection: The server is configured with a multi-core high-performance GPU and CPU to run the image preprocessing and algorithm detection processes on the GPU; ② Use embedded devices as carriers to deploy algorithms: embedded devices and cameras are directly installed in the intersection equipment cabinet.

9. A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 8, characterized in that, Step S4 includes the following steps: S41. When the alarm starts, a face detection algorithm is used to capture the face image with the highest confidence level from the overloaded electric vehicle area; S42. In the frames subsequently determined to be overloaded, faces are also captured, the confidence levels are compared between faces, and the best face is selected for subsequent identification of the person who drove the overloaded electric vehicle and for educational punishment.

10. A method for detecting overloading behavior of electric vehicles based on YOLOv5 and Deepsort according to claim 9, characterized in that, Step S41-step S42 specifically includes the following steps: S430, obtaining the current frame image, using the YOLOv5 algorithm for detection, and then directly using the Deepsort target tracking algorithm to assign a corresponding ID to the old target and a new ID to the new target based on the object information in the previous frame and the object information in the current frame; S431, according to the ID information of the overloaded electric vehicle recorded when the alarm starts, find the corresponding target electric vehicle in the current frame, and determine whether the electric vehicle is still overloaded according to the number of heads and helmets in the target electric vehicle area; S432, if the overloading behavior still exists, face detection is performed in the overloaded electric vehicle area, the confidence of the detected face is compared with that in the previous frame, and the face with higher confidence is retained, so as to obtain a face cutout with better quality; S433, repeat steps S430-S432 until the Deepsort target tracking algorithm can no longer track the overloaded electric vehicle recorded at the start of the alarm, or the target electric vehicle cannot be judged as overloaded for multiple consecutive frames. This is recorded as the alarm end time, and the previous frame of the current frame is saved locally as the alarm end image, and the writing of the MP4 video is ended, and an MP4 file is generated to save the face image with the highest confidence collected from the start to the end of the alarm. Thus, a complete alarm information is obtained; S434. Upload the face image to the cloud and compare it with the local face database to obtain the identity information of the actor of the overloaded electric vehicle for warning and punishment.

Images