Traffic intersection multi-camera cooperative electric vehicle safety equipment identification and detection method

Abstract

The invention discloses a traffic intersection multi-camera cooperative electric vehicle safety equipment identification and detection method, and relates to the technical field of visual inspection, and the method comprises the steps: obtaining a network public data set, and obtaining a sample data set; obtaining an optimal identification parameter and a human-vehicle detection model; obtaining a detection image based on the video stream; acquiring a target detection frame, and analyzing and acquiring a to-be-detected image of the helmet; performing face snapshot; the method is used for solving the problems that the existing identification and detection method for the safety equipment of the electric vehicle at the traffic intersection is relatively single, and the helmet wearing condition of an electric vehicle rider cannot be stably identified due to a single target detection algorithm when the types of electric vehicles are various and the helmet characteristics are not obvious; and report omission is caused by complex scenes and vehicle overlapping.

Description

Technical Field

[0001] The present invention relates to the technical field of visual detection, and in particular to a method for identifying and detecting safety equipment of electric vehicles in collaboration with multiple cameras at a traffic intersection. Background Art

[0002] For the management of new modes of transportation such as electric vehicles, ensuring that riders wear helmets is an important part of ensuring road safety; relying on pure human resources is obviously no longer able to meet the needs of the rapidly growing number of electric vehicles and increasingly complex traffic conditions; by installing cameras or other sensing devices, and combining image recognition and artificial intelligence algorithms, real-time monitoring of the helmet wearing status of electric vehicle riders can be achieved; at the same time, the system can also record relevant data for statistical analysis of traffic management, so as to better understand and optimize the traffic management process.

[0003] Existing methods for identifying and detecting electric vehicle safety equipment at traffic intersections usually establish a coordinate system and use the key nodes of the human body to analyze the human head based on artificial intelligence to determine whether the electric vehicle rider is wearing a helmet as required. Although this detection method can improve the recognition and judgment efficiency through artificial intelligence, the judgment method is relatively single. When there are many types of electric vehicles and the characteristics of the helmets are not obvious, the single target detection algorithm will lead to the inability to stably identify the helmet wearing status of the electric vehicle rider, and will cause the problem of underreporting due to complex scenes and overlapping vehicles. For example, in a patent application with publication number CN113537183A, an electric vehicle helmet recognition method based on artificial intelligence is disclosed. This solution is to establish Coordinate system, area division according to the coordinates of key nodes of the head, identification and judgment of whether electric vehicle riders are wearing helmets as required, based on artificial intelligence, greatly saves detection costs and improves work efficiency. Other improvements in the recognition and detection of electric vehicle safety equipment for traffic intersections are usually achieved by using existing models and combining them with license plates for detection. This recognition and detection method is still relatively simple and cannot solve the problem that when there are many types of electric vehicles and the characteristics of helmets are not obvious, a single target detection algorithm will lead to the inability to stably identify the helmet wearing status of electric vehicle riders, and will cause underreporting due to complex scenes and overlapping vehicles. In view of this, it is necessary to improve the existing recognition and detection methods for electric vehicle safety equipment for traffic intersections. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent. By proposing a method for identifying and detecting electric vehicle safety equipment in coordination with multiple cameras at traffic intersections, the present invention is used to solve the problem that the existing method for identifying and detecting electric vehicle safety equipment at traffic intersections is relatively single and cannot solve the problem that when there are many types of electric vehicles and the features of helmets are not obvious, a single target detection algorithm will lead to the inability to stably identify the helmet wearing status of electric vehicle riders, and will cause underreporting due to complex scenes and overlapping vehicles.

[0005] To achieve the above purpose, the present application provides a method for identifying and detecting electric vehicle safety equipment using multiple cameras at a traffic intersection, comprising the following steps: Obtain a public network data set, preprocess the public network data set, and obtain a sample data set based on the preprocessing result; use the sample data set to train the YOLOv8 network model, and obtain the optimal recognition parameters based on the training result, and record the YOLOv8 network model with the optimal recognition parameters as a human-vehicle detection model, wherein the human-vehicle detection model includes an electric vehicle detection model, a pedestrian detection model, and a helmet detection model; Obtain the video stream of the camera at the intersection, and perform video processing on the video stream to obtain a detection image; input the detection image into the electric vehicle detection model and the pedestrian detection model for detection, and obtain the target detection frame based on the detection results, wherein the target detection frame includes the electric vehicle detection frame and the pedestrian detection frame; The target detection frame is analyzed, and the helmet image to be detected is obtained based on the analysis result; the helmet image to be detected is input into the helmet detection model, and the face is captured based on the detection result.

[0006] Furthermore, obtaining a network public data set, and preprocessing the network public data set, and obtaining a sample data set based on the preprocessing result includes: Obtain a network public data set based on a web crawler, and record the image data and video data containing pedestrians at the intersection in the network public data set as pedestrian image data and pedestrian video data, wherein the number of images of the pedestrian image data is recorded as n, and the number of images of the pedestrian video data is recorded as m. When n is greater than or equal to m, the value of n divided by m and rounded up is recorded as the video mean. When n is less than m, the video mean is set to 1. For any pedestrian video data, the pedestrian video data is subjected to frame extraction processing, and the image corresponding to each obtained frame is recorded as a pedestrian video frame image. The number of pedestrians in each pedestrian video frame image is obtained based on AI, and the video average of a larger number of pedestrian video frame images is recorded as a pedestrian video image; the pedestrian video images corresponding to all pedestrian video data are obtained, and all pedestrian image data and pedestrian video images are recorded as pedestrian images.

[0007] Furthermore, obtaining a network public data set, preprocessing the network public data set, and obtaining a sample data set based on the preprocessing result also includes: The pedestrian images are processed based on personnel labeling and AI labeling, and the electric vehicles, pedestrians and helmets in each pedestrian image are labeled based on the processing results. The labeled pedestrian images are stored in a data set, and the data set is recorded as a sample data set.

[0008] Furthermore, the YOLOv8 network model is trained using the sample data set, and the preferred recognition parameters are obtained based on the training results. The YOLOv8 network model with the preferred recognition parameters is recorded as a human-vehicle detection model, including: Get the YOLOv8 network model and use the sample data set to train the YOLOv8 network model; The training method is: divide the sample data set into training set, validation set and test set, modify the source code in the YOLOv8 network model and use the training set to train the YOLOv8 network model once; One training is as follows: for any pedestrian image in the training set, the number of electric vehicles, pedestrians and helmets in the pedestrian image is recorded as α1, α2 and α3 respectively. When the number of electric vehicles, pedestrians and helmets obtained by the YOLOv8 network model for pedestrian image recognition is α1, α2 and α3 respectively and the recognition position is consistent with the marked position in the pedestrian image, the training is completed. Otherwise, it is recorded as training failure. When there is a training failure, modify the source code in the YOLOv8 network model until the YOLOv8 network model recognizes the electric vehicles, pedestrians and helmets of all pedestrian images in the training set and the marking results are consistent. Use the validation set and test set to perform secondary training on the YOLOv8 network model; The secondary training is to modify the source code in the YOLOv8 network model until the recognition results of the YOLOv8 network model for electric vehicles, pedestrians, and helmets in the validation set and the pedestrian images in the test set are consistent with the annotations.

[0009] Further, the YOLOv8 network model is trained using the sample data set, and the preferred recognition parameters are obtained based on the training results, and the YOLOv8 network model with the preferred recognition parameters is recorded as a human-vehicle detection model, which also includes: After the YOLOv8 network model completes the secondary training, the source code parameters of the YOLOv8 network model used to identify the electric vehicle are recorded as electric vehicle parameters, the source code parameters of the YOLOv8 network model used to identify the pedestrian are recorded as pedestrian parameters, the source code parameters of the YOLOv8 network model used to identify the helmet are recorded as helmet parameters, and the electric vehicle parameters, pedestrian parameters and helmet parameters are recorded as preferred identification parameters; The YOLOv8 network model containing electric vehicle parameters is recorded as an electric vehicle detection model, the YOLOv8 network model containing pedestrian parameters is recorded as a pedestrian detection model, and the YOLOv8 network model containing helmet parameters is recorded as a helmet detection model.

[0010] Furthermore, obtaining a video stream from a camera at the intersection and performing video processing on the video stream to obtain a detection image includes: The video stream of the camera at the intersection is obtained, and the video stream is processed to obtain the detection image. The video processing is as follows: the video stream is processed by frame extraction, and the image corresponding to each frame in the video stream is recorded as detection image JC1 to detection image JC based on the time sequence. k .

[0011] Furthermore, the detection image is input into the electric vehicle detection model and the pedestrian detection model for detection, and obtaining the target detection frame based on the detection results includes: Use the human-vehicle detection method to detect all the detection data JC in sequence. The human-vehicle detection method is as follows: for detection images JC1 to JC k Any detection image JC k1 , the detection image JC k1 Put it into the electric vehicle detection model and the pedestrian detection model for detection. When the detection result of the electric vehicle detection model is 0 or the detection result of the pedestrian detection model is 0, the detection image JC k1+1 Conduct testing; When the detection result of the electric vehicle detection model is greater than 0 and the detection result of the pedestrian detection model is greater than 0, the electric vehicle detected by the electric vehicle detection model is framed and recorded as electric vehicle detection frame DJ1 to electric vehicle detection frame DJ p , the pedestrians detected by the pedestrian detection model are framed and recorded as pedestrian detection frame XJ1 to pedestrian detection frame XJ q .

[0012] Further, analyzing the target detection frame and obtaining the helmet to-be-detected image based on the analysis result includes: For any detection image JC containing an electric vehicle detection frame DJ and a pedestrian detection frame XJ, the electric vehicle detection frame DJ in the detection image JC is stored in the set car_rects, and the pedestrian detection frame XJ in the detection image JC is stored in the set personal_rects; Based on the intersection calculation formula, use the OpenCV library to obtain the intersection value of the set car_rects and the set personal_rects.

[0013] Furthermore, analyzing the target detection frame and obtaining the helmet to-be-detected image based on the analysis result also includes: When the intersection value is 0, get the next detection image JC containing the electric vehicle detection frame DJ and the pedestrian detection frame XJ and calculate the intersection value of the detection image JC; when the intersection value is greater than 0, use the area ratio algorithm to get the pedestrian area ratio of the detection image JC. The area ratio algorithm is: ratio = (double)intersection.area() / personal_rects[i].area(), where ratio is the pedestrian area ratio, (double)intersection.area() is twice the area of ​​the intersection of the electric vehicle detection frame DJ and the pedestrian detection frame XJ, and personal_rects[i].area() is the area of ​​the pedestrian detection frame XJ; when the pedestrian area ratio is greater than the standard area ratio, the pedestrian detection frame XJ in the detection image JC is stored in the set personal_roi_rects; when the pedestrian area ratio is less than the standard area ratio, the next detection image JC with an intersection value greater than 0 is obtained and the pedestrian area ratio of the detection image JC is calculated, where the standard area ratio is obtained as follows: multiple pedestrian images of normally running electric vehicles are obtained, and after identification using the electric vehicle detection model and the pedestrian detection model, the average of the pedestrian area ratios of all pedestrian images based on the area ratio algorithm is recorded as the standard area ratio.

[0014] Further, the target detection frame is analyzed, and a helmet image to be detected is obtained based on the analysis result; the helmet image to be detected is input into a helmet detection model, and face capture is performed based on the detection result, including: The detection images JC whose number of pedestrian detection frames XJ in the set personal_roi_rects is greater than 0 are recorded as helmet detection images, and the helmet detection images are input into the helmet detection model. The identified helmets are framed and recorded as helmet detection frames TJ1 to TJ in sequence. c ; The pedestrian detection frame XJ in the helmet to be detected image that overlaps with the electric vehicle detection frame DJ is recorded as the wearing detection frame PJ. When the wearing detection frame PJ overlaps with any helmet detection frame TJ1, the wearing detection frame PJ is marked as worn; when the wearing detection frame PJ does not overlap with all helmet detection frames TJ, the wearing detection frame PJ is marked as not worn; Face capture is performed on all wearing detection frames marked as not wearing, and the behavior of electric bike riders not wearing helmets and facial information are reported based on the face capture results.

[0015] Beneficial effects of the present invention: The present invention first obtains a network public data set and performs preprocessing to obtain a sample data set; obtains preferred recognition parameters and a human-vehicle detection model, which has the advantage that, by obtaining the preferred recognition parameters and the human-vehicle detection model, it is helpful to accurately identify the positional relationship between the helmet, the electric vehicle and the person in the subsequent analysis, thereby improving the recognition accuracy of the riders of different electric vehicles while improving the recognition accuracy of the helmet wearing and reducing the false detection rate; The present invention also obtains a detection image by processing the video stream of the camera at the intersection; obtains a target detection frame based on the detection image; finally, analyzes the target detection frame, and obtains a helmet image to be detected based on the analysis result; inputs the helmet image to be detected into a helmet detection model, and performs face capture based on the detection result. The advantage of this is that, by obtaining the target detection frame and analyzing the target detection frame, it is possible to screen a personnel detection frame in which the ratio of the area of ​​the intersection area of ​​the electric vehicle and the person to the area of ​​the person detection frame is greater than the standard area ratio, thereby optimizing the false detection problem caused by vehicle overlap. At the same time, by using the helmet detection model to perform helmet detection, it is possible to accurately detect people who need to wear helmets, thereby improving efficiency while reducing the false detection rate, and preventing the problem of missed reports caused by complex scenes and vehicle overlap. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the steps of the method of the present invention; Figure 2 It is a schematic diagram of the analysis of a training of the present invention; Figure 3 This is a schematic diagram of identification in which the wearing detection frame PJ of the present invention is marked as not worn; Figure 4 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1, please refer to Figure 1 As shown, the present application provides a method for identifying and detecting electric vehicle safety equipment by coordinating multiple cameras at a traffic intersection, comprising the following steps: Step S1, obtaining a network public data set, preprocessing the network public data set, and obtaining a sample data set based on the preprocessing result; using the sample data set to train the YOLOv8 network model, and obtaining the preferred recognition parameters based on the training result, and recording the YOLOv8 network model with the preferred recognition parameters as a human-vehicle detection model, wherein the human-vehicle detection model includes an electric vehicle detection model, a pedestrian detection model, and a helmet detection model; In the specific implementation process, in this embodiment, YOLOv8 uses the C2f module to replace the C3 module of YOLOv5, thereby achieving lightweight model while maintaining unchanged accuracy; Mosaic is turned off in the last 10 iterations of the training phase, which significantly improves the detection accuracy. Therefore, by using YOLOv8, it is possible to reduce energy consumption while improving accuracy and improve overall detection efficiency.

[0019] Step S1 includes: step S101, obtaining a network public data set based on a network crawler, and recording image data and video data containing pedestrians at intersections in the network public data set as pedestrian image data and pedestrian video data, wherein the number of images of the pedestrian image data is recorded as n, and the number of images of the pedestrian video data is recorded as m, when n is greater than or equal to m, dividing n by m and rounding up is recorded as a video mean, and when n is less than m, the video mean is set to 1; In the specific implementation process, by distinguishing and setting the video mean, it is possible to prevent the problem of unbalanced analysis due to the large difference in the number of samples when the number of acquired video data differs greatly from the number of image data; for example, in a data processing, the number of pedestrian image data obtained is 100, and the number of pedestrian video data is 10. At this time, n is greater than m. In order to obtain a sufficient number of pedestrian video data samples, the video mean is set to 100 divided by 10, that is, 10. 10 pictures can be obtained from each pedestrian video data to ensure that the sample data of pedestrian image data and pedestrian video data are relatively balanced; and when the number of pedestrian image data is 10 and the number of pedestrian video data is 100, in order to ensure that all pedestrian video data are recorded, the video mean can be set to 1, that is, 1 picture is obtained from each pedestrian video data for analysis, to prevent the problem of missed detection due to lack of recognition standards in subsequent analysis due to missed selection.

[0020] Step S102: for any pedestrian video data, perform frame extraction processing on the pedestrian video data, record the image corresponding to each obtained frame as a pedestrian video frame image, obtain the number of pedestrians in each pedestrian video frame image based on AI, and record the video average of pedestrian video frame images with a larger number as pedestrian video images; obtain pedestrian video images corresponding to all pedestrian video data, and record all pedestrian image data and pedestrian video images as pedestrian images; In summary of the specific implementation process, before obtaining the number of pedestrians in each pedestrian video frame image based on AI, the video stream of the intersection can be used to train AI for recognition to ensure that AI can perform accurate recognition; for example, during a data analysis, the video mean obtained is 10, then the number of pedestrians in each pedestrian video frame image is obtained based on AI, and the 10 pedestrian video frame images with a larger number are recorded as pedestrian video images for subsequent analysis.

[0021] Step S103, processing the pedestrian image based on the personnel labeling and AI labeling, and labeling the electric vehicle, pedestrian and helmet in each pedestrian image based on the processing result, storing the labeled pedestrian image in a data set, and recording the data set as a sample data set; Step S104, obtaining a YOLOv8 network model, and using the sample data set to train the YOLOv8 network model; The training method is: divide the sample data set into training set, validation set and test set, modify the source code in the YOLOv8 network model and use the training set to train the YOLOv8 network model once; One training is as follows: for any pedestrian image in the training set, the number of electric vehicles, pedestrians and helmets in the pedestrian image is recorded as α1, α2 and α3 respectively. When the number of electric vehicles, pedestrians and helmets obtained by the YOLOv8 network model for pedestrian image recognition is α1, α2 and α3 respectively and the recognition position is consistent with the marked position in the pedestrian image, the training is completed. Otherwise, it is recorded as training failure. When there is a training failure, modify the source code in the YOLOv8 network model until the YOLOv8 network model recognizes the electric vehicles, pedestrians and helmets of all pedestrian images in the training set and the marking results are consistent. In the specific implementation process, for example, in a training, for any pedestrian image in the training set, the number of electric vehicles, pedestrians and helmets in the pedestrian image are 3, 10 and 3 respectively, and the position of the helmet is as follows: Figure 2 As shown in PP1 in the figure, the numbers of electric vehicles, pedestrians and helmets obtained by the YOLOv8 network model for pedestrian image recognition are 3, 10 and 3 respectively, and the positions of the helmets are as follows Figure 2 As shown in PP2 in , through analysis, it can be found that the helmet obtained by the YOLOv8 network model when recognizing pedestrian images is inconsistent with the labeled position in the pedestrian image. Therefore, it is recorded as a training failure. The source code in the YOLOv8 network model should be modified until the YOLOv8 network model's recognition results for electric vehicles, pedestrians, and helmets in all pedestrian images in the training set are consistent with the annotations.

[0022] Step S105, using the validation set and the test set to perform secondary training on the YOLOv8 network model; Secondary training is to modify the source code in the YOLOv8 network model until the YOLOv8 network model's recognition results of electric vehicles, pedestrians, and helmets in the validation set and the pedestrian images in the test set are consistent with the annotations; Step S106, after the YOLOv8 network model completes secondary training, the source code parameters of the YOLOv8 network model used to identify the electric vehicle are recorded as electric vehicle parameters, the source code parameters of the YOLOv8 network model used to identify the pedestrian are recorded as pedestrian parameters, the source code parameters of the YOLOv8 network model used to identify the helmet are recorded as helmet parameters, and the electric vehicle parameters, pedestrian parameters and helmet parameters are recorded as preferred identification parameters; In the specific implementation process, by obtaining the electric vehicle parameters, pedestrian parameters and helmet parameters, independent recognition models can be used when detecting electric vehicles, pedestrians and helmets, thereby preventing the problem of false detection caused by overlap; Step S107, record the YOLOv8 network model containing the electric vehicle parameters as the electric vehicle detection model, record the YOLOv8 network model containing the pedestrian parameters as the pedestrian detection model, and record the YOLOv8 network model containing the helmet parameters as the helmet detection model.

[0023] Step S2, obtaining a video stream from a camera at the intersection, and performing video processing on the video stream to obtain a detection image; inputting the detection image into an electric vehicle detection model and a pedestrian detection model for detection, and obtaining a target detection frame based on the detection results, wherein the target detection frame includes an electric vehicle detection frame and a pedestrian detection frame; Step S2 includes: Step S201, obtaining a video stream from a camera at the intersection, and performing video processing on the video stream to obtain a detection image, wherein the video processing is: performing frame extraction processing on the video stream, and recording the images corresponding to each frame in the video stream as detection images JC1 to JC based on the time sequence. k ; Step S202: Use the human-vehicle detection method to detect all the detection data JC in sequence. The human-vehicle detection method is as follows: Step S2021: For detection images JC1 to JC k Any detection image JC k1 , the detection image JC k1 Put it into the electric vehicle detection model and the pedestrian detection model for detection. When the detection result of the electric vehicle detection model is 0 or the detection result of the pedestrian detection model is 0, the detection image JC k1+1 Conduct testing; In the specific implementation process, for example, during a data processing, after the detection image JC2 is put into the electric vehicle detection model and the pedestrian detection model for detection, the detection result of the electric vehicle detection model is 2 and the detection result of the pedestrian detection model is 0, which means that there are only electric vehicles and no pedestrians, and the condition for detecting the helmet is not met, so the detection image JC3 continues to be detected; Step S2022: When the detection result of the electric vehicle detection model is greater than 0 and the detection result of the pedestrian detection model is greater than 0, the electric vehicle detected by the electric vehicle detection model is framed and recorded as electric vehicle detection frame DJ1 to electric vehicle detection frame DJ2. p , the pedestrians detected by the pedestrian detection model are framed and recorded as pedestrian detection frame XJ1 to pedestrian detection frame XJ q .

[0024] Step S3, analyzing the target detection frame, and obtaining a helmet image to be detected based on the analysis result; inputting the helmet image to be detected into a helmet detection model, and capturing a face based on the detection result; Step S3 includes: step S301, for any detection image JC containing an electric vehicle detection frame DJ and a pedestrian detection frame XJ, storing the electric vehicle detection frame DJ in the detection image JC into a set car_rects, and storing the pedestrian detection frame XJ in the detection image JC into a set personal_rects; Step S302, using the OpenCV library to obtain the intersection value of the set car_rects and the set personal_rects based on the intersection calculation formula, the intersection calculation formula is: cv::Rect intersection=personal_rects[i]&car_rects[j], where intersection is the intersection value, personal_rects[i] is the i-th pedestrian detection frame XJ in the set personal_rects, and car_rects[j] is the j-th electric vehicle detection frame DJ in the set car_rects; In the specific implementation process, by using the formula cv::Rect intersection=personal_rects[i]&car_rects[j] in the OpenCV library, the intersection value of the set car_rects and the set personal_rects can be obtained. When the intersection value is greater than 0, it means that the pedestrian and the electric vehicle have an intersection at this time, and the pedestrian is riding an electric vehicle. Subsequent helmet detection should be performed; when the intersection value is 0, it means that the pedestrian and the electric vehicle do not have an intersection at this time, and the pedestrian is not riding an electric vehicle. The next detection image JC containing the electric vehicle detection frame DJ and the pedestrian detection frame XJ can be analyzed; Step S303, when the intersection value is 0, obtain the next detection image JC containing the electric vehicle detection frame DJ and the pedestrian detection frame XJ and calculate the intersection value of the detection image JC; when the intersection value is greater than 0, use the area ratio algorithm to obtain the pedestrian area ratio of the detection image JC, and the area ratio algorithm is: ratio = (double)intersection.area() / personal_rects[i].area(), where ratio is the pedestrian area ratio, (double)intersection.area() is twice the area of ​​the intersection area of ​​the electric vehicle detection frame DJ and the pedestrian detection frame XJ, and personal_rects[i].area() is the area of ​​the pedestrian detection frame XJ; In the specific implementation process, for example, during a data processing, due to the different sizes of the display during analysis, the values ​​of (double)intersection.area() and personal_rects[i].area() will be different, so it should be ensured that the device used to calculate the pedestrian area ratio is consistent with the device used to calculate the standard area ratio; for example, during a data processing, the (double)intersection.area() of the detection image JC obtained is 10cm², and personal_rects[i].area() is 5cm², then it can be calculated that the pedestrian area ratio is 2; Step S304: when the pedestrian area ratio is greater than the standard area ratio, the pedestrian detection frame XJ in the detection image JC is stored in the set personal_roi_rects; when the pedestrian area ratio is less than the standard area ratio, the next detection image JC with an intersection value greater than 0 is obtained and the pedestrian area ratio of the detection image JC is calculated, wherein the standard area ratio is obtained by: obtaining a plurality of pedestrian images of electric vehicles that are running normally, and after identification using the electric vehicle detection model and the pedestrian detection model, the average of the pedestrian area ratios of all pedestrian images is recorded as the standard area ratio based on the area ratio algorithm; In the specific implementation process, for example, during a data processing, the standard area ratio is obtained to be 1, and the pedestrian area ratio is 2, which means that the intersection area of ​​the pedestrian and the electric vehicle in the detection image JC is large and meets the intersection area of ​​riding under normal circumstances. Therefore, the pedestrian detection frame XJ in the detection image JC can be stored in the collection personal_roi_rects for subsequent helmet wearing detection; Step S305: record the detection images JC in which the number of pedestrian detection frames XJ in the set personal_roi_rects is greater than 0 as helmet detection images, input the helmet detection images into the helmet detection model, select the identified helmets and record them as helmet detection frames TJ1 to helmet detection frames TJ c ; Step S306, the pedestrian detection frame XJ in the helmet to be detected image that overlaps with the electric vehicle detection frame DJ is recorded as the wearing detection frame PJ. When the wearing detection frame PJ overlaps with any helmet detection frame TJ1, the wearing detection frame PJ is marked as worn; when the wearing detection frame PJ does not overlap with all helmet detection frames TJ, the wearing detection frame PJ is marked as not worn; In the specific implementation process, for example, in a data processing, the positions of all helmet detection frames TJ and wearing detection frames PJ are obtained as follows: Figure 3 As shown, the circular virtual frame is the wearing detection frame, and the rectangular virtual frame is the helmet detection frame. Through analysis, it can be obtained that the wearing detection frame PJ1 does not overlap with all the helmet detection frames, so the wearing detection frame PJ1 is marked as not worn; Step S307: capture faces of all wearing detection frames marked as not wearing helmets, and report the behavior of the electric bike rider not wearing a helmet and facial information based on the face capture results.

[0025] Example 2, please refer to Figure 4 As shown, Figure 4 The structural schematic diagram of an electronic device is illustrated, and the electronic device may include: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The memory stores computer-readable instructions, and the processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, the steps in a method for identifying and detecting safety equipment of electric vehicles in cooperation with multiple cameras at a traffic intersection are executed to achieve the following functions: the present invention first obtains a network public data set and performs preprocessing to obtain a sample data set; obtains preferred identification parameters and a human-vehicle detection model; then processes the video stream of the camera at the intersection to obtain a detection image; obtains a target detection frame based on the detection image; finally, analyzes the target detection frame, and obtains a helmet image to be detected based on the analysis result; inputs the helmet image to be detected into the helmet detection model, and performs face capture based on the detection result.

[0026] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0027] Embodiment 3, the present application also provides a computer program product, the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by the computer, the computer can execute a traffic intersection multi-camera collaborative electric vehicle safety equipment recognition and detection method provided by the above methods, the method includes: the present invention first obtains a network public data set and preprocesses it to obtain a sample data set; obtains preferred recognition parameters and a human and vehicle detection model; then processes the video stream of the camera at the intersection to obtain a detection image; obtains a target detection frame based on the detection image; finally, analyzes the target detection frame, and obtains a helmet image to be detected based on the analysis result; inputs the helmet image to be detected into the helmet detection model, and performs face capture based on the detection result.

[0028] Embodiment 4, the present application also provides a computer-readable storage medium, the present application provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method for identifying and detecting electric vehicle safety equipment in collaboration with multiple cameras at a traffic intersection are executed to achieve the following functions: the present invention first obtains a public data set on the Internet and preprocesses it to obtain a sample data set; obtains preferred recognition parameters and a human and vehicle detection model; then processes the video stream of the camera at the intersection to obtain a detection image; obtains a target detection frame based on the detection image; finally, analyzes the target detection frame, and obtains a helmet image to be detected based on the analysis result; inputs the helmet image to be detected into the helmet detection model, and performs face capture based on the detection result.

[0029] Through the description of the above implementation methods, the embodiments of the present invention can be provided as methods, systems or computer program products. Based on such an understanding, the above technical solutions can be essentially or partly contributed to the prior art in the form of software products, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and include several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0030] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying and detecting electric vehicle safety equipment using multiple cameras at a traffic intersection, characterized in that: The steps include: Obtain a public network data set, preprocess the public network data set, and obtain a sample data set based on the preprocessing result; use the sample data set to train the YOLOv8 network model, and obtain the optimal recognition parameters based on the training result, and record the YOLOv8 network model with the optimal recognition parameters as a human-vehicle detection model, wherein the human-vehicle detection model includes an electric vehicle detection model, a pedestrian detection model, and a helmet detection model; Obtain the video stream of the camera at the intersection, and perform video processing on the video stream to obtain a detection image; input the detection image into the electric vehicle detection model and the pedestrian detection model for detection, and obtain the target detection frame based on the detection results, wherein the target detection frame includes the electric vehicle detection frame and the pedestrian detection frame; The target detection frame is analyzed, and the helmet image to be detected is obtained based on the analysis result; the helmet image to be detected is input into the helmet detection model, and the face is captured based on the detection result.

2. According to claim 1, a method for identifying and detecting electric vehicle safety equipment using multiple cameras at traffic intersections is characterized in that: Obtain a public data set from the Internet, preprocess the public data set from the Internet, and obtain a sample data set based on the preprocessing results, including: Obtain a network public data set based on a web crawler, and record the image data and video data containing pedestrians at the intersection in the network public data set as pedestrian image data and pedestrian video data, wherein the number of images of the pedestrian image data is recorded as n, and the number of images of the pedestrian video data is recorded as m. When n is greater than or equal to m, the value of n divided by m and rounded up is recorded as the video mean. When n is less than m, the video mean is set to 1. For any pedestrian video data, the pedestrian video data is subjected to frame extraction processing, and the image corresponding to each obtained frame is recorded as a pedestrian video frame image. The number of pedestrians in each pedestrian video frame image is obtained based on AI, and the video average of a larger number of pedestrian video frame images is recorded as a pedestrian video image; the pedestrian video images corresponding to all pedestrian video data are obtained, and all pedestrian image data and pedestrian video images are recorded as pedestrian images.

3. According to claim 2, a method for identifying and detecting electric vehicle safety equipment using multiple cameras at a traffic intersection is characterized in that: Obtaining a public network data set, preprocessing the public network data set, and obtaining a sample data set based on the preprocessing result also includes: The pedestrian images are processed based on personnel labeling and AI labeling, and the electric vehicles, pedestrians and helmets in each pedestrian image are labeled based on the processing results. The labeled pedestrian images are stored in a data set, and the data set is recorded as a sample data set.

4. According to claim 3, a method for identifying and detecting electric vehicle safety equipment using multiple cameras at traffic intersections is characterized in that: The YOLOv8 network model is trained using the sample data set, and the optimal recognition parameters are obtained based on the training results. The YOLOv8 network model with the optimal recognition parameters is recorded as a human-vehicle detection model including: Get the YOLOv8 network model and use the sample data set to train the YOLOv8 network model; The training method is: divide the sample data set into training set, validation set and test set, modify the source code in the YOLOv8 network model and use the training set to train the YOLOv8 network model once; One training is as follows: for any pedestrian image in the training set, the number of electric vehicles, pedestrians and helmets in the pedestrian image is recorded as α1, α2 and α3 respectively. When the number of electric vehicles, pedestrians and helmets obtained by the YOLOv8 network model for pedestrian image recognition is α1, α2 and α3 respectively and the recognition position is consistent with the marked position in the pedestrian image, the training is completed. Otherwise, it is recorded as training failure. When there is a training failure, modify the source code in the YOLOv8 network model until the YOLOv8 network model recognizes the electric vehicles, pedestrians and helmets of all pedestrian images in the training set and the marking results are consistent. Use the validation set and test set to perform secondary training on the YOLOv8 network model; The secondary training is to modify the source code in the YOLOv8 network model until the recognition results of the YOLOv8 network model for electric vehicles, pedestrians, and helmets in the validation set and the pedestrian images in the test set are consistent with the annotations.

5. A method for identifying and detecting electric vehicle safety equipment using multiple cameras at traffic intersections according to claim 4, characterized in that: The YOLOv8 network model is trained using the sample data set, and the optimal recognition parameters are obtained based on the training results. The YOLOv8 network model with the optimal recognition parameters is recorded as the human-vehicle detection model, which also includes: After the YOLOv8 network model completes the secondary training, the source code parameters of the YOLOv8 network model used to identify the electric vehicle are recorded as electric vehicle parameters, the source code parameters of the YOLOv8 network model used to identify the pedestrian are recorded as pedestrian parameters, the source code parameters of the YOLOv8 network model used to identify the helmet are recorded as helmet parameters, and the electric vehicle parameters, pedestrian parameters and helmet parameters are recorded as preferred identification parameters; The YOLOv8 network model containing electric vehicle parameters is recorded as an electric vehicle detection model, the YOLOv8 network model containing pedestrian parameters is recorded as a pedestrian detection model, and the YOLOv8 network model containing helmet parameters is recorded as a helmet detection model.

6. A method for identifying and detecting electric vehicle safety equipment using multiple cameras at a traffic intersection according to claim 5, characterized in that: Obtain the video stream from the camera at the intersection and process the video stream to obtain the detection image, including: The video stream of the camera at the intersection is obtained, and the video stream is processed to obtain the detection image. The video processing is as follows: the video stream is processed by frame extraction, and the image corresponding to each frame in the video stream is recorded as detection image JC1 to detection image JC based on the time sequence. k .

7. A method for identifying and detecting electric vehicle safety equipment using multiple cameras at traffic intersections according to claim 6, characterized in that: The detection image is input into the electric vehicle detection model and the pedestrian detection model for detection. The target detection frame is obtained based on the detection results, including: Use the human-vehicle detection method to detect all the detection data JC in sequence. The human-vehicle detection method is as follows: for detection images JC1 to JC k Any detection image JC k1 , the detection image JC k1 Put it into the electric vehicle detection model and the pedestrian detection model for detection. When the detection result of the electric vehicle detection model is 0 or the detection result of the pedestrian detection model is 0, the detection image JC k1+1 Conduct testing; When the detection result of the electric vehicle detection model is greater than 0 and the detection result of the pedestrian detection model is greater than 0, the electric vehicle detected by the electric vehicle detection model is framed and recorded as electric vehicle detection frame DJ1 to electric vehicle detection frame DJ p , the pedestrians detected by the pedestrian detection model are framed and recorded as pedestrian detection frame XJ1 to pedestrian detection frame XJ q .

8. The method for identifying and detecting electric vehicle safety equipment using multiple cameras at traffic intersections according to claim 7 is characterized in that: Analyze the target detection frame and obtain the helmet to be detected image based on the analysis results, including: For any detection image JC containing an electric vehicle detection frame DJ and a pedestrian detection frame XJ, the electric vehicle detection frame DJ in the detection image JC is stored in the set car_rects, and the pedestrian detection frame XJ in the detection image JC is stored in the set personal_rects; Based on the intersection calculation formula, use the OpenCV library to obtain the intersection value of the set car_rects and the set personal_rects.

9. A method for identifying and detecting electric vehicle safety equipment using multiple cameras at traffic intersections according to claim 8, characterized in that: Analyzing the target detection frame and obtaining the helmet to be detected image based on the analysis result also includes: When the intersection value is 0, get the next detection image JC containing the electric vehicle detection frame DJ and the pedestrian detection frame XJ and calculate the intersection value of the detection image JC; when the intersection value is greater than 0, use the area ratio algorithm to get the pedestrian area ratio of the detection image JC. The area ratio algorithm is: ratio = (double)intersection.area() / personal_rects[i].area(), where ratio is the pedestrian area ratio, (double)intersection.area() is twice the area of ​​the intersection of the electric vehicle detection frame DJ and the pedestrian detection frame XJ, and personal_rects[i].area() is the area of ​​the pedestrian detection frame XJ; when the pedestrian area ratio is greater than the standard area ratio, the pedestrian detection frame XJ in the detection image JC is stored in the set personal_roi_rects; when the pedestrian area ratio is less than the standard area ratio, the next detection image JC with an intersection value greater than 0 is obtained and the pedestrian area ratio of the detection image JC is calculated, where the standard area ratio is obtained as follows: multiple pedestrian images of normally running electric vehicles are obtained, and after identification using the electric vehicle detection model and the pedestrian detection model, the average of the pedestrian area ratios of all pedestrian images based on the area ratio algorithm is recorded as the standard area ratio.

10. A method for identifying and detecting electric vehicle safety equipment using multiple cameras at traffic intersections according to claim 9, characterized in that: Analyze the target detection frame and obtain the helmet image to be detected based on the analysis result; Input the helmet image to be detected into the helmet detection model, and perform face capture based on the detection result, including: The detection images JC whose number of pedestrian detection frames XJ in the set personal_roi_rects is greater than 0 are recorded as helmet detection images, and the helmet detection images are input into the helmet detection model. The identified helmets are framed and recorded as helmet detection frames TJ1 to TJ in sequence. c ; The pedestrian detection frame XJ in the helmet to be detected image that overlaps with the electric vehicle detection frame DJ is recorded as the wearing detection frame PJ. When the wearing detection frame PJ overlaps with any helmet detection frame TJ1, the wearing detection frame PJ is marked as worn; When the wearing detection frame PJ does not overlap with all helmet detection frames TJ, the wearing detection frame PJ is marked as not worn; Face capture is performed on all wearing detection frames marked as not wearing, and the behavior of electric bike riders not wearing helmets and facial information are reported based on the face capture results.

Images