Method for detecting whether driver or passenger wears clothes or not based on image recognition

Through the three-stage detection method, the YOLOv5s and darknet53 networks are used to detect whether the driver and passengers are wearing clothes, solving the problems of reduced detection accuracy and high cost in the existing technology, and achieving efficient and accurate detection of the driver and passengers are not wearing clothes.

CN118968433APending Publication Date: 2024-11-15SICHUAN UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410991095.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art cannot effectively and quickly detect whether the driver and passengers are wearing clothes, resulting in a decrease in detection accuracy and high cost, which cannot meet the real-time detection needs.

Method used

The three-stage detection method is adopted. First, the YOLOv5s model is used to detect the position of the front window of the vehicle, then the position of the driver and passengers is detected, and finally the darknet53 network is used to classify whether to wear clothes, so as to achieve rapid and automatic detection of the situation of the driver and passengers not wearing clothes.

Benefits of technology

It realizes high-precision and low-cost inspection of drivers and passengers without clothing, and can quickly process in scenarios with large traffic flow, with a detection accuracy of 82.2%, which is less time-consuming, easy to deploy and high efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968433A_ABST
    Figure CN118968433A_ABST
Patent Text Reader

Abstract

The invention provides a driver and passenger non-dressed detection method based on image recognition, and the method comprises the following steps: inputting any traffic checkpoint vehicle picture with the size of M * N, and carrying out the detection through a front vehicle window detection model MODEL1 obtained through the training of yolov5s, and obtaining the coordinates (x1, y1) of the upper left corner and the coordinates (x2, y2) of the lower right corner of the front vehicle window of a vehicle; a front window image is intercepted from the vehicle image, a driver and passenger detection model MODEL2 obtained through YOLOv5s training is used for detection, and driver and passenger position coordinates are obtained and include the upper left corner coordinates (left * 1, left * 1) and the lower right corner coordinates (right * 1, right * 1) and the upper left corner coordinates (left * 2, left * 2) and the lower right corner coordinates (right * 2, right * 2); and performing classification by using a classification model MODEL3 which is obtained by training and verifying according to a darknet53 network and is used for judging whether the driver and the passengers wear clothes or not. According to the invention, the problems of high cost and low efficiency of the existing manual detection method are solved, and the final detection precision is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention provides a method for detecting a driver or passenger not wearing clothes based on image recognition, and belongs to the technical field of image processing and pattern recognition in the computer field. Background Art

[0002] With the rapid development of the automobile industry, the number of motor vehicles and drivers is increasing rapidly. The increasing number of motor vehicles and drivers poses a huge challenge to road traffic safety. Under the existing traffic laws and safety standards, it is illegal and unsafe for drivers and passengers to be unclothed. Drivers and passengers need to wear appropriate clothing while driving to ensure the safety of themselves and other road users. Traditional detection of unclothed drivers and passengers mainly relies on manual judgment of road checkpoint images, which requires huge manpower costs. The detection accuracy will decrease during the long detection process, and it cannot meet the requirements of real-time and rapid detection. With the development of computer deep learning technology, the use of deep learning technology for analysis will reduce a lot of investment.

[0003] At present, the method of detecting whether a driver or passenger is not wearing clothing is to manually observe the vehicle images at the road checkpoints to determine whether the driver or passenger is wearing clothing. Manual observation requires a lot of manpower and cannot be performed for a long time. In addition, during holidays and other times when there is heavy traffic, manual inspection cannot determine multiple vehicles in the image in a short time. Summary of the invention

[0004] The present invention provides a driver's nakedness detection technology based on image processing and recognition, which performs recognition and detection on road checkpoint images. Under the premise of ensuring the recognition speed, it has high recognition accuracy and can realize rapid and automatic detection of the nakedness of drivers and passengers.

[0005] The specific technical solution is:

[0006] A method for detecting a driver or passenger not wearing clothing based on image recognition comprises the following steps:

[0007] Step 1: Input any image of a vehicle at a traffic checkpoint of size M*N, and use the front window detection model MODEL1 trained by yolov5s to detect it, and get the upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2) of the front window of the vehicle;

[0008] Step 2: According to the front window coordinates (x1, y1) and (x2, y2) obtained in step 1, the front window image is cut out from the vehicle image, and the driver and passenger detection model MODEL2 trained by YOLOv5s is used for detection to obtain the driver and passenger position coordinates: the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2);

[0009] Step 3: According to step 2, the coordinates of the driver and passenger position are obtained: the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2). The driver and passenger image is cut out from the front window image obtained in step 2, and then classified using the driver and passenger clothing classification model MODEL3 obtained by training and verification based on the darknet53 network.

[0010] The detailed method is:

[0011] Step 1 includes the following sub-steps:

[0012] Step 11, data preparation:

[0013] Obtain 1,500 color images of various vehicles’ front faces taken by roadside flashlights;

[0014] Step 12, data annotation:

[0015] Label the front window of the bayonet image obtained in step 11; manually label 1500 images using LabelImg software, mark the front window position in each image, and obtain the labeled label file; divide the image file obtained in step 11 and the corresponding label file into a training set of 1200 and a test set of 300 in a ratio of 4:1 to obtain a vehicle image dataset;

[0016] Step 13: Model training and verification:

[0017] Use the training set in the vehicle image dataset in step 12 to train the YOLOv5s model, adjust the training category classes = 1, the number of iterations epochs = 900, the number of samples selected for one training batch_size = 32, the rectangle training rect = True, the image size imgsz = 992, and the other training parameters use the default settings; after training, the front window detection model MODEL1 is obtained. The average detection time of MODEL1 for a single image is 15ms, and the verification accuracy of the test set is 99.7%;

[0018] Step 14, front window detection:

[0019] Input a traffic checkpoint image of any size M*N, and use the front window detection model MODEL1 to detect the upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2) of the vehicle's front window.

[0020] Step 2 includes the following sub-steps:

[0021] Step 21, obtain the front window image:

[0022] Using the coordinate values ​​of the upper left corner and the lower right corner of the front window obtained in step 14, a total of 1523 images of the front window area are cut out from the camera vehicle image;

[0023] Step 22, data labeling:

[0024] The front window images obtained in step 21 are annotated with front windows; 1523 images are manually annotated using LabelImg software, and the positions of the drivers and passengers in each image are annotated to obtain an annotated label file; the image files obtained in step 21 and the corresponding label files are divided into a training set of 1223 images and a test set of 300 images in a ratio of 4:1 to obtain a front window data set;

[0025] Step 23, model training and verification:

[0026] Use the training set in the front window data set of step 22 to train the YOLOv5s model, adjust the training category classes = 1, the number of iterations epochs = 900, the number of samples selected for one training batch_size = 32, the rectangle training rect = True, the image size imgsz = 992, and the other training parameters use the default settings; after the training is completed, the driver and passenger detection model MODEL2 is obtained. The average detection time of MODEL2 for a single image is 15ms, and the verification accuracy of the test set is 99.5%;

[0027] Step 24: Driver and passenger detection:

[0028] The front window image captured in step 21 is detected using the driver and occupant detection model MODEL2 to obtain the driver and occupant coordinates in the image: the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2).

[0029] Step 3 includes the following sub-steps:

[0030] Step 31: Data Preparation:

[0031] Using the coordinate values ​​of the driver and passenger obtained in step 24, the front window image obtained in step 21 is intercepted to obtain a total of 1955 images of the driver and passenger;

[0032] Step 32: Dataset creation:

[0033] The images obtained in step 31 are manually classified into three categories: 0-dressed, 1-undressed, and 2-cannot be determined; the three categories of image files are divided into a test set of 390 images and a training set of 1565 images in a ratio of 1:4, and the image file names corresponding to the test set and the training set are written into the corresponding files to obtain the label files, and the data set of whether the driver and passengers are dressed is obtained for training the classification model;

[0034] Step 33: Model training and testing:

[0035] Use the darknet53 network to train the data set obtained in step 32, modify the cfg configuration file to adjust the training parameters, training category filters = 3, iterations max_batches = 900, batch = 64, subdivisions = 16, image size height = 480, weight = 480, learning rate learning_rate = 0.001, and use the default settings for other training parameters; after training, the driver and passenger unclothed classification model MODEL3 is obtained, and the test set verifies that the accuracy of driver and passenger unclothed recognition reaches 82.2%, and the average detection time is 14ms;

[0036] Step 34: Identification of passengers without clothing:

[0037] The image of the driver and passenger area captured in step 31 is used as input, and the trained MODEL3 model is used for reclassification, and finally three classification results of whether the driver and passenger are dressed are obtained: 0-dressed, 1-undressed, and 2-cannot be determined.

[0038] The present invention adopts a three-stage detection method, first detecting the position of the front window, detecting the driver and passengers according to the position of the front window, and finally detecting whether the driver and passengers are wearing clothes. The three-stage detection method greatly guarantees the detection accuracy. The present invention uses YOLOv5s as the detection network, which overcomes the problems of high cost and low efficiency of the existing manual detection method, and uses darknet53 as the classification network to ensure the final detection accuracy. It can still detect quickly in scenes with large traffic volume. The example of the present invention is simple and feasible, with an accuracy of 82.2% after training, and the fastest processing of a picture reaches 50ms. Compared with the existing methods, it has low cost, less time consumption, easy deployment, and more efficient detection efficiency, and has obvious advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is the flowchart of step 1;

[0040] Figure 2 is the flowchart of step 2;

[0041] Figure 3 This is the flowchart for step 3. DETAILED DESCRIPTION

[0042] The present invention provides a method for detecting a driver or passenger not wearing clothing based on image recognition, comprising the following steps:

[0043] Step 1: Input any image of a traffic checkpoint vehicle of size M*N, and use the model MODEL1 trained by yolov5s to detect it, and get the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2) of the vehicle's front window.

[0044] like Figure 1 As shown, step 1 specifically includes the following sub-steps:

[0045] Among them, steps 11-13 are for data set preparation and model training and testing; step 14 is to use the front window detection model MODEL1 to detect the bayonet image of step 11:

[0046] Step 11, data preparation:

[0047] A total of 1,500 color images of the front faces of various vehicles were obtained using flash lights at road bays.

[0048] Step 12, data annotation:

[0049] Label the front window of the bayonet image obtained in step 11. Use LabelImg software to manually label 1500 images, mark the front window position in each image, and obtain the labeled label file. The image file obtained in step 11 and the corresponding label file are divided into 1200 training sets and 300 test sets in a ratio of 4:1 to obtain a vehicle image dataset.

[0050] Step 13: Model training and verification:

[0051] Use the training set in the vehicle image dataset in step 12 to train the YOLOv5s model, adjust the training category classes = 1, the number of iterations epochs = 900, the number of samples selected for one training batch_size = 32, the rectangle training rect = True, the image size imgsz = 992, and the other training parameters use the default settings. After training, the front window detection model MODEL1 is obtained. The average detection time of MODEL1 for a single image is 15ms, and the verification accuracy of the test set is 99.7%.

[0052] Step 14, front window detection:

[0053] Input a traffic checkpoint image of any size M*N, and use the front window detection model MODEL1 to detect the upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2) of the vehicle's front window.

[0054] Step 2: According to the front window coordinates (x1, y1) and (x2, y2) obtained in step 1, the front window image is cut out from the vehicle image, and the driver and passenger detection model MODEL2 trained by YOLOv5s is used for detection to obtain the driver and passenger position coordinates: the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2).

[0055] like Figure 2 As shown, step 2 specifically includes the following sub-steps:

[0056] Among them, steps 21-23 are for data set preparation and model training and testing; step 24 is to use the driver and passenger detection model MODEL2 to detect the front window image of step 21:

[0057] Step 21, obtain the front window image:

[0058] Using the coordinate values ​​of the upper left corner and the lower right corner of the front window obtained in step 14, a total of 1523 images of the front window area are cut out from the camera vehicle image.

[0059] Step 22, data labeling:

[0060] The front window images obtained in step 21 are annotated. 1523 images are manually annotated using LabelImg software, and the positions of the drivers and passengers in each image are annotated to obtain the annotated label file. The image files obtained in step 21 and the corresponding label files are divided into a training set of 1223 images and a test set of 300 images in a ratio of 4:1 to obtain the front window data set.

[0061] Step 23, model training and verification:

[0062] Use the training set in the front window data set in step 22 to train the YOLOv5s model, adjust the training category classes = 1, the number of iterations epochs = 900, the number of samples selected for one training batch_size = 32, the rectangle training rect = True, the image size imgsz = 992, and the rest of the training parameters use the default settings. After training, the driver and passenger detection model MODEL2 is obtained. The average detection time of MODEL2 for a single image is 15ms, and the verification accuracy of the test set is 99.5%.

[0063] Step 24: Driver and passenger detection:

[0064] The front window image intercepted in step 21 is detected using the driver and passenger detection model MODEL2 to obtain the driver and passenger coordinates in the image): the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2);

[0065] Step 3: According to step 2, the coordinates of the driver and passenger position are obtained: the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2). The driver and passenger image is cut out from the front window image obtained in step 2, and then the driver and passenger clothing classification model MODEL3 obtained by training and verification based on the darknet53 network is used for classification. Three classification results are obtained: 0-dressed, 1-undressed, and 2-cannot be judged.

[0066] like Figure 3 As shown, step 3 specifically includes the following sub-steps:

[0067] Steps 31-33 are for data set preparation and model training and testing; Step 34 is for re-identifying the driver image in step 31 using the driver unclothed classification model MODEL3:

[0068] Step 31: Data Preparation:

[0069] Using the coordinate values ​​of the driver and passenger obtained in step 24, the front window image obtained in step 21 is cropped to obtain a total of 1955 images of the driver and passenger.

[0070] Step 32: Dataset creation:

[0071] The images obtained in step 31 are manually classified into three categories: 0-dressed, 1-undressed, and 2-unable to determine. The three types of image files are divided into a test set of 390 images and a training set of 1565 images at a ratio of 1:4, and the image file names corresponding to the test set and the training set are written into the corresponding files to obtain the label file, and the driver and passenger clothing dataset is obtained for training the classification model.

[0072] Step 33: Model training and testing:

[0073] Use the darknet53 network to train the data set obtained in step 32, modify the cfg configuration file to adjust the training parameters, training category filters = 3, iterations max_batches = 900, batch = 64, subdivisions = 16, image size height = 480, weight = 480, learning rate learning_rate = 0.001, and use the default settings for other training parameters. After training, the driver and passenger unclothed classification model MODEL3 is obtained. The test set verifies that the accuracy of driver and passenger unclothed recognition is 82.2%, and the average detection time is 14ms.

[0074] Step 34: Identification of passengers without clothing:

[0075] The image of the driver and passenger area captured in step 31 is used as input, and the trained MODEL3 model is used for reclassification, and finally three classification results of whether the driver and passenger are dressed are obtained: 0-dressed, 1-undressed, and 2-cannot be determined.

Claims

1. A method for detecting drivers and passengers not wearing clothes based on image recognition. It is characterized in that The following steps are involved: Step 1: Input any image of a vehicle at a traffic checkpoint of size M*N, and use the front window detection model MODEL1 trained by yolov5s to detect it, and get the upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2) of the front window of the vehicle; Step 2: According to the front window coordinates (x1, y1) and (x2, y2) obtained in step 1, the front window image is cut out from the vehicle image, and the driver and passenger detection model MODEL2 trained by YOLOv5s is used for detection to obtain the driver and passenger position coordinates: the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2); Step 3: According to step 2, the coordinates of the driver and passenger position are obtained: the upper left corner coordinates (left_x1, left_y1) and the lower right corner coordinates (right_x1, right_y1), as well as the upper left corner coordinates (left_x2, left_y2) and the lower right corner coordinates (right_x2, right_y2). The driver and passenger image is cut out from the front window image obtained in step 2, and then classified using the driver and passenger clothing classification model MODEL3 obtained by training and verification based on the darknet53 network.

2. According to the method for detecting unclothed drivers and passengers based on image recognition according to claim 1, It is characterized in that Step 1 includes the following sub-steps: Step 11, data preparation: Obtain 1,500 color images of various vehicles’ front faces taken by roadside flashlights; Step 12, data annotation: Label the front window of the bayonet image obtained in step 11; manually label 1500 images using LabelImg software, mark the front window position in each image, and obtain the labeled label file; divide the image file obtained in step 11 and the corresponding label file into a training set of 1200 and a test set of 300 in a ratio of 4:1 to obtain a vehicle image dataset; Step 13: Model training and verification: Use the training set in the vehicle image dataset in step 12 to train the YOLOv5s model, adjust the training category classes = 1, the number of iterations epochs = 900, the number of samples selected for one training batch_size = 32, the rectangle training rect = True, the image size imgsz = 992, and the other training parameters use the default settings; after training, the front window detection model MODEL1 is obtained. The average detection time of MODEL1 for a single image is 15ms, and the verification accuracy of the test set is 99.7%; Step 14, front window detection: Input a traffic checkpoint image of any size M*N, and use the front window detection model MODEL1 to detect the upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2) of the vehicle's front window.

3. According to the method for detecting unclothed drivers and passengers based on image recognition as claimed in claim 1, It is characterized in that Step 2 includes the following sub-steps: Step 21, obtain the front window image: Using the coordinate values ​​of the upper left corner and the lower right corner of the front window obtained in step 14, a total of 1523 images of the front window area are cut out from the camera vehicle image; Step 22, data labeling: The front window images obtained in step 21 are annotated with front windows; 1523 images are manually annotated using LabelImg software, and the positions of the drivers and passengers in each image are annotated to obtain an annotated label file; the image files obtained in step 21 and the corresponding label files are divided into a training set of 1223 images and a test set of 300 images in a ratio of 4:1 to obtain a front window data set; Step 23, model training and verification: Use the training set in the front window data set of step 22 to train the YOLOv5s model, adjust the training category classes = 1, the number of iterations epochs = 900, the number of samples selected for one training batch_size = 32, the rectangle training rect = True, the image size imgsz = 992, and the other training parameters use the default settings; after the training is completed, the driver and passenger detection model MODEL2 is obtained. The average detection time of MODEL2 for a single image is 15ms, and the verification accuracy of the test set is 99.5%; Step 24: Driver and passenger detection: The front window image captured in step 21 is detected using the driver and passenger detection model MODEL2 to obtain the driver and passenger coordinates (left_x1, left_y1), (right_x1, right_y1) and (left_x2, left_y2), (right_x2, right_y2) in the image.

4. The method for detecting unclothed drivers and passengers based on image recognition according to claim 1, It is characterized in that Step 3 includes the following sub-steps: Step 31: Data Preparation: Using the coordinate values ​​of the driver and passenger obtained in step 24, the front window image obtained in step 21 is intercepted to obtain a total of 1955 images of the driver and passenger; Step 32: Dataset creation: The images obtained in step 31 are manually classified into three categories: 0-dressed, 1-undressed, and 2-cannot be determined; the three categories of image files are divided into a test set of 390 images and a training set of 1565 images in a ratio of 1:4, and the image file names corresponding to the test set and the training set are written into the corresponding files to obtain the label files, and the data set of whether the driver and passengers are dressed is obtained for training the classification model; Step 33: Model training and testing: Use the darknet53 network to train the data set obtained in step 32, modify the cfg configuration file to adjust the training parameters, training category filters = 3, iterations max_batches = 900, batch = 64, subdivisions = 16, image size height = 480, weight = 480, learning rate learning_rate = 0.001, and use the default settings for other training parameters; after training, the driver and passenger unclothed classification model MODEL3 is obtained, and the test set verifies that the accuracy of driver and passenger unclothed recognition reaches 82.2%, and the average detection time is 14ms; Step 34: Identification of passengers without clothing: The image of the driver and passenger area captured in step 31 is used as input, and the trained MODEL3 model is used for reclassification, and finally three classification results of whether the driver and passenger are dressed are obtained: 0-dressed, 1-undressed, and 2-cannot be determined.

Citation Information

Patent Citations

  • Deep learning-based safety belt wearing identification method

    CN104657752A

  • Driver safety belt wearing recognition method used for intelligent transportation system

    CN108182385A

  • Safety belt detection method and device based on deep learning

    CN113516049A

  • Safety belt detection method and device, electronic equipment and storage medium

    CN116071734A