A driver smoking detection method based on multiple models

By combining smoking gestures and cigarette area recognition models, and using example segmentation technology, the problem of high false alarm rate of driver smoking detection in the prior art is solved, and more stable and accurate detection results are achieved.

CN113591615BActive Publication Date: 2024-11-15SHARPVISION CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110795174.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-14
Publication Date
2024-11-15
Estimated Expiration
2041-07-14

AI Technical Summary

Technical Problem

Existing driver smoking detection methods are easily affected by the environment, leading to misjudgment, and traditional machine learning methods are easily affected by light, while deep learning methods are difficult to accurately detect cigarette areas, resulting in a high false alarm rate.

Method used

The multi-model-based driver smoking detection method is adopted, and the smoking gesture recognition model and the cigarette area recognition model are combined, and the instance segmentation technology recognizes the smoking behavior in the image to reduce the false alarm rate.

Benefits of technology

By detecting smoking gestures and cigarettes, the instance segmentation technology avoids background influence, improves detection stability, reduces false alarm rates, and enhances detection accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113591615B_ABST
    Figure CN113591615B_ABST
Patent Text Reader

Abstract

This invention discloses a multi-model-based method for detecting driver smoking. The method includes acquiring an image of the driver smoking detection area, inputting this image into a smoking gesture recognition model and a cigarette region recognition model to obtain probability maps for smoking gesture recognition and cigarette recognition, analyzing the results of these probability maps, and detecting that the driver is smoking when both a smoking gesture and a cigarette are simultaneously detected. This multi-model-based driver smoking detection method reduces the false alarm rate by detecting both smoking gestures and cigarettes. Furthermore, the cigarette detection uses instance segmentation, which avoids the influence of background pixels for elongated targets like cigarettes, thus improving the stability of the final detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a driver smoking detection method based on multiple models. Background Art

[0002] With the development of economy and automobile technology, there are more and more cars in the world and traffic is becoming more and more developed. However, people's safety awareness has not been enhanced accordingly, which has led to more and more frequent traffic accidents; therefore, safety assisted driving technology has also received more and more attention. Drivers' smoking behavior can cause distracted driving, so smoking detection technology is an important part of safe driving technology. Traditional smoking detection methods include applying traditional machine learning methods and applying deep learning methods. The traditional machine learning method extracts the LBP features, color features, and irregular driver mouth area features of the smoke, and then uses SVM to classify whether the smoker is smoking again. This method is easily affected by the environment, such as lighting, which may lead to misjudgment. The deep learning method uses the target detection model to detect cigarettes, and obtains the classification and location information of cigarettes. Since the area of ​​cigarettes occupies a small area of ​​the smoking area, this method is also prone to misjudgment. Summary of the invention

[0003] The object of the present invention is to provide a driver smoking detection method based on multiple models which can reduce the false alarm rate of smoking detection.

[0004] The present invention provides a driver smoking detection method based on multiple models, comprising:

[0005] Acquire the driver smoking detection area image;

[0006] Inputting the driver's smoking detection area image into the smoking gesture recognition model and the cigarette area recognition model to obtain a probability map of smoking gesture recognition and a probability map of cigarette recognition;

[0007] The results of the probability map of smoking gesture recognition and the probability map of cigarette recognition are analyzed. When the smoking gesture and the cigarette are recognized at the same time, it is detected that the driver is smoking.

[0008] The multi-model based driver smoking detection method adopted by the present invention can reduce the false alarm rate of smoking detection by detecting smoking gestures and cigarettes, and cigarette detection adopts instance segmentation, that is, identifying the probability that each pixel of the image belongs to a cigarette. For slender targets such as cigarettes, instance segmentation avoids the influence of background pixels, thereby improving the stability of the final detection;. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A schematic flow chart of a driver smoking detection method based on multiple models provided in an embodiment of the present invention;

[0010] Figure 2 A schematic diagram of a smoking gesture recognition model provided by an embodiment of the present invention;

[0011] Figure 3 A schematic diagram of a cigarette region recognition model provided by an embodiment of the present invention;

[0012] Figure 4 A schematic diagram of a sample image generation process including a smoking gesture provided by an embodiment of the present invention;

[0013] Figure 5 A schematic diagram of a flow chart of generating a sample image of a cigarette region provided by an embodiment of the present invention;

[0014] Figure 6 A schematic diagram of an example of cigarette detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] like Figure 1 As shown, an embodiment of the present invention provides a driver smoking detection method based on multiple models, including acquiring an image through a camera device, performing full-image face detection on the image, obtaining the area of ​​the face, calculating the area of ​​each face area, extracting the face image with the largest area, inputting the face image into a face alignment model, extracting a series of face images with face feature points, extracting images of the area around the driver's mouth based on the face images with face feature points, obtaining the driver's smoking detection area image based on the extracted images of the area around the driver's mouth, inputting the driver's smoking detection area image into a smoking gesture recognition model and a cigarette area recognition model to obtain a probability map of smoking gesture recognition and a probability map of cigarette recognition, analyzing the results of the probability map of smoking gesture recognition and the probability map of cigarette recognition, and when the smoking gesture and the cigarette are recognized at the same time, it is detected that the driver is smoking.

[0016] The driver smoking detection area image is input into the smoking gesture recognition model and the cigarette area recognition model to obtain the probability map of smoking gesture recognition and the probability map of cigarette recognition, which are specifically:

[0017] The smoking gesture recognition model consists of a convolutional module, a MobileNetV2 module, and a fully connected module. The small area image of smoking detection, such as Figure 2 As shown, first zoom to ,in and represents height and width; then, after a series of convolution modules, MobileNetV2 modules, fully connected modules and softmax operations, the probability map of smoking gesture recognition is finally obtained.

[0018] The cigarette area recognition model consists of a convolution module, a MobileNetV2 module, and a MobileNetV2 funnel module. The large area image of smoking detection is as follows: Figure 3 As shown, first zoom to ; Then, after a series of calculation operations of convolution module, MobileNetV2 module, and MobileNetV2 funnel module, the probability map of cigarette recognition is finally obtained.

[0019] The smoking gesture recognition model is obtained through training and learning of the deep learning framework. Training and learning requires a large number of samples and sample labels, so the specific process of generating samples required for training the smoking gesture recognition model is as follows: Figure 4 As shown, first obtain the face image, the coordinates of the facial feature points, the smoking gesture with cigarette, and mark the cigarette anchor point; calculate the mean and variance of the pixels of the cheek areas C and D on both sides of the face according to the facial feature points: , in, , Represent the pixel values ​​and pixel numbers of regions C and D respectively. Similarly, calculate the mean value of the smoking gesture and variance ,as follows: , Then, according to the above four formulas, the smoking gesture is standardized so that its variance and mean are equal to those in areas C and D. ,in, The pixel value representing the normalization of the smoking gesture. The normalized smoking gesture is then superimposed on the face image by scaling and rotating, and the smoking anchor point of the smoking gesture is within the mouth area; then the edge of the smoking gesture is Gaussian filtered to eliminate jagged edges; finally, a sample containing the smoking gesture is obtained.

[0020] The cigarette area recognition model is obtained through training and learning of the deep learning framework, and the process of synthesizing samples required for training the cigarette area recognition model is as follows: Figure 5 As shown in the figure, first obtain a face image with feature points, a cigarette, and a cigarette anchor mark; then scale and rotate the cigarette image and superimpose it on the face image, and ensure that the cigarette anchor is in the mouth area; then perform Gaussian filtering on the edge of the cigarette; finally, obtain a face image with a cigarette in it and a cigarette area mark. The generation of sample images with smoking gestures and sample images of cigarette areas does not require manual collection and marking, which greatly reduces the time and development cost. Figure 6 This is an example of cigarette area recognition model detection, where the 2nd, 4th, 6th, and 8th columns are input images, and the 1st, 3rd, 5th, and 7th columns are probability maps of cigarette recognition output by the cigarette area recognition model.

[0021] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A driver smoking detection method based on multiple models, It is characterized in that include: Acquire an image through a camera device, perform full-image face detection on the image, obtain the area of ​​the face, calculate the area of ​​each face area, extract the face image with the largest area, input the face image into a face alignment model, extract a series of face images with face feature points, extract the image of the area around the driver's mouth based on the face image with face feature points, obtain the driver's smoking detection area image based on the extracted image of the area around the driver's mouth, input the driver's smoking detection area image into a smoking gesture recognition model and a cigarette area recognition model to obtain a probability map of smoking gesture recognition and a probability map of cigarette recognition, analyze the results of the probability map of smoking gesture recognition and the probability map of cigarette recognition, and when the smoking gesture and the cigarette are recognized at the same time, it is detected that the driver is smoking; The driver smoking detection area image is input into the smoking gesture recognition model and the cigarette area recognition model to obtain the probability map of smoking gesture recognition and the probability map of cigarette recognition, which are specifically: The smoking gesture recognition model consists of a convolution module, a MobileNetV2 module, and a fully connected module. The small area image of smoking detection is first scaled; then, after a series of convolution modules, MobileNetV2 modules, fully connected modules, and softmax operations, the probability map of smoking gesture recognition is finally obtained; The cigarette area recognition model consists of a convolution module, a MobileNetV2 module, and a MobileNetV2 funnel module. The large area image of smoking detection is first scaled; then it undergoes a series of calculation operations through a convolution module, a MobileNetV2 module, and a MobileNetV2 funnel module, and finally obtains a probability map for cigarette recognition.

2. The driver smoking detection method based on multiple models according to claim 1, It is characterized in that The smoking gesture recognition model is obtained through training and learning of a deep learning framework, and is obtained through training supervision using a sample image set containing smoking gestures.

3. The driver smoking detection method based on multiple models according to claim 1, It is characterized in that The cigarette region recognition model is obtained through training and learning of a deep learning framework, and the model is obtained through training supervision using a set of images containing cigarette samples.