Method and device for rapidly detecting abnormal behavior of driver, vehicle and storage medium

By using deep convolutional neural network model in the driver monitoring system combined with face recognition technology, the abnormal behavior of the driver is identified, and the problem of reducing detection accuracy caused by external environmental impact in the prior art is solved, and driving safety is improved.

CN118982815AInactive Publication Date: 2024-11-19CHERY AUTOMOBILE CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410701346.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-05-31
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, the process of identifying driver status through computer vision technology is susceptible to the influence of the external environment, resulting in a decrease in detection accuracy.

Method used

By collecting the driver's image data, using the trained face recognition model to obtain the face position coordinate information, and combining the deep convolutional neural network model to detect the face image data abnormal behavior to identify the driver's current behavior status, including Normal and abnormal states such as fatigue, distraction, drinking water and smoking, sending alarm messages.

Benefits of technology

It effectively reduces external interference, improves the accuracy of driver abnormal behavior recognition, and improves driving safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982815A_ABST
    Figure CN118982815A_ABST
Patent Text Reader

Abstract

The invention discloses a driver abnormal behavior rapid detection method and device, a vehicle and a storage medium, and relates to the technical field of driver monitoring, and the method comprises the steps: collecting image data in the driving process of a driver; the image data comprises a face image of the driver; inputting the image data into a trained face recognition model to obtain position coordinate information of the face of the driver in the image data; intercepting face image data of the driver from the image data based on the position coordinate information; and inputting the face image data into the trained driver abnormal behavior detection model to obtain the current behavior state of the driver. The technical problem that in the prior art, the process of recognizing the state of a driver through a computer vision technology is prone to being affected by the external environment, and consequently the detection accuracy is reduced is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of driver monitoring, and in particular to a method, device, vehicle and storage medium for rapid detection of abnormal driver behavior. Background Art

[0002] Driver behavior is the basis of driving safety. Reducing human errors to avoid traffic accidents or reduce traffic casualties is an effective means. Therefore, how to perceive driver behavior without affecting the driver's normal driving plays a very important role in improving road safety.

[0003] With the rapid development of computer vision technology, driver status monitoring algorithms and systems based on computer vision technology have become a research hotspot, which can identify the driver's status contactlessly through a camera. However, in the prior art, the process of identifying the driver's status through computer vision technology is easily affected by the external environment, resulting in reduced detection accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, vehicle and storage medium for quickly detecting abnormal driver behavior in order to solve at least one of the above technical problems.

[0005] In the first aspect, an embodiment of the present invention provides a method for quickly detecting abnormal driver behavior, comprising: collecting image data of the driver during driving; the image data includes a facial image of the driver; inputting the image data into a trained face recognition model to obtain position coordinate information of the driver's face in the image data; based on the position coordinate information, extracting the facial image data of the driver from the image data; inputting the facial image data into a trained driver abnormal behavior detection model to obtain the driver's current behavior state.

[0006] Furthermore, before inputting the image data into a trained face recognition model to obtain the position coordinate information of the driver's face in the image data, the method also includes: collecting an image data set of the driver during driving, and annotating the faces in the image data set to establish a face recognition data set; and training a preset face recognition model based on the face recognition data set to obtain the trained face recognition model.

[0007] Furthermore, the trained driver abnormal behavior detection model includes a deep convolutional neural network model; before inputting the facial image data into the trained driver abnormal behavior detection model to obtain the driver's current behavior state, the method also includes: annotating the driver's facial image data set captured using the face position coordinate information to establish a driver state recognition data set; training a preset deep convolutional neural network model based on the driver state recognition data set to obtain the trained driver abnormal behavior detection model.

[0008] Furthermore, the current behavior state of the driver includes: a normal state and an abnormal behavior state; the abnormal behavior state includes fatigue, distraction, drinking water and smoking.

[0009] Furthermore, after obtaining the current behavior state of the driver, the method further includes: if the current behavior state of the driver is the abnormal behavior state, issuing an alarm message to remind the driver to regulate driving behavior.

[0010] In the second aspect, an embodiment of the present invention also provides a device for rapid detection of abnormal driver behavior, comprising: an image acquisition module, a face recognition module, a face capture module and a behavior detection module; wherein the image acquisition module is used to collect image data of the driver during driving; the image data includes a face image of the driver; the face recognition module is used to input the image data into a trained face recognition model to obtain the position coordinate information of the driver's face in the image data; the face capture module is used to capture the face image data of the driver in the image data based on the position coordinate information; the behavior detection module is used to input the face image data into a trained driver abnormal behavior detection model to obtain the current behavior state of the driver.

[0011] Furthermore, it also includes: a first training module and a second training module; wherein, the first training module is used to collect an image data set of the driver during driving, and to annotate the faces in the image data set to establish a face recognition data set; based on the face recognition data set, a preset face recognition model is trained to obtain the trained face recognition model; the second training module is used to annotate the face image data set of the driver captured using the position coordinate information of the face to establish a driver state recognition data set; based on the driver state recognition data set, a preset deep convolutional neural network model is trained to obtain the trained driver abnormal behavior detection model.

[0012] Furthermore, the driver's current behavior state includes: a normal state and an abnormal behavior state; the abnormal behavior state includes fatigue, distraction, drinking water and smoking; the device also includes an alarm module, which is used to issue an alarm message to remind the driver to regulate driving behavior if the driver's current behavior state is the abnormal behavior state.

[0013] In a third aspect, an embodiment of the present invention further provides a vehicle for rapid detection of abnormal driver behavior, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described in the first aspect above is implemented.

[0015] The embodiments of the present invention provide a method, device, vehicle and storage medium for rapid detection of abnormal driver behavior. The driver's facial image data captured by the in-vehicle camera, combined with the powerful learning ability of a deep convolutional neural network, can effectively reduce external interference and accurately identify the driver's current behavior state through visual information, thereby improving driving safety and alleviating the technical problem in the prior art that the process of identifying the driver's state through computer vision technology is easily affected by the external environment, resulting in reduced detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0017] Figure 1 A flowchart of a method for quickly detecting abnormal driver behavior provided by an embodiment of the present invention;

[0018] Figure 2 A schematic diagram of the architecture of a driver abnormal behavior detection model provided by an embodiment of the present invention;

[0019] Figure 3 A schematic diagram of a device for rapid detection of abnormal driver behavior provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0021] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present invention belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present invention.

[0022] Embodiment 1

[0023] Figure 1 1 is a flow chart of a method for rapid detection of abnormal driver behavior according to an embodiment of the present invention. Figure 1 As shown, the method specifically comprises the following steps:

[0024] Step S102, collecting image data of the driver during driving; the image data includes the driver's face image.

[0025] Step S104: input the image data into a trained face recognition model to obtain the position coordinate information of the driver's face in the image data.

[0026] Preferably, the face recognition model includes a deep convolutional neural network model.

[0027] Step S106: extracting the driver's facial image data from the image data based on the position coordinate information.

[0028] Step S108, inputting the facial image data into the trained driver abnormal behavior detection model to obtain the driver's current behavior state.

[0029] The embodiment of the present invention provides a method for quickly detecting abnormal driver behavior. The driver's facial image data captured by the in-vehicle camera is combined with the powerful learning ability of the deep convolutional neural network to effectively reduce external interference and accurately identify the driver's current behavior state through visual information, thereby improving driving safety.

[0030] Specifically, before step S104, the method provided by the embodiment of the present invention further includes the following steps:

[0031] Collect image datasets of drivers while driving, annotate faces in the image datasets, and establish face recognition datasets;

[0032] The preset face recognition model is trained based on the face recognition data set to obtain a trained face recognition model.

[0033] Preferably, the trained driver abnormal behavior detection model includes a deep convolutional neural network model.

[0034] Specifically, before step S108, the method provided by the embodiment of the present invention further includes the following steps:

[0035] Annotate the driver's face image dataset extracted by using the position coordinate information of the face to establish a driver status recognition dataset;

[0036] The preset deep convolutional neural network model is trained based on the driver state recognition dataset to obtain a trained driver abnormal behavior detection model.

[0037] Figure 2 Schematic diagram of the architecture of a driver abnormal behavior detection model provided according to an embodiment of the present invention. Figure 2 As shown in the figure, the driver abnormal behavior detection model consists of multiple convolutional layers, multiple pooling layers and fully connected layers.

[0038] Preferably, the current behavior state of the driver includes: a normal state and an abnormal behavior state; wherein the abnormal behavior state includes fatigue, distraction, drinking water and smoking.

[0039] After step S108, the method provided by the embodiment of the present invention further includes: if the current behavior state of the driver is an abnormal behavior state, issuing a warning message to remind the driver to regulate driving behavior.

[0040] From the above description, it can be seen that an embodiment of the present invention provides a method for quickly detecting abnormal driver behavior. First, a face recognition model is used to identify the position coordinate information of the driver's face, and then the position coordinate information is used to intercept the face image data of the driver in the image data. Finally, a deep learning model is used to identify the behavior state of the intercepted face image, which can effectively reduce external influences and improve the accuracy of identifying the driver's abnormal behavior state.

[0041] Embodiment 2

[0042] Figure 3 Schematic diagram of a device for rapid detection of abnormal driver behavior according to an embodiment of the present invention. Figure 3 As shown, it includes: an image acquisition module 10, a face recognition module 20, a face interception module 30 and a behavior detection module 40.

[0043] Specifically, the image acquisition module 10 is used to acquire image data of the driver during driving; the image data includes the driver's facial image.

[0044] The face recognition module 20 is used to input the image data into a trained face recognition model to obtain the position coordinate information of the driver's face in the image data.

[0045] The face capture module 30 is used to capture the driver's face image data from the image data based on the position coordinate information.

[0046] The behavior detection module 40 is used to input the face image data into the trained driver abnormal behavior detection model to obtain the current behavior state of the driver.

[0047] Specifically, Figure 3 As shown, it also includes: a first training module 50 and a second training module 60.

[0048] The first training module 50 is used to collect an image data set of the driver during driving, and to annotate faces in the image data set to establish a face recognition data set; and to train a preset face recognition model based on the face recognition data set to obtain a trained face recognition model;

[0049] The second training module 60 is used to annotate the driver's facial image data set captured using the facial position coordinate information to establish a driver status recognition data set; and to train a preset deep convolutional neural network model based on the driver status recognition data set to obtain a trained driver abnormal behavior detection model.

[0050] Preferably, the current behavior state of the driver includes: a normal state and an abnormal behavior state; the abnormal behavior state includes fatigue, distraction, drinking water and smoking.

[0051] Specifically, Figure 3 As shown, the device provided by the embodiment of the present invention further includes an alarm module 70, which is used to issue an alarm message to remind the driver to regulate driving behavior if the current behavior state of the driver is an abnormal behavior state.

[0052] An embodiment of the present invention also provides a vehicle for rapid detection of abnormal driver behavior, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method in the above-mentioned embodiment 1 is implemented.

[0053] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, the method in the above-mentioned embodiment 1 is implemented.

[0054] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

[0055] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for rapid detection of abnormal driver behavior, characterized in that: include: Collect image data of the driver during driving; The image data includes a facial image of the driver; Inputting the image data into a trained face recognition model to obtain the position coordinate information of the driver's face in the image data; Based on the position coordinate information, extracting the driver's facial image data from the image data; The face image data is input into a trained driver abnormal behavior detection model to obtain the current behavior state of the driver.

2. The method according to claim 1, characterized in that Before inputting the image data into a trained face recognition model to obtain the position coordinate information of the driver's face in the image data, the method further includes: Collecting image datasets of drivers during driving, and annotating faces in the image datasets to establish face recognition datasets; A preset face recognition model is trained based on the face recognition data set to obtain the trained face recognition model.

3. The method according to claim 1, characterized in that The trained driver abnormal behavior detection model includes a deep convolutional neural network model; before inputting the face image data into the trained driver abnormal behavior detection model to obtain the current behavior state of the driver, the method further includes: Annotate the driver's face image dataset extracted by using the position coordinate information of the face to establish a driver status recognition dataset; A preset deep convolutional neural network model is trained based on the driver state recognition data set to obtain the trained driver abnormal behavior detection model.

4. The method according to claim 1, characterized in that: The current behavior state of the driver includes: a normal state and an abnormal behavior state; the abnormal behavior state includes fatigue, distraction, drinking water and smoking.

5. The method according to claim 4, characterized in that After obtaining the current behavior state of the driver, the method further includes: If the current behavior state of the driver is the abnormal behavior state, a warning message is issued to remind the driver to regulate driving behavior.

6. A device for rapid detection of abnormal driver behavior, characterized in that: include: Image acquisition module, face recognition module, face capture module and behavior detection module; among them, The image acquisition module is used to collect image data of the driver during driving; the image data includes a facial image of the driver; The face recognition module is used to input the image data into a trained face recognition model to obtain the position coordinate information of the driver's face in the image data; The face capture module is used to capture the driver's face image data from the image data based on the position coordinate information; The behavior detection module is used to input the face image data into a trained driver abnormal behavior detection model to obtain the current behavior state of the driver.

7. The device according to claim 6, characterized in that Also includes: A first training module and a second training module; wherein, The first training module is used to collect an image data set of the driver during driving, and to annotate faces in the image data set to establish a face recognition data set; and to train a preset face recognition model based on the face recognition data set to obtain the trained face recognition model; The second training module is used to annotate the driver's facial image data set captured using the position coordinate information of the face, and establish a driver status recognition data set; based on the driver status recognition data set, a preset deep convolutional neural network model is trained to obtain the trained driver abnormal behavior detection model.

8. The device according to claim 6, characterized in that The driver's current behavior state includes: a normal state and an abnormal behavior state; the abnormal behavior state includes fatigue, distraction, drinking water and smoking; the device also includes an alarm module, which is used to issue an alarm message to remind the driver to regulate driving behavior if the driver's current behavior state is the abnormal behavior state.

9. A vehicle for rapid detection of abnormal driver behavior, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Cited By

  • Privacy protection method, device and system for in-vehicle camera, computer equipment and medium

    CN119622800A

  • Abnormal driving behavior detection method and system, electronic equipment and storage medium

    CN120747926A