End-to-end driver alcohol detection method and device, storage medium and vehicle

Abstract

The embodiment of the invention provides an end-to-end driver alcohol detection method and device, a storage medium and a vehicle, relates to the technical field of alcohol detection, can realize drunk driving detection without active cooperation of a driver, and has the advantages of high real-time performance and privacy protection. The method provided by the embodiment of the invention comprises the following steps: acquiring face video data of a driver by using a vehicle-mounted image acquisition module; a plurality of video frames in the face video data are preprocessed; sending the plurality of preprocessed video frames into a pre-trained deep learning network model to extract spatial-temporal characteristic signals related to alcohol, and obtaining a drunk driving detection result of whether drunk driving exists; and displaying the drunk driving detection result on a vehicle-mounted display screen.

Description

Technical Field

[0001] The present invention relates to the technical field of alcohol detection, and in particular, to an end-to-end driver alcohol detection method, device, storage medium, and vehicle. Background Art

[0002] Drunk driving is one of the important issues in the global traffic safety field, seriously threatening road traffic safety and people's lives and property. Alcohol can significantly affect a driver's judgment, reaction speed, coordination ability, and concentration, thus leading to traffic accidents. According to statistics from the World Health Organization (WHO), drunk driving is one of the main causes of global traffic deaths. Especially at high-risk times such as nights and holidays, the incidence of drunk driving is relatively high, and the frequency and severity of accidents caused are also more significant. Therefore, accurately and timely detecting the alcohol content of drivers is of great significance for preventing traffic accidents and ensuring road safety. Currently, for the detection of drunk driving behavior, it mainly relies on breath alcohol testers (breath measurement method) and blood alcohol tests (blood sampling method). Among them, breath alcohol testers are the most commonly used devices, which judge the alcohol content by detecting the concentration of alcohol in the driver's exhaled gas. However, these traditional alcohol detection methods have some obvious limitations and drawbacks:

[0003] (1) Require manual cooperation: Breath testing requires the driver's active cooperation, which may affect the accuracy of the test results due to reasons such as fatigue and psychological resistance.

[0004] (2) The detection equipment is large in size and high in cost: Traditional alcohol detection equipment is often large in size, requires specialized personnel to operate, and has a high maintenance cost, making it not very convenient to use.

[0005] (3) Limit the real-time detection ability: Most traditional methods cannot detect the driver's alcohol content in real time, relying on manual inspections or being carried out during traffic police road inspections, resulting in a certain time delay. Summary of the Invention

[0006] To address the above problems, the present invention provides an end-to-end, non-contact driver alcohol detection method, device, storage medium, and vehicle, which can obtain the driver's facial image in real time through an in-vehicle image acquisition module, and use deep learning technology to extract spatio-temporal feature signals and detect drunk driving, thereby realizing drunk driving detection without the driver's active cooperation, and having the advantages of high real-time performance and privacy protection.

[0007] To achieve the above object, according to one aspect of the present invention, an end-to-end driver alcohol detection method is provided.

[0008] An end-to-end driver alcohol detection method according to an embodiment of the present invention includes: using an in-vehicle image acquisition module to acquire face video data of a driver; preprocessing a plurality of video frames in the face video data; sending the preprocessed plurality of video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result on whether the driver is driving under the influence; and displaying the drunk driving detection result on an in-vehicle display screen.

[0009] Preferably, the deep learning network model is trained according to the following steps: acquiring image data of a plurality of drivers of different genders, different ages, and different races before alcohol intake, labeling the image data before alcohol intake with a non-drunk driving label, and determining the image data labeled with the non-drunk driving label as negative samples; acquiring image data of a plurality of drivers of different genders, different ages, and different races after alcohol intake, labeling the image data after alcohol intake with a drunk driving label, and determining the image data labeled with the drunk driving label as positive samples; combining the positive samples and the negative samples into training samples, and using the training samples to train the deep learning network model.

[0010] Preferably, the driver alcohol detection method further includes: using a driver health monitoring system to acquire the driver's health physiological index data and behavioral characteristic data, and determining whether the health physiological index data and the behavioral characteristic data meet preset fatigue driving or sudden illness discrimination conditions; if they meet, sending a reminder of fatigue driving or sudden illness; the step of sending the preprocessed plurality of video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result on whether the driver is driving under the influence includes: sending the preprocessed plurality of video frames, the health physiological index data, and the behavioral characteristic data into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result on whether the driver is driving under the influence.

[0011] Preferably, the driver alcohol detection method further includes: before using the in-vehicle image acquisition module to acquire the driver's face video data, performing identity authentication on the driver to obtain driver identity information; acquiring individual characteristic data corresponding to the driver identity information, using the individual characteristic data to adjust the model parameters of the deep learning network model, using the adjusted deep learning network model to extract the spatio-temporal feature signals and obtain the drunk driving detection result; and after obtaining the drunk driving detection result, associating the drunk driving detection result with the driver identity information.

[0012] Preferably, the driver alcohol detection method further includes: pre-deploying a plurality of computing nodes, and distributing the preprocessing of the plurality of video frames, the extraction of the spatio-temporal feature signals, and the calculation of the drunk driving detection results to the plurality of computing nodes based on distributed computing technology and edge computing technology to improve the computing real-time performance and response speed; after obtaining the drunk driving detection results, if the drunk driving detection results indicate drunk driving, a voice warning and a reminder to stop driving are sent to the driver through the in-vehicle voice system, the in-vehicle safety system or the autonomous driving assistance system is triggered to enhance driving safety, and real-time safety suggestions are provided; after obtaining the drunk driving detection results, the drunk driving detection results are uploaded to the cloud platform and the vehicle owner's intelligent terminal through a wireless network to realize remote monitoring and management of the driver's drunk driving behavior.

[0013] Preferably, the use of the in-vehicle image acquisition module to acquire the face video data of the driver includes: using a plurality of in-vehicle image acquisition modules at different angles to acquire the face video data; the preprocessing includes image denoising, image enhancement, brightness adjustment, face detection and image cropping; the display of the drunk driving detection results on the in-vehicle display screen includes: displaying the drunk driving detection results in the form of an icon or text on the central control display screen or the head-up display device HUD for easy viewing by the driver; the driver alcohol detection method further includes: configuring, controlling and remotely managing the driver alcohol detection through a variety of user input methods, and the user input methods include: voice input, touch operation of the in-vehicle display screen, or operation based on the intelligent terminal application.

[0014] Preferably, the in-vehicle image acquisition module is a near-infrared camera or a red-green-blue RGB camera; the deep learning network model is: 3D convolutional neural network 3D-CNN, long short-term memory network LSTM, gated recurrent unit GRU, spatio-temporal graph convolutional network ST-GCN, spatio-temporal convolutional network TCN, convolutional neural network CNN combined with long short-term memory network LSTM, spatio-temporal attention network, or transformer Transformer model; the behavior feature data includes eye movement data or head tilt data, the individual feature data includes age data, gender data, or driving experience data, the in-vehicle safety system performs automatic deceleration or improves vehicle control stability after being triggered, and the real-time safety suggestions include suggestions to stop and rest or traffic safety suggestions.

[0015] To achieve the above object, according to another aspect of the present invention, an end-to-end driver alcohol detection device is provided.

[0016] The end-to-end driver alcohol detection device according to an embodiment of the present invention includes: an image acquisition unit for acquiring face video data of a driver using an in-vehicle image acquisition module; a data processing unit for preprocessing a plurality of video frames in the face video data; a drunk driving detection unit for sending the preprocessed plurality of video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result indicating whether it is drunk driving; and a display unit for displaying the drunk driving detection result on an in-vehicle display screen.

[0017] Preferably, the end-to-end driver alcohol detection device further includes a model training unit for: acquiring image data of a plurality of drivers covering different genders, different ages, and different races before alcohol intake, labeling the image data before alcohol intake with a non-drunk driving label, and determining the image data labeled with the non-drunk driving label as negative samples; acquiring image data of a plurality of drivers covering different genders, different ages, and different races after alcohol intake, labeling the image data after alcohol intake with a drunk driving label, and determining the image data labeled with the drunk driving label as positive samples; combining the positive samples and the negative samples into training samples, and using the training samples to train the deep learning network model.

[0018] Preferably, the data processing unit is further configured to: acquire the driver's health physiological index data and behavior characteristic data using a driver health monitoring system, and determine whether the health physiological index data and the behavior characteristic data meet preset fatigue driving or sudden illness discrimination conditions; if so, issue a reminder of fatigue driving or sudden illness; the drunk driving detection unit is further configured to: send the preprocessed plurality of video frames, the health physiological index data, and the behavior characteristic data into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result indicating whether it is drunk driving.

[0019] Preferably, the end-to-end driver alcohol detection device further includes an identity calculation unit for: performing identity authentication on the driver before acquiring the face video data of the driver using the in-vehicle image acquisition module to obtain driver identity information; acquiring individual characteristic data corresponding to the driver identity information, using the individual characteristic data to adjust the model parameters of the deep learning network model, and using the adjusted deep learning network model to extract the spatio-temporal feature signals and obtain the drunk driving detection result; after obtaining the drunk driving detection result, associating the drunk driving detection result with the driver identity information.

[0020] Preferably, the end-to-end driver alcohol detection device further includes a specific computing unit for: pre-deploying a plurality of computing nodes, and distributing the preprocessing of the plurality of video frames, the extraction of the spatio-temporal feature signals, and the calculation of the drunk driving detection results to the plurality of computing nodes based on distributed computing technology and edge computing technology to improve the computing real-time performance and response speed; after obtaining the drunk driving detection results, if the drunk driving detection results indicate drunk driving, issuing a voice warning and a reminder to stop driving to the driver through the in-vehicle voice system, triggering the in-vehicle safety system or the autonomous driving assistance system to enhance driving safety, and providing real-time safety suggestions; after obtaining the drunk driving detection results, uploading the drunk driving detection results to the cloud platform and the vehicle owner's intelligent terminal through a wireless network to achieve remote monitoring and management of the driver's drunk driving behavior.

[0021] Preferably, the image acquisition unit is further used for: collecting the face video data by using a plurality of in-vehicle image acquisition modules at different angles; the preprocessing includes image denoising, image enhancement, brightness adjustment, face detection, and image cropping; the display unit is further used for: displaying the drunk driving detection results in the form of an icon or text on the central control display screen or the head-up display device HUD for the driver to view conveniently; the specific computing unit is further used for: configuring, controlling, and remotely managing the driver alcohol detection through a variety of user input methods, and the user input methods include: voice input, touch operation on the in-vehicle display screen, or operation based on the intelligent terminal application.

[0022] Preferably, the in-vehicle image acquisition module is a near-infrared camera or a red-green-blue RGB camera; the deep learning network model is: 3D convolutional neural network 3D-CNN, long short-term memory network LSTM, gated recurrent unit GRU, spatio-temporal graph convolutional network ST-GCN, spatio-temporal convolutional network TCN, convolutional neural network CNN combined with long short-term memory network LSTM, spatio-temporal attention network, or transformer Transformer model; the behavioral feature data includes eye movement data or head tilt data, the individual feature data includes age data, gender data, or driving experience data, the in-vehicle safety system performs automatic deceleration or improves vehicle control stability after being triggered, and the real-time safety suggestions include suggestions to take a break or traffic safety suggestions.

[0023] To achieve the above object, according to another aspect of the present invention, a computer-readable storage medium is provided.

[0024] The computer-readable storage medium of the embodiment of the present invention stores a computer program, and the program is executed by a processor to be used for implementing the end-to-end driver alcohol detection method provided by the present invention.

[0025] To achieve the above object, according to another aspect of the present invention, a vehicle is provided.

[0026] The vehicle in the embodiment of the present invention includes an electronic device, and the electronic device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. Wherein, when the processor executes the program, the end-to-end driver alcohol detection method provided by the present invention can be realized.

[0027] According to the technical solution of the present invention, one embodiment of the above invention has the following advantages or beneficial effects:

[0028] (1) Non-contact detection: There is no need for traditional physical devices such as alcohol testers, avoiding the driver's resistance to the device and inconvenient operation, and improving the convenience of detection and user acceptance.

[0029] (2) End-to-end detection: Directly process the original video data through a deep learning model, without manual feature extraction or complex signal processing steps, and realize automated processing from input to output.

[0030] (3) Low cost and high scalability: It gets rid of the dependence on special hardware such as breath alcohol testers, and only needs an in-vehicle camera and a processing unit to complete the detection, reducing the system cost.

[0031] The further effects of the above non-conventional optional methods will be described in combination with specific embodiments below. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings are used to better understand the present invention and do not constitute an improper limitation to the present invention. Among them:

[0033] Figure 1 is a schematic diagram of the basic steps of the end-to-end driver alcohol detection method of the present invention;

[0034] Figure 2 is a schematic flow diagram of the end-to-end driver alcohol detection method of the present invention;

[0035] Figure 3 is a schematic structural diagram of the end-to-end driver alcohol detection device of the present invention;

[0036] Figure 4 is a schematic structural diagram of the electronic device in the vehicle of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The following describes exemplary embodiments of the present invention with reference to the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0038] Figure 1 is a schematic diagram of the basic steps of the end-to-end driver alcohol detection method of the present invention. Refer to Figure 1 .

[0039] The end-to-end driver alcohol detection method of the present invention includes: Step S1: Use an in-vehicle image acquisition module to collect face video data of the driver; Step S2: Preprocess multiple video frames in the face video data; Step S3: Send the preprocessed multiple video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result on whether it is drunk driving; Step S4: Display the drunk driving detection result on an in-vehicle display screen.

[0040] Through the above steps, an accurate alcohol detection result can be obtained by using a contactless, end-to-end drunk driving detection method without cumbersome intermediate processes such as manual feature extraction. Compared with traditional alcohol detection methods, the present invention gets rid of the requirements for hardware devices such as alcohol testers, and does not require the driver's active cooperation. It has the characteristics of non-contact, high real-time, and high accuracy, and has broad market application prospects. The following will be specifically described. Refer to Figure 2 .

[0041] Step S1: The system uses an in-vehicle image acquisition module to collect face video data of the driver. In this step, the system first uses the in-vehicle image acquisition module to collect face video data of the driver. The in-vehicle image acquisition module can be a near-infrared camera or a red-green-blue (RGB) camera, such as an in-vehicle DMS (driver monitoring system) camera, an OMS (occupant monitoring system) camera, or an RMS (compartment monitoring system) camera, etc. In this way, the face image of the driver in the vehicle can be effectively captured in daytime, night, and complex lighting environments, ensuring that an accurate drunk driving detection result can still be obtained under low-light conditions. In practical applications, multiple in-vehicle image acquisition modules at different angles can be used to continuously collect video of the driver's facial area to obtain face video data, so as to avoid a single camera being blocked or the acquisition angle being limited.

[0042] Step S2: The system preprocesses multiple video frames in the face video data. The above preprocessing may include image denoising, image enhancement, brightness adjustment, face detection, and image cropping. Face detection can use known face detection algorithms such as OpenCV, dlib, or YOLO. Thereafter, the face area in the video frame is located, and the effective part is cropped. The above steps can ensure the accurate extraction of the driver's face area and exclude background interference, thereby improving the accuracy of drunk driving detection.

[0043] Step S3: The system sends the preprocessed multiple video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result of whether it is drunk driving. The above deep learning network model can be the following models: 3D convolutional neural network 3D-CNN, long short-term memory network LSTM, gated recurrent unit GRU, spatio-temporal graph convolutional network ST-GCN, spatio-temporal convolutional network TCN, convolutional neural network CNN combined with long short-term memory network LSTM, spatio-temporal attention network, or transformer Transformer model. The above spatio-temporal feature signals include spatial dimension signals and time dimension signals related to alcohol consumption extracted from the face video data, and may include rPPG (remote Photoplethysmography) signals in practical applications.

[0044] In a specific application, the deep learning network model can be trained according to the following steps: First, obtain image data of multiple drivers of different genders, different ages, and different races before alcohol intake, label the image data before alcohol intake with a non-drunk driving label, and determine the image data labeled with the non-drunk driving label as negative samples; at the same time, obtain image data of multiple drivers of different genders, different ages, and different races after alcohol intake, label the image data after alcohol intake with a drunk driving label, and determine the image data labeled with the drunk driving label as positive samples; thereafter, combine the positive samples and negative samples into training samples and use the training samples to train the deep learning network model. The above model training process can cover driver individuals of different genders, ages, and races, thereby ensuring that the model has wide applicability and high robustness.

[0045] In an exemplary embodiment, before, after, or at the same time as collecting the face video data of the driver, the system can also use a driver health monitoring system to obtain the driver's health physiological index data and behavior characteristic data. The health physiological index data can include: heart rate data, blood pressure data, etc. The behavior characteristic data can include eye movement data or head tilt data. The system can judge whether the driver has a sudden illness through the health physiological index data, and can judge whether the driver is currently fatigued through the behavior characteristic data. After obtaining the driver's health physiological index data and behavior characteristic data, the system can judge whether the health physiological index data and the behavior characteristic data meet the preset discriminant conditions for driving fatigue or sudden illness; if they meet, a reminder of driving fatigue or sudden illness is issued to quickly respond to the above emergencies. The above steps can continuously collect and analyze the driver's state during driving to ensure the driving safety of the driver throughout the journey.

[0046] After obtaining the driver's health physiological index data and behavior characteristic data, the system can also send the preprocessed multiple video frames, health physiological index data, and behavior characteristic data into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result on whether the driver is driving under the influence of alcohol. Inputting the above health physiological index data and behavior characteristic data together with the collected video frames into the model can improve the data comprehensiveness and further improve the accuracy of drunk driving detection.

[0047] In an exemplary embodiment, before using the in-vehicle image acquisition module to collect the face video data of the driver, the system can perform identity authentication on the driver. After the identity authentication is passed, the driver identity information can be obtained and the subsequent process can be executed. When the identity authentication fails, the process ends. After the identity authentication is passed, the system can obtain the individual characteristic data corresponding to the driver identity information. Exemplarily, the individual characteristic data can include age data, gender data, or driving experience data, and can also include any other applicable data. Thereafter, the system can use the individual characteristic data to adjust the model parameters of the deep learning network model, and use the adjusted deep learning network model to extract spatio-temporal feature signals and obtain a drunk driving detection result. The above adjustment can be fine-tuning on the basis of the pre-trained deep learning network model. After obtaining the drunk driving detection result output by the deep learning network model, the system can associate the drunk driving detection result with the driver identity information.

[0048] In practical applications, to accelerate the operation speed, multiple computing nodes can be pre-deployed. The above computing nodes can be arranged locally or in the cloud. Based on distributed computing technology and edge computing technology, the system evenly distributes computing tasks such as the preprocessing of the above multiple video frames, the extraction of spatio-temporal feature signals, and the calculation of drunk driving detection results to the above multiple computing nodes, thereby reducing the computing burden on the in-vehicle unit and improving the computing real-time performance and response speed.

[0049] Step S4: The system displays the drunk driving detection result on the in-vehicle display screen. In this step, the system can display the drunk driving detection result in the form of an icon or text on the central control display screen or the head-up display device HUD for the driver to view. For example, if the drunk driving detection result is no drunk driving, the system can display "Driving normally" on the in-vehicle display screen; if the drunk driving detection result is drunk driving, the system can display "Abnormal drunk driving detection" on the in-vehicle display screen.

[0050] In practical applications, after obtaining the drunk driving detection result, if the drunk driving detection result is drunk driving, the system can issue a voice warning and a reminder to stop driving to the driver through the in-vehicle voice system, and trigger the in-vehicle safety system or the autonomous driving assistance system to enhance driving safety and provide real-time safety suggestions. After the above in-vehicle safety system is triggered, it performs automatic deceleration or improves the vehicle control stability. The above autonomous driving assistance system (such as the advanced driving assistance system ADAS) can take over the driver's control of driving or control the vehicle to gradually pull over to the side of the road, thereby preventing the driver from continuing to drive dangerously. The above real-time safety suggestions can include suggestions for parking and resting or traffic safety suggestions.

[0051] In an exemplary embodiment, after obtaining the drunk driving detection result, the system can upload the drunk driving detection result to the cloud platform and the owner's intelligent terminal through a wireless network to achieve remote monitoring and management of the driver's drunk driving behavior. In addition, the system can configure, control, and remotely manage the driver's alcohol detection through a variety of user input methods. The above user input methods can include: voice input, touch operation on the in-vehicle display screen, or operation based on the intelligent terminal application, thereby providing a flexible user experience and operation convenience.

[0052] In actual scenarios, the system can make adaptive adjustments according to different driving scenarios. Whether on highways, urban roads, or under night driving conditions, the system can accurately detect the alcohol of the driver.

[0053] In the technical solution of the present invention, an end-to-end non-contact drunk driving detection method is adopted. The in-vehicle image acquisition module is used to obtain the face video data of the driver in the vehicle. The original image is directly processed by combining with a deep learning network model, and the alcohol detection result is output without cumbersome intermediate processes such as manual feature extraction. Compared with the traditional alcohol detection method, the present invention gets rid of the requirement for hardware devices such as alcohol testers, and does not require the driver's active cooperation. It has the characteristics of non-contact, high real-time, high accuracy, convenient use, and low maintenance cost, and has broad market application prospects.

[0054] Figure 3 It is a schematic structural diagram of the end-to-end driver alcohol detection device of the present invention. As Figure 3 shown, the end-to-end driver alcohol detection device 200 according to an embodiment of the present invention includes: an image acquisition unit 201, configured to acquire the face video data of the driver by using an in-vehicle image acquisition module; a data processing unit 202, configured to preprocess a plurality of video frames in the face video data; a drunk driving detection unit 203, configured to send the preprocessed plurality of video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals, and obtain a drunk driving detection result of whether it is drunk driving; a display unit 204, configured to display the drunk driving detection result on an in-vehicle display screen.

[0055] In one embodiment, the end-to-end driver alcohol detection device 200 further includes a model training unit, configured to: acquire image data of a plurality of drivers of different genders, different ages, and different races before alcohol intake, label the image data before alcohol intake with a non-drunk driving label, and determine the image data labeled with the non-drunk driving label as negative samples; acquire image data of a plurality of drivers of different genders, different ages, and different races after alcohol intake, label the image data after alcohol intake with a drunk driving label, and determine the image data labeled with the drunk driving label as positive samples; combine the positive samples and the negative samples into training samples, and use the training samples to train the deep learning network model.

[0056] In one embodiment, the data processing unit 202 is further configured to: acquire the healthy physiological index data and behavior characteristic data of the driver by using a driver health monitoring system, and determine whether the healthy physiological index data and the behavior characteristic data meet the preset discrimination conditions for fatigue driving or sudden illness; if they meet, issue a reminder of fatigue driving or sudden illness; the drunk driving detection unit 203 is further configured to: send the preprocessed plurality of video frames, the healthy physiological index data, and the behavior characteristic data into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals, and obtain a drunk driving detection result of whether it is drunk driving.

[0057] In one embodiment, the end-to-end driver alcohol detection device 200 further includes an identity calculation unit, which is configured to: perform identity authentication on the driver before collecting the face video data of the driver by using the in-vehicle image acquisition module, so as to obtain the driver identity information; acquire the individual feature data corresponding to the driver identity information, use the individual feature data to adjust the model parameters of the deep learning network model, use the adjusted deep learning network model to extract the spatio-temporal feature signal and obtain the drunk driving detection result; after obtaining the drunk driving detection result, associate the drunk driving detection result with the driver identity information.

[0058] In one embodiment, the end-to-end driver alcohol detection device 200 further includes a specific calculation unit, which is configured to: pre-deploy a plurality of computing nodes, and allocate the preprocessing of the plurality of video frames, the extraction of the spatio-temporal feature signal, and the calculation of the drunk driving detection result to the plurality of computing nodes based on distributed computing technology and edge computing technology, so as to improve the calculation real-time performance and response speed; after obtaining the drunk driving detection result, if the drunk driving detection result is drunk driving, issue a voice warning and a reminder to stop driving to the driver through the in-vehicle voice system, trigger the in-vehicle safety system or the autonomous driving assistance system to enhance driving safety, and provide real-time safety suggestions; after obtaining the drunk driving detection result, upload the drunk driving detection result to the cloud platform and the vehicle owner intelligent terminal through a wireless network, so as to realize remote monitoring and management of the driver's drunk driving behavior.

[0059] In one embodiment, the image acquisition unit 201 is further configured to: collect the face video data by using a plurality of in-vehicle image acquisition modules at different angles; the preprocessing includes image denoising, image enhancement, brightness adjustment, face detection, and image cropping; the display unit 204 is further configured to: display the drunk driving detection result in the form of an icon or text on the center control display screen or the head-up display device HUD, so as to facilitate the driver to view; the specific calculation unit is further configured to: perform configuration, control, and remote management of the driver alcohol detection through a variety of user input methods, and the user input methods include: voice input, touch operation on the in-vehicle display screen, or operation based on the intelligent terminal application program.

[0060] In one embodiment, the vehicle-mounted image acquisition module is a near-infrared camera or a red-green-blue (RGB) camera; the deep learning network model is: a 3D convolutional neural network (3D-CNN), a long short-term memory network (LSTM), a gated recurrent unit (GRU), a spatio-temporal graph convolutional network (ST-GCN), a spatio-temporal convolutional network (TCN), a convolutional neural network (CNN) combined with a long short-term memory network (LSTM), a spatio-temporal attention network, or a Transformer model; the behavioral feature data includes eye movement data or head tilt data, the individual feature data includes age data, gender data, or driving experience data, the vehicle-mounted safety system performs automatic deceleration or improves vehicle control stability after being triggered, and the real-time safety advice includes a parking and rest advice or a traffic safety advice.

[0061] It should be noted that the end-to-end driver alcohol detection device according to the embodiment of the present invention, as software, can be installed in devices such as computers and mobile terminals.

[0062] In the technical solution of the present invention, an end-to-end non-contact drunk driving detection method is adopted. The vehicle-mounted image acquisition module is used to obtain the face video data of the driver in the vehicle, and the deep learning network model is combined to directly process the original image and output the alcohol detection result, without cumbersome intermediate processes such as manual feature extraction. Compared with traditional alcohol detection methods, the present invention gets rid of the requirement for hardware devices such as alcohol testers, does not require the driver's active cooperation, and has the characteristics of non-contact, high real-time, high accuracy, convenient use, and low maintenance cost, and has broad market application prospects.

[0063] It should be noted that in the technical solution of the present invention, in terms of the collection, analysis, use, transmission, storage, etc. of the user's personal information involved, they all comply with the provisions of relevant laws and regulations, are used for legal and reasonable purposes, are not shared, leaked, or sold outside these legal uses, and are subject to the supervision and management of the regulatory authorities. Necessary measures should be taken for the user's personal information to prevent illegal access to such personal information data, ensure that the personnel with the right to access the personal information data comply with the provisions of relevant laws and regulations, and ensure the security of the user's personal information. Once these user personal information data are no longer needed, the risk should be minimized by restricting or even prohibiting data collection and / or deleting the data. When using, including in some relevant application programs, user privacy is protected by de-identifying the data. For example, when using, specific identifiers (such as date of birth) are removed, the amount or specificity of the stored data is controlled (such as collecting location data at the city level instead of at the specific address level), how the data is stored is controlled, and / or other methods are used to de-identify.

[0064] According to an embodiment of the present invention, the present invention further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium of the embodiment of the present invention, and the program is executed by a processor to implement the end-to-end driver alcohol detection method provided by the present invention.

[0065] Figure 4 It is a schematic structural diagram of an electronic device in a vehicle provided by an embodiment of the present invention. The vehicle includes an electronic device, and the electronic device may include: a memory 301, a processor 302, and a computer program stored on the memory 301 and executable on the processor 302. When the processor 302 executes the program, it implements an end-to-end driver alcohol detection method provided in the above embodiment. Further, the memory 301 is used to store a computer program executable on the processor 302. Further, the memory 301 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory. Further, the electronic device further includes a communication interface 303 for communication between the memory 301 and the processor 302.

[0066] The above structure can execute the end-to-end driver alcohol detection method provided by the embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the end-to-end driver alcohol detection method. For technical details not described in detail in this embodiment, reference may be made to the end-to-end driver alcohol detection method provided by the embodiment of the present invention.

[0067] In the technical solution of the present invention, an end-to-end non-contact drunk driving detection method is adopted. The face video data of the driver in the vehicle is obtained through an in-vehicle image acquisition module, and the deep learning network model is combined to directly process the original image and output the alcohol detection result, without cumbersome intermediate processes such as manual feature extraction. Compared with traditional alcohol detection methods, the present invention gets rid of the requirements for hardware devices such as alcohol testers, does not require the driver's active cooperation, and has the characteristics of non-contact, high real-time, high accuracy, convenient use, and low maintenance cost, and has broad market application prospects.

[0068] The above specific implementation manners do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An end-to-end driver alcohol detection method, characterized in that, Including: Collecting the face video data of the driver by using an in-vehicle image acquisition module; Preprocessing multiple video frames in the face video data; Feeding the preprocessed multiple video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtaining a drunk driving detection result on whether there is drunk driving; Displaying the drunk driving detection result on an in-vehicle display screen.

2. The driver alcohol detection method according to claim 1, characterized in that, The deep learning network model is trained according to the following steps: Obtaining image data of multiple drivers covering different genders, different ages, and different ethnic groups before alcohol intake, labeling the image data before alcohol intake with a non-drunk driving label, and determining the image data labeled with the non-drunk driving label as negative samples; Obtaining image data of multiple drivers covering different genders, different ages, and different ethnic groups after alcohol intake, labeling the image data after alcohol intake with a drunk driving label, and determining the image data labeled with the drunk driving label as positive samples; Combining the positive samples and the negative samples into training samples and using the training samples to train the deep learning network model.

3. The driver alcohol detection method according to claim 2, wherein It also includes: obtaining the healthy physiological index data and behavioral characteristic data of the driver by using a driver health monitoring system, and judging whether the healthy physiological index data and the behavioral characteristic data meet the preset discrimination conditions for fatigue driving or sudden illness; if they meet, sending a reminder of fatigue driving or sudden illness; The step of feeding the preprocessed multiple video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtaining a drunk driving detection result on whether there is drunk driving includes: feeding the preprocessed multiple video frames, the healthy physiological index data, and the behavioral characteristic data into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtaining a drunk driving detection result on whether there is drunk driving.

4. The driver alcohol detection method according to claim 3, wherein, It also includes: Performing identity authentication on the driver before collecting the face video data of the driver by using the in-vehicle image acquisition module to obtain driver identity information; Obtaining individual feature data corresponding to the driver identity information, using the individual feature data to adjust the model parameters of the deep learning network model, and using the adjusted deep learning network model to extract the spatio-temporal feature signals and obtain the drunk driving detection result; After obtaining the drunk driving detection result, associating the drunk driving detection result with the driver identity information.

5. The driver alcohol detection method according to claim 4, characterized in that The driver alcohol detection method also includes: Pre-deploying multiple computing nodes, and distributing the preprocessing of the multiple video frames, the extraction of the spatio-temporal feature signals, and the calculation of the drunk driving detection result to the multiple computing nodes based on distributed computing technology and edge computing technology to improve the calculation real-time performance and response speed; After obtaining the drunk driving detection result, if the drunk driving detection result is drunk driving, sending a voice warning and a reminder to stop driving to the driver through an in-vehicle voice system, triggering an in-vehicle safety system or an autonomous driving assistance system to enhance driving safety, and providing real-time safety suggestions; After obtaining the drunk driving detection result, the drunk driving detection result is uploaded to the cloud platform and the vehicle owner's intelligent terminal through a wireless network to realize remote monitoring and management of the driver's drunk driving behavior.

6. The driver alcohol detection method according to claim 5, characterized in that, The use of the vehicle-mounted image acquisition module to acquire the driver's face video data includes: using multiple vehicle-mounted image acquisition modules at different angles to acquire the face video data; The preprocessing includes image denoising, image enhancement, brightness adjustment, face detection, and image cropping; The display of the drunk driving detection result on the vehicle-mounted display screen includes: displaying the drunk driving detection result in the form of an icon or text on the central control display screen or the head-up display device HUD for the driver to view conveniently; The driver alcohol detection method further includes: configuring, controlling, and remotely managing the driver alcohol detection through multiple user input methods, and the user input methods include: voice input, touch operation on the vehicle-mounted display screen, or operation based on the intelligent terminal application.

7. The driver alcohol detection method according to claim 6, characterized in that, The vehicle-mounted image acquisition module is a near-infrared camera or a red-green-blue RGB camera; The deep learning network model is: 3D convolutional neural network 3D-CNN, long short-term memory network LSTM, gated recurrent unit GRU, spatio-temporal graph convolutional network ST-GCN, spatio-temporal convolutional network TCN, convolutional neural network CNN combined with long short-term memory network LSTM, spatio-temporal attention network, or transformer Transformer model; The behavioral feature data includes eye movement data or head tilt data, the individual feature data includes age data, gender data, or driving experience data, the vehicle-mounted safety system performs automatic deceleration or improves vehicle control stability after being triggered, and the real-time safety advice includes a parking rest advice or a traffic safety advice.

8. An end-to-end driver alcohol detection device, characterized in that, Including: An image acquisition unit for using the vehicle-mounted image acquisition module to acquire the driver's face video data; A data processing unit for preprocessing multiple video frames in the face video data; A drunk driving detection unit for sending the preprocessed multiple video frames into a pre-trained deep learning network model to extract alcohol-related spatio-temporal feature signals and obtain a drunk driving detection result of whether it is drunk driving; A display unit for displaying the drunk driving detection result on the vehicle-mounted display screen.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to be used to implement the end-to-end driver alcohol detection method according to any one of claims 1-7.

10. A vehicle, characterized in that, Including an electronic device, the electronic device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the end-to-end driver alcohol detection method according to any one of claims 1-7 can be implemented.

Images