Multi-mode drunk driving reminding method and system

Abstract

The invention provides a multi-mode drunk driving reminding method and system, and belongs to the technical field of vehicle safe driving, and the method comprises the steps: S1, employing a camera for face recognition to judge a drunk driving probability result X1; s2, judging a drunk driving probability result X2 by using alcohol detection equipment; s3, judging a drunk driving probability result X3 by using the driving track; s4, calculating the drunk driving probability P after weighting the drunk driving probability results X1, X2 and X3; and S5, according to the drunk driving probability P, when drunk driving is judged, a user is reminded. According to the multi-mode drunk driving reminding method provided by the invention, the drunk driving probability is comprehensively judged through the detection results of various modes, weighted logic judgment is performed according to various drunk driving data such as the camera video stream, the alcohol detection equipment and the driving track, and the user is reminded according to the judgment result, so that the drunk driving reminding accuracy is ensured; family members and designated drivers are contacted at the same time, and the possibility of drunk driving is reduced.

Description

Technical Field

[0001] This invention relates to the field of vehicle safe driving technology, and in particular to a multimodal drunk driving warning method and system. Background Technology

[0002] In daily driving, dangerous driving situations often occur due to driver factors, such as distracted driving, fatigued driving, and drunk driving, which can lead to misjudgment, slow reaction, or even loss of control.

[0003] Current vehicle safety alerts are merely routine reminders and offer little benefit to safe driving. Furthermore, they generally rely on cameras to recognize typical driver actions, such as blinking, yawning, or making phone calls, making them prone to misjudgment.

[0004] The current vehicle safety alerts do not include a drunk driving warning function. Summary of the Invention

[0005] The present invention aims to solve at least one of the above-mentioned technical problems existing in the prior art, and proposes a multimodal drunk driving warning method and system.

[0006] In a first aspect, embodiments of the present invention provide a multimodal drunk driving alert method, comprising: step S1, using facial recognition via camera to determine the drunk driving probability result X1; step S2, using an alcohol detection device to determine the drunk driving probability result X2; step S3, using driving trajectory to determine the drunk driving probability result X3; step S4, calculating the drunk driving probability probability P by weighting the drunk driving probability results X1, X2, and X3; and step S5, alerting the user when drunk driving is determined to be caused by drunk driving based on the drunk driving probability P.

[0007] Preferably, step S1, using camera facial recognition to determine the probability result X1 of drunk driving, includes: step S101, capturing the driver's facial image or video frame through camera facial recognition; step S102, inputting the driver's facial image or video frame into the trained AI model; step S103, the AI ​​model outputs the probability result X1 of drunk driving, where X1 is 1 and the driver's drunk driving status is determined to be drunk driving, and X1 is 0 and the driver's drunk driving status is determined to be sober.

[0008] Preferably, the alcohol detection device includes multiple alcohol detection sensors respectively installed at positions B1 and B2 on the left and right sides of the driver's headrest and at position B3 in the center of the steering wheel. Step S2, using the alcohol detection device to determine the probability result X2 of drunk driving includes: Step S201, acquiring data measured at the three positions B1, B2, and B3; Step S202, judging based on the data measured at the three positions B1, B2, and B3 respectively, with the judgment result represented by 1 when the alcohol content is greater than or equal to a preset content threshold, and the judgment result represented by 0 when the alcohol content is less than the preset content threshold; Step S203, then combining the judgment results of the three positions and taking the average value B = (judgment result at position B1 + judgment result at position B2 + judgment result at position B3) / 3; Step S204, when the average value B is greater than or equal to 2 / 3, the probability result X2 of drunk driving is 1, and the driver's drunk driving status is judged as drunk driving; when the average value B is less than or equal to 1 / 3, the probability result X2 of drunk driving is 0, and the driver's drunk driving status is judged as sober.

[0009] Preferably, step S3, using the driving trajectory to determine the probability of drunk driving X3, includes: step S301, setting factors based on the driving trajectory as follows: whether it is near an entertainment venue C1, whether it is within the time range C2, whether it is serpentine driving C3, whether it is lane departure C4, whether it is rapid acceleration and deceleration C5, and whether it is speeding C6. C1, C2, C3, C4, C5, and C6 are 1 when they are, and 0 when they are not; step S302, then combining the results of the 6 factors to take the average C = (C1 + C2 + C3 + C4 + C5 + C6) / 6; step S303, when the average C is greater than or equal to 0.5, the probability of drunk driving X3 is 1, and the driver's drunk driving status is determined to be drunk driving; when the average C is less than 0.5, the probability of drunk driving X3 is 0, and the driver's drunk driving status is determined to be sober.

[0010] Preferably, step S4, calculating the drunk driving probability P by weighting the drunk driving probability results X1, X2, and X3, includes setting the drunk driving probability P = X1*40% + X2*40% + X3*20%; step S5, when drunk driving is determined to be drunk driving based on the drunk driving probability P, reminding the user includes: when the drunk driving probability P is greater than or equal to 0.4, the driver's drunk driving status is determined to be drunk driving; when the drunk driving probability P is less than 0.4, the driver's drunk driving status is determined to be sober; or, when the drunk driving probability P is greater than or equal to 0.2, the driver's drunk driving status is determined to be drunk driving; when the drunk driving probability P is less than 0.2, the driver's drunk driving status is determined to be sober; or, when the drunk driving probability P is equal to 1, the driver's drunk driving status is determined to be drunk driving; when the drunk driving probability P is less than 1, the driver's drunk driving status is determined to be sober.

[0011] Secondly, embodiments of the present invention provide a multimodal drunk driving warning system, comprising: in-vehicle environmental components, a driving APP, a smart cockpit controller, and a human-machine interaction medium; the in-vehicle environmental components include: an in-vehicle camera, an in-vehicle alcohol detection sensor, and GPS, used to output driver facial data, alcohol concentration data, and vehicle location data to the smart cockpit controller; the driving APP is used to output driving trajectory data to the smart cockpit controller; the smart cockpit controller includes: a TBOX, a smart reminder module, and an AI recognition module, used to perform logical judgment processing and transmit the reminder content to the human-machine interaction medium in the form of audio data, image data, and / or vehicle bus signals; the human-machine interaction medium includes: a display screen, a speaker, and a steering wheel, serving as a window for interaction between the vehicle and the user, providing visual reminders, voice reminders, and / or steering wheel vibration reminders to the user.

[0012] Preferably, the intelligent cockpit controller captures the driver's facial image or video frame through in-vehicle camera facial recognition, inputs the driver's facial image or video frame into the AI ​​recognition module, and the AI ​​recognition module outputs the drunk driving probability result X1. When the drunk driving probability result X1 is 1, the driver's drunk driving status is judged as drunk driving, and when the drunk driving probability result X2 is 0, the driver's drunk driving status is judged as sober.

[0013] There are three in-vehicle alcohol detection sensors, installed on the left and right sides of the driver's headrest at positions B1 and B2, and in the center of the steering wheel at position B3. The intelligent cockpit controller judges the alcohol concentration data measured by the three in-vehicle alcohol detection sensors. When the alcohol content is greater than or equal to a preset threshold, the judgment result is represented by 1; when the alcohol content is less than the preset threshold, the judgment result is represented by 0. The intelligent cockpit controller calculates the average value B of the three judgment results. When the average value B is greater than or equal to 2 / 3, the probability result X2 of drunk driving is 1, and the driver's drunk driving status is judged as drunk driving; when the average value B is less than or equal to 1 / 3, the probability result X2 of drunk driving is 0, and the driver's drunk driving status is judged as sober.

[0014] The intelligent cockpit controller determines the probability of drunk driving (X3) based on the driving trajectory input by the driving APP. The determination is made by setting the following criteria: whether the driving trajectory is near an entertainment venue (C1), whether it falls within the specified time frame (C2), whether it involves serpentine driving (C3), whether it deviates from its lane (C4), whether it involves sudden acceleration and deceleration (C5), and whether it involves speeding (C6). C1, C2, C3, C4, C5, and C6 are set to 1 if they are true and 0 if they are false. The average value C is calculated as (C1 + C2 + C3 + C4 + C5 + C6) / 6. When the average value C is greater than or equal to 0.5, the probability of drunk driving (X3) is 1, indicating the driver is driving under the influence. When the average value C is less than 0.5, the probability of drunk driving (X3) is 0, indicating the driver is sober.

[0015] The intelligent cockpit controller calculates the drunk driving probability P by weighting the drunk driving probability results X1, X2, and X3. Based on the drunk driving probability P, if drunk driving is determined, the intelligent reminder module is notified to remind the user using the human-computer interaction medium.

[0016] Preferably, the intelligent cockpit controller calculates the probability of drunk driving as: P = X1 * 40% + X2 * 40% + X3 * 20%; the intelligent cockpit controller determines drunk driving based on the probability P as follows: when the probability P is greater than or equal to 0.4, the driver's drunk driving status is determined to be drunk driving; when the probability P is less than 0.4, the driver's drunk driving status is determined to be sober; or, when the probability P is greater than or equal to 0.2, the driver's drunk driving status is determined to be drunk driving; when the probability P is less than 0.2, the driver's drunk driving status is determined to be sober; or, when the probability P is equal to 1, the driver's drunk driving status is determined to be drunk driving; when the probability P is less than 1, the driver's drunk driving status is determined to be sober.

[0017] Thirdly, embodiments of the present invention provide an electronic device comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described in any embodiment of the first aspect.

[0018] Fourthly, embodiments of the present invention provide a computer-readable medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any embodiment of the first aspect.

[0019] The multimodal drunk driving alert method provided by this invention comprehensively judges the probability of drunk driving by combining the detection results of multiple methods. It performs weighted logical judgment based on multiple drunk driving data such as camera video stream, alcohol detection equipment and driving trajectory, and reminds users according to the judgment results, ensuring the accuracy of drunk driving alerts, and simultaneously contacting family members and designated drivers to reduce the possibility of drunk driving. Attached Figure Description

[0020] Figure 1 This is a step-by-step flowchart of a multimodal drunk driving warning method provided by an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the overall process of a multimodal drunk driving warning method provided in an embodiment of the present invention;

[0022] Figure 3 This is a structural block diagram of an application scenario of the method provided in the embodiments of the present invention;

[0023] Figure 4 This is a flowchart illustrating an optional specific implementation method of step S1 in an embodiment of the present invention;

[0024] Figure 5 This is a flowchart illustrating an optional specific implementation method of step S2 in an embodiment of the present invention;

[0025] Figure 6 This is a flowchart illustrating an optional specific implementation method of step S3 in an embodiment of the present invention;

[0026] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0029] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0031] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0032] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0033] In related technologies, the current safety reminder solutions for vehicles use cameras to recognize typical actions to provide reminders. However, this solution does not cover all scenarios and cannot provide reminders when the scenario is not preset, which can easily lead to misjudgments.

[0034] To address at least one of the technical problems existing in the aforementioned related technologies, the present invention provides a multimodal drunk driving warning method and system.

[0035] Figure 1 This is a step-by-step flowchart of a multimodal drunk driving warning method provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall process of a multimodal drunk driving warning method provided by an embodiment of the present invention; as shown below. Figure 1 , Figure 2 As shown, the method includes: step S1, using facial recognition via camera to determine the probability of drunk driving X1; step S2, using an alcohol detection device to determine the probability of drunk driving X2; step S3, using driving trajectory to determine the probability of drunk driving X3; step S4, calculating the probability of drunk driving P by weighting the probability results X1, X2, and X3; step S5, based on the probability of drunk driving P, reminding the user when drunk driving is determined.

[0036] In some embodiments, such as Figure 2 , Figure 4 As shown, step S1, using camera facial recognition to determine the probability of drunk driving X1, includes:

[0037] Step S101: Capture the driver's facial image or video frame through camera facial recognition;

[0038] Step S102: Input the driver's facial image or video frame into the trained AI model;

[0039] Step S103: The AI ​​model outputs the probability result of drunk driving X1. When X1 is 1, the driver's drunk driving status is judged as drunk driving. When X1 is 0, the driver's drunk driving status is judged as sober.

[0040] This invention utilizes machine learning algorithms to analyze facial features (such as facial features, gaze direction, and head position) and possible facial movements to determine whether a driver is intoxicated. This technology relies on big data and deep learning models (an AI model) to identify intoxicated drivers by comparing the differences in facial features between sober and intoxicated states.

[0041] Therefore, it is necessary to train an AI model that accurately identifies drunk driving states using a large amount of video and image data. Training the AI ​​model includes:

[0042] Collect facial data of conscious individuals: Collect a large number of facial images and videos of drivers in a conscious state. This data should cover different ages, genders, facial features, and lighting conditions to increase the diversity and generalization ability of the dataset.

[0043] Collect facial data of drivers under the influence of alcohol: Collect facial images and videos of drivers under the influence of alcohol;

[0044] Next, data annotation is performed. The collected data is labeled to clearly indicate whether the driver is sober or intoxicated in each image or video clip. The annotation process should be as accurate as possible to reduce errors in subsequent model training.

[0045] The AI ​​model is trained to identify drunk drivers by comparing the differences in facial features between sober and drunk driving states.

[0046] In some embodiments, the alcohol detection device includes multiple alcohol detection sensors respectively installed at positions B1 and B2 on the left and right sides of the driver's headrest and at position B3 in the center of the steering wheel, ensuring that they are as close as possible to the driver. Figure 3 As shown;

[0047] like Figure 2 , Figure 5 As shown, step S2, determining the probability of drunk driving X2 using an alcohol testing device, includes:

[0048] Step S201: Obtain data measured at three locations B1, B2, and B3;

[0049] Step S202: Based on the data measured at three locations B1, B2, and B3, a judgment is made. If the alcohol content is greater than or equal to the preset threshold, it is considered drunk driving, and the judgment result is represented by 1 (the driver's drunk driving status is judged as drunk driving). If the alcohol content is less than the preset threshold, the judgment result is represented by 0 (the driver's drunk driving status is judged as sober). The preset threshold can be the national standard of 20 mg / 100 mL, or other values ​​can be used.

[0050] Step S203: Combine the judgment results of the three positions and take the average value B = (judgment result of position B1 + judgment result of position B2 + judgment result of position B3) / 3;

[0051] In step S204, when the average value B is greater than or equal to 2 / 3, the probability result X2 of drunk driving is 1, and the driver's drunk driving status is judged as drunk driving; when the average value B is less than or equal to 1 / 3, the probability result X2 of drunk driving is 0, and the driver's drunk driving status is judged as sober.

[0052] The output of the probability of drunk driving multiplied by 2 is the result of judging the driver's drunk driving status as either sober or drunk, represented by 0 and 1 respectively, as follows:

[0053]

[0054] In some embodiments, step S3 involves using the driving trajectory to determine the probability of drunk driving, X3. The driving trajectory itself cannot directly determine whether the driver has been drinking, as there is no direct causal relationship between the destination and whether the driver has consumed alcohol. However, in some situations, the driving destination may be indirectly associated with the driver's drinking behavior, but this requires comprehensive analysis of other factors. For example... Figure 2 , Figure 6 As shown, step S3, using the driving trajectory to determine the probability of drunk driving X3, includes: Step S301, setting factors based on the driving trajectory: whether it is near an entertainment venue C1, whether it is within the time range C2, trajectory stability (whether it is serpentine driving C3, whether it is lane departure C4), speed and acceleration analysis (whether it is rapid acceleration and deceleration C5, whether it is speeding C6). C1, C2, C3, C4, C5, and C6 are 1 when they are, and 0 when they are not. Step S302, then combining the results of the 6 factors to take the average C = (C1 + C2 + C3 + C4 + C5 + C6) / 6. Step S303, when the average C is greater than or equal to 0.5, the probability of drunk driving X3 is 1, and the driver's drunk driving status is determined to be drunk driving; when the average C is less than 0.5, the probability of drunk driving X3 is 0, and the driver's drunk driving status is determined to be sober.

[0055] The output of the drunk driving probability result X3 is the result of judging the driver's drunk driving status as either sober or drunk, represented by 0 and 1 respectively, as follows:

[0056]

[0057]

[0058]

[0059]

[0060] In some embodiments, step S4, which calculates the drunk driving probability P by weighting the drunk driving probability results X1, X2, and X3, includes setting the drunk driving probability P = X1*40% + X2*40% + X3*20%.

[0061] In some embodiments, step S5, when determining drunk driving based on the probability P, includes reminding the user as follows: when the probability P is greater than or equal to 0.4, the driver's drunk driving status is determined to be drunk driving; when the probability P is less than 0.4, the driver's drunk driving status is determined to be sober. When the probability P is greater than or equal to 0.4, it means that at least one of the drunk driving probability results X1 or X2 is 1, indicating that at least one of the two detection methods X1 or X2 considers the driver to be drunk driving, and the accuracy of determining the driver's drunk driving status is relatively high.

[0062] In some embodiments, when the probability of drunk driving P is greater than or equal to 0.2, the driver's drunk driving status is determined to be drunk driving; when the probability of drunk driving P is less than 0.2, the driver's drunk driving status is determined to be sober. When the probability of drunk driving P is greater than or equal to 0.2, it means that at least one of the drunk driving probability results X1, X2, or X3 is 1, indicating that at least one of the three detection methods considers the driver to be drunk driving. In this case, the accuracy of the drunk driving status determination is lower, but the safety for the driver is higher, that is, the driver is alerted when there is a risk of drunk driving.

[0063] In some embodiments, when the probability P of drunk driving is equal to 1, the driver's drunk driving status is determined to be drunk driving; when the probability P of drunk driving is less than 1, the driver's drunk driving status is determined to be sober. When the probability P of drunk driving is equal to 1, it means that the drunk driving probability results X1, X2, and X3 are all 1, indicating that all three detection methods consider the driver to be drunk driving, and the driver's drunk driving status determination result is very accurate in this case.

[0064] In some embodiments, this invention proposes a multimodal drunk driving warning system, which comprises four main parts: in-vehicle environmental components, a driving APP (for acquiring driving trajectory), a smart cockpit controller, and a human-machine interaction medium. The in-vehicle environmental components mainly include an in-vehicle camera, an in-vehicle alcohol detection sensor, GPS, etc., primarily used to output driver facial data, alcohol concentration data, vehicle location, and other information to the smart cockpit controller. The driving APP is mainly used to output the driving trajectory to the smart cockpit controller. The smart cockpit controller mainly includes a TBOX, a smart warning module, and an AI recognition module, used to receive vehicle data, driving trajectory data, etc., perform logical judgment processing, and transmit the warning content to the human-machine interaction medium in the form of audio data, image data, and / or vehicle bus signals. The human-machine interaction medium mainly includes a display screen, speakers, steering wheel, etc., serving as a window for interaction between the vehicle and the user, providing visual warnings, voice warnings, and / or steering wheel vibration warnings to the user.

[0065] In some embodiments, the intelligent cockpit controller captures the driver's facial image or video frame through in-vehicle camera facial recognition. The driver's facial image or video frame is input into the AI ​​recognition module, which outputs a drunk driving probability result X1. When the drunk driving probability result X1 is 1, the driver's drunk driving status is determined to be drunk driving; when the drunk driving probability result X2 is 0, the driver's drunk driving status is determined to be sober. There are three in-vehicle alcohol detection sensors, installed on the left and right sides of the driver's headrest (B1 and B2) and in the center of the steering wheel (B3). The intelligent cockpit controller judges the alcohol concentration data measured by the three in-vehicle alcohol detection sensors. A result of 1 represents a drunk driving status when the alcohol content is greater than or equal to a preset threshold, and a result of 0 represents a sober driving status when the alcohol content is less than the preset threshold. The intelligent cockpit controller calculates the average value B of the three judgment results. When the average value B is greater than or equal to 2 / 3, the probability result X2 of drunk driving is 1, and the driver's drunk driving status is judged as drunk driving. When the average value B is less than or equal to 1 / 3, the probability result X2 of drunk driving is 0, and the driver's drunk driving status is judged as sober. The intelligent cockpit controller determines the probability of drunk driving (X3) based on the driving trajectory input by the driving app. The parameters are defined as follows: whether the driving trajectory is near an entertainment venue (C1), whether it falls within the specified time frame (C2), whether it involves serpentine driving (C3), whether it deviates from its lane (C4), whether it involves sudden acceleration or deceleration (C5), and whether it involves speeding (C6). C1, C2, C3, C4, C5, and C6 are set to 1 if they are true, and 0 if they are not. The average value C is calculated as (C1 + C2 + C3 + C4 + C5 + C6) / 6. When the average value C is greater than or equal to 0.5, the probability of drunk driving (X3) is 1, and the driver is judged to be driving under the influence (DUI). When the average value C is less than 0.5, the probability of drunk driving (X3) is 0, and the driver is judged to be sober. The intelligent cockpit controller calculates the probability of drunk driving (P) by weighting the probabilities X1, X2, and X3. If drunk driving is determined based on probability P, the intelligent reminder module is notified to remind the user using the human-machine interface.

[0066] In some embodiments, the intelligent cockpit controller calculates the probability of drunk driving as: P = X1 * 40% + X2 * 40% + X3 * 20%;

[0067] The intelligent cockpit controller determines whether a person is driving under the influence of alcohol based on the probability P of drunk driving, including:

[0068] When the probability P of drunk driving is greater than or equal to 0.4, the driver's drunk driving status is judged as drunk driving; when the probability P of drunk driving is less than 0.4, the driver's drunk driving status is judged as sober.

[0069] Alternatively, if the probability of drunk driving P is greater than or equal to 0.2, the driver's drunk driving status is determined to be drunk driving; if the probability of drunk driving P is less than 0.2, the driver's drunk driving status is determined to be sober.

[0070] Alternatively, if the probability P of drunk driving is equal to 1, the driver's drunk driving status is determined to be drunk driving; if the probability P of drunk driving is less than 1, the driver's drunk driving status is determined to be sober.

[0071] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 7 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the drunk driving warning methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0072] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0073] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0074] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0075] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps of any of the drunk driving warning methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0076] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0077] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0078] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0079] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0080] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0081] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0082] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0083] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0084] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0085] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A multimodal method for alerting drivers to drunk driving, characterized in that, include: Step S1: Use the camera's facial recognition to determine the probability of drunk driving, result X1. Step S2: Use an alcohol testing device to determine the probability of drunk driving X2; Step S3: Use the driving trajectory to determine the probability of drunk driving (X3). Step S4: Calculate the drunk driving probability P by weighting the drunk driving probability results X1, X2, and X3. Step S5: Based on the probability P of drunk driving, if drunk driving is determined, a reminder is given to the user.

2. The method according to claim 1, wherein, Step S1, using camera facial recognition to determine the probability of drunk driving, result X1 includes: Step S101: Capture the driver's facial image or video frame through camera facial recognition; Step S102: Input the driver's facial image or video frame into the trained AI model; Step S103: The AI ​​model outputs the probability result of drunk driving X1. When X1 is 1, the driver's drunk driving status is judged as drunk driving. When X1 is 0, the driver's drunk driving status is judged as sober.

3. The method according to claim 1, wherein, The alcohol detection device includes: multiple alcohol detection sensors installed at positions B1 and B2 on the left and right sides of the driver's headrest and at position B3 in the center of the steering wheel; Step S2, using an alcohol testing device to determine the probability of drunk driving X2 includes: Step S201: Obtain data measured at three locations B1, B2, and B3; Step S202: Based on the data measured at three locations B1, B2, and B3, make judgments respectively. When the alcohol content is greater than or equal to the preset content threshold, the judgment result is represented by 1, and when the alcohol content is less than the preset content threshold, the judgment result is represented by 0. Step S203, then combine the judgment results of the three positions and take the average value B = (judgment result of position B1 + judgment result of position B2 + judgment result of position B3) / 3; In step S204, when the average value B is greater than or equal to 2 / 3, the probability result X2 of drunk driving is 1, and the driver's drunk driving status is judged as drunk driving; when the average value B is less than or equal to 1 / 3, the probability result X2 of drunk driving is 0, and the driver's drunk driving status is judged as sober.

4. The method according to claim 1, wherein, Step S3, using the driving trajectory to determine the probability of drunk driving, the result X3 includes: Step S301: The driving trajectory is set according to the following factors: whether it is near an entertainment venue C1, whether it is within the time range C2, whether it is serpentine driving C3, whether it is lane departure C4, whether it is rapid acceleration and deceleration C5, and whether it is speeding C6. C1, C2, C3, C4, C5, and C6 are set to 1 when they are, and 0 when they are not. Step S302, then combine the results of the 6 factors and take the average value C = (C1 + C2 + C3 + C4 + C5 + C6) / 6; In step S303, when the average value C is greater than or equal to 0.5, the probability result X3 of drunk driving is 1, and the driver's drunk driving status is judged as drunk driving; when the average value B is less than 0.5, the probability result X3 of drunk driving is 0, and the driver's drunk driving status is judged as sober.

5. The method according to claim 1, wherein, Step S4 involves calculating the drunk driving probability P by weighting the drunk driving probability results X1, X2, and X3, including setting the drunk driving probability P = X1 * 40% + X2 * 40% + X3 * 20%; Step S5, based on the probability P of drunk driving, if drunk driving is determined, a reminder is given to the user, including: When the probability P of drunk driving is greater than or equal to 0.4, the driver's drunk driving status is judged as drunk driving; when the probability P of drunk driving is less than 0.4, the driver's drunk driving status is judged as sober. Alternatively, if the probability of drunk driving P is greater than or equal to 0.2, the driver's drunk driving status is determined to be drunk driving; if the probability of drunk driving P is less than 0.2, the driver's drunk driving status is determined to be sober. Alternatively, if the probability P of drunk driving is equal to 1, the driver's drunk driving status is determined to be drunk driving; if the probability P of drunk driving is less than 1, the driver's drunk driving status is determined to be sober.

6. A multimodal drunk driving warning system, characterized in that, include: In-vehicle environmental components, driving apps, smart cockpit controllers, and human-machine interaction media; The in-vehicle environment components include: an in-vehicle camera, an in-vehicle alcohol detection sensor, and GPS, which are used to output driver facial data, alcohol concentration data, and vehicle location data to the intelligent cockpit controller. The driving app is used to output driving trajectory data to the intelligent cockpit controller; The intelligent cockpit controller includes: TBOX, intelligent reminder module, and AI recognition module, which are used to perform logical judgment processing and transmit the reminder content to the human-machine interaction medium in the form of audio data, image data and / or vehicle bus signals; The human-machine interaction medium includes: a display screen, a speaker, and a steering wheel, which serve as a window for interaction between the vehicle and the user, providing visual reminders, voice reminders, and / or steering wheel vibration reminders to the user.

7. The system according to claim 6, wherein, The intelligent cockpit controller captures the driver's facial image or video frame through the in-vehicle camera's facial recognition, inputs the driver's facial image or video frame into the AI ​​recognition module, and the AI ​​recognition module outputs the drunk driving probability result X1. When the drunk driving probability result X1 is 1, the driver's drunk driving status is judged as drunk driving; when the drunk driving probability result X2 is 0, the driver's drunk driving status is judged as sober. There are three in-vehicle alcohol detection sensors, installed on the left and right sides of the driver's headrest at positions B1 and B2, and in the center of the steering wheel at position B3. The intelligent cockpit controller judges the alcohol concentration data measured by the three in-vehicle alcohol detection sensors. When the alcohol content is greater than or equal to a preset threshold, the judgment result is represented by 1; when the alcohol content is less than the preset threshold, the judgment result is represented by 0. The intelligent cockpit controller calculates the average value B of the three judgment results. When the average value B is greater than or equal to 2 / 3, the probability result X2 of drunk driving is 1, and the driver's drunk driving status is judged as drunk driving; when the average value B is less than or equal to 1 / 3, the probability result X2 of drunk driving is 0, and the driver's drunk driving status is judged as sober. The intelligent cockpit controller determines the probability of drunk driving (X3) based on the driving trajectory input by the driving APP. The determination is made by setting the following criteria: whether the driving trajectory is near an entertainment venue (C1), whether it falls within the specified time frame (C2), whether it involves serpentine driving (C3), whether it deviates from its lane (C4), whether it involves sudden acceleration and deceleration (C5), and whether it involves speeding (C6). C1, C2, C3, C4, C5, and C6 are set to 1 if they are true and 0 if they are false. The average value C is calculated as (C1 + C2 + C3 + C4 + C5 + C6) / 6. When the average value C is greater than or equal to 0.5, the probability of drunk driving (X3) is 1, indicating the driver is driving under the influence. When the average value C is less than 0.5, the probability of drunk driving (X3) is 0, indicating the driver is sober. The intelligent cockpit controller calculates the drunk driving probability P by weighting the drunk driving probability results X1, X2, and X3. Based on the drunk driving probability P, if drunk driving is determined, the intelligent reminder module is notified to remind the user using the human-computer interaction medium.

8. The system according to claim 7, wherein, The intelligent cockpit controller calculates the probability of drunk driving as: P = X1*40% + X2*40% + X3*20%; The intelligent cockpit controller determines whether a person is driving under the influence of alcohol based on the probability P of drunk driving, including: When the probability P of drunk driving is greater than or equal to 0.4, the driver's drunk driving status is judged as drunk driving; when the probability P of drunk driving is less than 0.4, the driver's drunk driving status is judged as sober. Alternatively, if the probability of drunk driving P is greater than or equal to 0.2, the driver's drunk driving status is determined to be drunk driving; if the probability of drunk driving P is less than 0.2, the driver's drunk driving status is determined to be sober. Alternatively, if the probability P of drunk driving is equal to 1, the driver's drunk driving status is determined to be drunk driving; if the probability P of drunk driving is less than 1, the driver's drunk driving status is determined to be sober.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 5.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images