Driving safety monitoring method and device based on multiple sensors and AI judgment, equipment and storage medium

Abstract

The invention discloses a driving safety monitoring method, device and equipment based on multiple sensors and AI judgment and a storage medium, and relates to the technical field of driving safety, and the method comprises the steps: collecting hardware information which comprises alcohol concentration and driver physiological information; processing the hardware information through a preset algorithm to obtain a processing result; when the processing result is that the state is abnormal, analyzing the sober degree of the driver through voice interaction to obtain an analysis result; and according to the processing result and / or the analysis result, obtaining a danger level evaluation result and taking corresponding processing measures to realize driving safety monitoring. According to the invention, the accuracy and real-time performance of driving safety monitoring can be improved, and the privacy and safety protection of user data are enhanced.

Description

Technical Field

[0001] The present application relates to the field of driving safety technology, and in particular to a driving safety monitoring method, device, equipment and storage medium based on multiple sensors and AI judgment. Background Art

[0002] With the development of economy, the number of cars has increased, and the traffic accidents caused by them have brought great loss of life and property to the world. Therefore, driving safety is becoming more and more important. Drunk driving and fatigue driving are important causes of traffic accidents. Every year, there are tens of thousands of traffic accidents caused by drunk driving and fatigue driving, which seriously threaten people's lives and property safety. Many current on-board drunk driving and fatigue driving detection systems often have problems such as low data fusion accuracy, limited real-time performance, single detection system, privacy data leakage and information security risks, which affect the efficiency of driving safety monitoring. Summary of the invention

[0003] The main purpose of this application is to provide a driving safety monitoring method, device, equipment and storage medium based on multi-sensor and AI judgment, aiming to solve the technical problem of how to improve the accuracy and real-time performance of driving safety monitoring and enhance user data privacy and security protection.

[0004] To achieve the above objectives, the present application proposes a driving safety monitoring method based on multi-sensor and AI judgment, the method comprising:

[0005] Collecting hardware information, wherein the hardware information includes alcohol concentration and driver physiological information;

[0006] Processing the hardware information by a preset algorithm to obtain a processing result;

[0007] When the processing result is an abnormal state, analyzing the driver's sobriety level through voice interaction to obtain an analysis result;

[0008] According to the processing result and / or the analysis result, a danger level assessment result is obtained and corresponding processing measures are taken to achieve driving safety monitoring.

[0009] In one embodiment, the step of processing the hardware information by a preset algorithm to obtain a processing result includes:

[0010] Acquiring alcohol concentration and driver physiological information from the hardware information;

[0011] Based on a preset algorithm, the alcohol concentration is compared with an alcohol concentration threshold to obtain a first comparison result;

[0012] Based on the preset algorithm, the driver's physiological information is compared with normal physiological information to obtain a second comparison result;

[0013] When the first comparison result is that the driver has not drunk alcohol and the second comparison result is that the driver's physiological state is normal, determining that the processing result is a normal state;

[0014] When the first comparison result is driving under the influence of alcohol or drunk driving, or the second comparison result is that the driver's physiological state is abnormal, the processing result is determined to be an abnormal state.

[0015] In one embodiment, when the processing result is an abnormal state, the step of analyzing the driver's sobriety through voice interaction to obtain the analysis result includes:

[0016] When the processing result is an abnormal state, initiating a question-and-answer interaction through voice to obtain question-and-answer data;

[0017] Analyze the driver's reaction time, language clarity, and logical judgment ability based on the question and answer data;

[0018] When the reaction time, language clarity and logical judgment ability meet the preset standards, the analysis result is determined to be that the driver is conscious;

[0019] When the reaction time, language clarity and logical judgment ability do not meet the preset standards, it is determined that the analysis result is that the driver is not conscious.

[0020] In one embodiment, the step of obtaining a risk level assessment result and taking corresponding treatment measures according to the processing result and / or the analysis result includes:

[0021] According to the processing result and / or the analysis result, evaluating the danger level of the driver's state;

[0022] When the processing result is that the state is normal, determining the danger level is no danger;

[0023] When the processing result is that the state is abnormal, obtaining a first comparison result and a second comparison result;

[0024] When the analysis result shows that the driver is conscious, the first comparison result shows that the driver is driving under the influence of alcohol, and the second comparison result shows that the driver is in a normal physiological state, the danger level is determined to be a slight danger, and a danger warning is issued;

[0025] When the analysis result is that the driver is unconscious or the first comparison result is drunk driving or the second comparison result is that the driver is in an abnormal physiological state, the danger level is determined to be severe danger, and the vehicle is controlled to lock and enter emergency contact mode.

[0026] In one embodiment, when the analysis result is that the driver is unconscious, determining the danger level as severe danger, and controlling the vehicle to lock and enter the emergency contact mode, the steps include:

[0027] When the vehicle enters the emergency contact mode, automatically select to contact an emergency contact and / or a designated driver platform;

[0028] When choosing to contact the designated driving platform, record the audio and video data of the designated driving process and monitor the safety status in the car;

[0029] When the safety condition in the vehicle is abnormal, an alarm is automatically sounded and the audio and video data is transmitted to the backend server.

[0030] In one embodiment, the step of collecting hardware information through sensors, wherein the hardware information includes alcohol concentration and driver physiological information, includes:

[0031] The driver's facial expression, eye state and head posture are collected through the camera;

[0032] collecting the alcohol concentration in the driver's exhaled gas by means of an alcohol concentration sensor;

[0033] Collecting the driver's heart rate, pulse and blood pressure through a biometric information sensor;

[0034] The facial expression, the eye state, the head posture, the heart rate, the pulse and the blood pressure are used as the driver's physiological information.

[0035] In one embodiment, the step of obtaining a danger level assessment result and taking corresponding treatment measures according to the processing result and / or the analysis result to realize intelligent monitoring of driving safety includes:

[0036] When the processing result is an abnormal state, obtaining question and answer data of the voice interaction;

[0037] Encrypting the hardware information and / or the question-and-answer data to obtain encrypted information, and storing the encrypted information locally;

[0038] In an emergency, the encrypted information is uploaded to the backend server.

[0039] In addition, to achieve the above purpose, the present application also proposes a driving safety monitoring device based on multiple sensors and AI judgment, the device comprising:

[0040] An information collection module, used to collect hardware information through sensors, wherein the hardware information includes alcohol concentration and driver physiological information;

[0041] A data processing module, used to process the hardware information through a preset algorithm to obtain a processing result;

[0042] A state analysis module, used to analyze the driver's sobriety through voice interaction to obtain an analysis result when the processing result is an abnormal state;

[0043] The evaluation control module is used to obtain a danger level evaluation result and take corresponding processing measures according to the processing result and / or the analysis result to achieve driving safety monitoring.

[0044] In addition, to achieve the above-mentioned objectives, the present application also proposes a driving safety monitoring device based on multiple sensors and AI judgment, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the driving safety monitoring method based on multiple sensors and AI judgment as described above.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the driving safety monitoring method based on multi-sensors and AI judgment as described above are implemented.

[0046] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the driving safety monitoring method based on multiple sensors and AI judgment as described above.

[0047] One or more technical solutions proposed in this application have at least the following technical effects:

[0048] By adopting multiple sensors and AI voice interactive judgment, driving safety is monitored in real time and corresponding measures are taken in risky situations, solving the technical problems of low data fusion accuracy, limited real-time performance, single detection system, privacy data leakage and information security risks in the driving monitoring system. Compared with the existing technology, it has achieved the improvement of the accuracy and real-time performance of driving safety monitoring and enhanced the privacy and security protection of user data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0051] Figure 1 A flowchart diagram of a driving safety monitoring method based on multiple sensors and AI judgment according to the present application is provided in Example 1;

[0052] Figure 2 A flowchart diagram of the second embodiment of the driving safety monitoring method based on multi-sensor and AI judgment of the present application is provided;

[0053] Figure 3 A flowchart diagram of the third embodiment of the driving safety monitoring method based on multi-sensor and AI judgment of the present application is provided;

[0054] Figure 4 A flowchart diagram of a fourth embodiment of a driving safety monitoring method based on multiple sensors and AI judgment according to the present application;

[0055] Figure 5 A flowchart diagram of a fifth embodiment of a driving safety monitoring method based on multiple sensors and AI judgment according to the present application;

[0056] Figure 6 A flowchart diagram of a sixth embodiment of a driving safety monitoring method based on multiple sensors and AI judgment according to the present application;

[0057] Figure 7 This is a schematic diagram of the module structure of a driving safety monitoring device based on multi-sensors and AI judgment according to an embodiment of the present application;

[0058] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the driving safety monitoring method based on multiple sensors and AI judgment in an embodiment of the present application.

[0059] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0061] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0062] The main solution of the embodiment of the present application is: collecting hardware information, the hardware information including alcohol concentration and driver's physiological information; processing the hardware information through a preset algorithm to obtain a processing result; when the processing result is an abnormal state, analyzing the driver's sobriety through voice interaction to obtain an analysis result; based on the processing result and / or the analysis result, obtaining a danger level assessment result and taking corresponding treatment measures to achieve driving safety monitoring.

[0063] In this embodiment, for ease of description, the following description is made taking the internal actuator of the driving safety monitoring system as the execution subject.

[0064] The existing driving monitoring system has technical problems such as low data fusion accuracy, limited real-time performance, single detection system, privacy data leakage and information security risks.

[0065] This application provides a solution to improve the accuracy and real-time performance of driving safety monitoring and enhance user data privacy and security protection.

[0066] It can be seen from the above embodiments that the present application adopts multiple sensors and AI voice interaction judgment to monitor driving safety in real time and take corresponding measures in risky situations, thereby solving the technical problems of low data fusion accuracy, limited real-time performance, single detection system, privacy data leakage and information security risks in the driving monitoring system, and achieving the improvement of the accuracy and real-time performance of driving safety monitoring and the enhancement of user data privacy and security protection.

[0067] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc. The following takes the internal actuator of the driving safety monitoring system as an example to illustrate this embodiment and the following embodiments.

[0068] Based on this, the embodiment of the present application provides a driving safety monitoring method based on multi-sensor and AI judgment, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the driving safety monitoring method based on multiple sensors and AI judgment of the present application.

[0069] In this embodiment, the driving safety monitoring method based on multi-sensor and AI judgment includes steps S10 to S40:

[0070] Step S10, collecting hardware information, wherein the hardware information includes alcohol concentration and driver physiological information.

[0071] It should be noted that alcohol concentration usually refers to the content of alcohol in a liquid, which can be expressed in different units, such as volume percentage (% vol), mass percentage (% m / m), etc. In drunk driving detection, the unit of alcohol concentration is milligrams per 100 milliliters (mg / 100ml). According to the "Thresholds and Inspections of Blood and Breath Alcohol Contents for Vehicle Drivers" (GB19522-2004) issued by the State Administration of Quality Supervision, Inspection and Quarantine, when the blood alcohol content exceeds 20mg / 100ml but is lower than 80mg / 100ml, it is regarded as drunk driving. When the blood alcohol content reaches or exceeds 80mg / 100ml, it is regarded as drunk driving. Both situations are illegal and will be subject to corresponding legal penalties.

[0072] In addition, it should be noted that the driver's physiological information is the driver's physiological status information collected through cameras and multiple sensors, including facial expressions, eye conditions, head posture, heart rate, pulse and blood pressure, and may also include body temperature, respiratory rate, blood oxygen saturation, electrocardiogram signals and other information.

[0073] Step S20, processing the hardware information by a preset algorithm to obtain a processing result.

[0074] It should be noted that the preset algorithm is a system algorithm, which refers to a series of well-defined computing steps used in a computer system to perform specific tasks or solve specific problems. These algorithms can be simple or complex, and they are the basis of computer science and software engineering, including sorting algorithms, graph algorithms, search algorithms, dynamic programming algorithms, machine learning algorithms, etc. This embodiment and the following embodiments do not specifically limit the selection of the preset algorithm.

[0075] After receiving the hardware information collected by the camera and multiple sensors, the driving safety monitoring system transmits this information to the central processing unit inside the system. The alcohol concentration and the driver's physiological information in the hardware information are analyzed and processed by the system algorithm of the central processing unit to determine whether the driver is driving under the influence of alcohol or whether the physiological state is abnormal, and obtain the processing result. The processing result includes two situations: normal state and abnormal state. Normal state means that the driver is not currently driving under the influence of alcohol, and the physiological state is normal, without fatigue driving and disease driving behavior. Abnormal state means that the driver is currently driving under the influence of alcohol and / or fatigue driving and / or disease driving behavior.

[0076] Step S30, when the processing result is an abnormal state, the driver's sobriety level is analyzed through voice interaction to obtain an analysis result.

[0077] It should be noted that voice interaction is voice question-and-answer interaction through AI. It is an interactive method that uses speech recognition and natural language processing technology to enable computer systems to understand and respond to human voice commands. This interactive mode imitates natural conversations between people, allowing users to control devices, obtain information or perform tasks by speaking. Voice interaction includes speech recognition, natural language understanding, question-and-answer management, natural language generation and other components, which are used to examine the driver's reaction time, language clarity and logical judgment ability to assess his or her level of consciousness.

[0078] In addition, it should be noted that the analysis results include whether the driver is conscious and whether the driver is not conscious. If the driver is conscious, it means that there is no obvious problem with the driver's current state of consciousness, and the driver can communicate normally and answer some basic and common sense questions. If the driver is not conscious, it means that there is an obvious problem with the driver's current state of consciousness, and the driver cannot communicate normally or answer some basic and common sense questions.

[0079] Step S40, obtaining a danger level assessment result and taking corresponding processing measures according to the processing result and / or the analysis result to achieve driving safety monitoring.

[0080] It should be noted that the danger level assessment result is the result of judging the driver's current state based on the processing results and / or analysis results, and assessing the current driving safety level based on the state, including mild danger and severe danger. Mild danger means that the driver's alcohol concentration slightly exceeds the standard and / or is slightly fatigued driving and / or is physically unwell, but the voice interaction judges that the driver is conscious. Severe danger means that the driver's alcohol concentration exceeds the standard and / or is fatigued driving and / or driving due to illness, and is not conscious.

[0081] Drunk driving will greatly affect the driver's reaction ability, judgment ability and operation accuracy, making it impossible for the driver to respond promptly and correctly to various emergencies. Fatigue driving is also extremely dangerous. Long-term driving will cause the driver to be inattentive, slow to react, and even have a short sleep state, which can easily cause major traffic accidents when driving at high speeds. Sick driving may cause the driver to feel unwell, suffer from sudden illness, or even lose consciousness, and be unable to control the vehicle normally. Under different danger levels, different handling measures are adopted to ensure the driver's driving safety, avoid driving accidents caused by drunk driving, fatigue driving or sick driving, so as to monitor the driver's driving safety.

[0082] This embodiment provides a driving safety monitoring method based on multi-sensors and AI judgment. By adopting multi-sensors and AI voice interactive judgment, driving safety is monitored in real time, and corresponding measures are taken in risky situations. The technical problems of low data fusion accuracy, limited real-time performance, single detection system, privacy data leakage and information security risks in the driving monitoring system are solved, thereby improving the accuracy and real-time performance of driving safety monitoring and enhancing user data privacy and security protection.

[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S20 includes steps S21 to S25:

[0084] Step S21, obtaining alcohol concentration and driver physiological information from the hardware information.

[0085] It should be noted that hardware information not only includes alcohol concentration and driver physiological information collected by cameras and multiple sensors, but also includes vehicle current speed, current acceleration, vehicle environment information, road obstacle information, driving visibility information and other information. These external information and driver status information are jointly used as hardware information and processed in real time by the system's central processing unit. External information is used to monitor the vehicle's driving conditions in real time to avoid external conditions that pose a threat to driving safety, such as speeding due to high speed, rollover due to sudden obstacles, slipping due to slippery ground, etc. Driver status information is used to monitor the driver's driving conditions in real time to avoid threats to driving safety due to the driver's personal status.

[0086] Step S22: Based on a preset algorithm, the alcohol concentration is compared with an alcohol concentration threshold to obtain a first comparison result.

[0087] It should be noted that the alcohol concentration threshold is the driver's blood alcohol content standard specified in the "Blood and Breath Alcohol Content Threshold and Test for Vehicle Drivers" (GB19522-2004). There are three standard ranges: 0-20mg / 100ml, 20mg / 100ml-80mg / 100ml, and 80mg / 100ml-∞ (∞ represents an infinite value).

[0088] In the central processing unit of the system, the alcohol concentration collected by the sensor is analyzed by the system algorithm, and the alcohol concentration is compared with the alcohol concentration threshold to determine the range of alcohol concentration. The first comparison results include no drinking, drinking and driving, and drunk driving. When the blood alcohol content does not exceed 20mg / 100ml, the first comparison result is no drinking. When the blood alcohol content exceeds 20mg / 100ml but is lower than 80mg / 100ml, the first comparison result is drinking and driving. When the blood alcohol content reaches or exceeds 80mg / 100ml, the first comparison result is drunk driving.

[0089] Step S23: Based on the preset algorithm, the driver's physiological information is compared with the normal state physiological information to obtain a second comparison result.

[0090] It should be noted that normal physiological information is a series of physiological parameters of the human body in a healthy and disease-free state, which can be used as a benchmark for assessing individual health status. For example, the following are the normal physiological information ranges of some healthy adults:

[0091] Heart Rate: 60-100 beats / minute. Blood Pressure: Systolic (high pressure) 90-120 mmHg, Diastolic (low pressure) 60-80 mmHg. Pulse: 60-100 beats per minute. Body Temperature: Oral temperature: 36.5-37.5°C (97.7-99.5°F); Axillary temperature: 36.0-37.0°C (96.8-98.6°F). Respiratory Rate: 12-20 times / minute. Oxygen Saturation: 95%-100%.

[0092] In a normal and healthy state, a person's facial expression usually shows: relaxed facial muscles, no tense or deliberate expressions, no frequent yawning, no distortion or frowning due to pain or discomfort, no dark circles or bags under the eyes, no signs of excessive fatigue, no swelling, redness or inflammation, etc. In a normal and healthy state, a person's eye condition usually shows: no frequent blinking or long-term eye closure, the white of the eye (sclera) usually appears white or slightly bluish, no congestion or yellowing, no abnormal eye secretions, such as excessive tears, eye mucus or purulent secretions, no pain, itching or burning in the eyes, and the pupil has a normal contraction and expansion reaction to light changes, etc. In a normal and healthy state, a person's head posture usually shows: the head is kept straight forward or tilted moderately, the neck muscles maintain the stability of the head with appropriate tension, so that the eyes can look straight ahead, so as to effectively receive visual information from the front, and the head remains stable without frequent twisting or shaking.

[0093] When the driver's physiological information is compared with normal physiological information and both are within the standard range or perform normally, the second comparison result is determined to be that the driver's physiological state is normal. When any item of the driver's physiological information is compared with normal physiological information and both are not within the standard range or perform abnormally, the second comparison result is determined to be that the driver's physiological state is abnormal.

[0094] Step S24, when the first comparison result is that the driver has not drunk alcohol and the second comparison result is that the driver's physiological state is normal, determining that the processing result is a normal state.

[0095] After comparing and analyzing the driver's alcohol concentration and physiological information, the processing result after processing the hardware information is determined according to the first comparison result and the second comparison result. When the first comparison result is that the driver has not drunk alcohol and the second comparison result is that the driver's physiological state is normal, the processing result is determined to be a normal state. At this time, the driver's alcohol concentration test is normal, there is no drunk driving behavior, the physiological state is normal, and there is no fatigue driving or disease driving behavior. The driver in this state can drive the vehicle.

[0096] Step S25, when the first comparison result is driving under the influence of alcohol or drunk driving, or the second comparison result is abnormal physiological state of the driver, determining that the processing result is abnormal state.

[0097] After comparing and analyzing the driver's alcohol concentration and physiological information, the processing result after processing the hardware information is determined according to the first comparison result and the second comparison result. When the first comparison result is drunk driving or drunk driving, or the second comparison result is that the driver's physiological state is abnormal, the processing result is determined to be abnormal state. At this time, the driver's alcohol concentration exceeds the standard and / or the physiological state is abnormal, and there is drunk driving or drunk driving behavior, or fatigue driving and / or disease driving behavior.

[0098] The present embodiment provides a driving safety monitoring method based on multi-sensors and AI judgment, which obtains alcohol concentration and driver's physiological information from the hardware information; based on a preset algorithm, compares the alcohol concentration with an alcohol concentration threshold to obtain a first comparison result; based on the preset algorithm, compares the driver's physiological information with normal physiological information to obtain a second comparison result; when the first comparison result is no drinking and the second comparison result is that the driver's physiological state is normal, determines that the processing result is a normal state; when the first comparison result is drinking and driving or drunk driving, or the second comparison result is that the driver's physiological state is abnormal, determines that the processing result is an abnormal state, thereby realizing real-time monitoring of the driver's drinking and physiological state.

[0099] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S30 includes steps S31 to S34:

[0100] Step S31, when the processing result is an abnormal state, initiating a question-and-answer interaction through voice to obtain question-and-answer data.

[0101] When the processing result is abnormal status, the driver's alcohol concentration exceeds the standard and / or his physiological state is abnormal, and he has driven under the influence of alcohol or drunk driving, or has driven while fatigued and / or sick. At this time, it is necessary to further determine whether the driver is conscious and whether he meets the consciousness conditions for normal driving of the vehicle.

[0102] By initiating question-and-answer interaction through voice, the AI ​​voice can communicate with the driver through questions and answers. The content of the conversation can be some simple reaction test questions. The question types can include quick question and answer reaction test questions, digital reaction test questions, color word reaction test questions, factual question and answer reaction test questions, logical reaction test questions, etc. These types of questions can automatically generate question stems and answers in the AI ​​voice system.

[0103] For example, a quick quiz reaction test question: If I say "daytime", what do you say? Answer: Night. A number reaction test question: Please say a number that is 5 more than 7 as quickly as possible. Answer: 12. A color word reaction test question: If the word "red" is written on a card, but the card itself is blue, what is the actual color of the card? Answer: Blue. A factual quiz reaction test question: What is the capital of China? Answer: Beijing. A logical reaction test question: If all cats are afraid of water, and Tom is a cat, then is Tom afraid of water? Answer: Yes, Tom is afraid of water.

[0104] The question types may also include questions based on the driver's personal information, such as please answer your name, please answer your height, please answer your mobile phone number, please answer your parents' ages, etc. These types of questions require the driver to input the question stem and answer in advance in the AI ​​voice system.

[0105] During the question-and-answer process, the AI ​​voice system records the driver's accuracy, time, and pronunciation of his answers in real time, and stores these data as question-and-answer data in the system's storage module. When the system needs these data later, it can obtain them from the storage module at any time.

[0106] It should be noted that AI voice technology, namely artificial intelligence voice technology, refers to the technology that uses artificial intelligence algorithms to understand and generate human voice, enabling machines to recognize, understand and generate voice, thereby achieving natural voice interaction with humans.

[0107] Step S32, analyzing the driver's reaction time, language clarity and logical judgment ability based on the question and answer data.

[0108] According to the driver's answering time and pronunciation in the question and answer data, the driver's reaction time and language clarity are analyzed, and according to the correct rate of the driver's answering questions, the driver's logical judgment ability is analyzed. If the time to answer the question is short, it is judged that the driver reacts quickly, and if the time to answer the question is long, it is judged that the driver reacts slowly. For example, if the average answer reaction time exceeds 3s, and the answer time of a single question exceeds 8s, it is determined that the time to answer the question is long. When the pronunciation of the answering question is more coherent, it is judged that the driver's language clarity is good. When the pronunciation of the answering question is incoherent, it is judged that the driver's language clarity is poor. For example, when the driver's speech is unclear, the pronunciation of words is sticky and incoherent, and the speech is stuttering, it is determined that the pronunciation of the answering question is incoherent. If the correct rate of answering is high, it is judged that the driver's logical judgment ability is good, and if the correct rate of answering is low, it is judged that the driver's logical judgment ability is poor. For example, if the correct rate is greater than 80%, it is determined that the correct rate of answering is high.

[0109] Step S33, when the reaction time, language clarity and logical judgment ability meet the preset standards, determine that the analysis result is that the driver is conscious.

[0110] It should be noted that the preset standard means that the driver's reaction time, speech clarity and logical judgment ability all meet the standard of good performance, that is, quick reaction, clear speech and strong logical judgment ability.

[0111] During the voice interaction process, when the driver's reaction time, language clarity and logical judgment ability all meet the preset standards, it means that the driver's consciousness level at this time is at a normal level, and the analysis result is determined to be that the driver is conscious.

[0112] For example, if the driver's average response time does not exceed 3 seconds, his answers are clear and coherent, and his correct answers are higher than 80%, it is determined that the driver's response time, language clarity, and logical judgment ability meet the preset standards and the driver is conscious.

[0113] Step S34, when the reaction time, language clarity and logical judgment ability do not meet the preset standards, determine that the analysis result is that the driver is not conscious.

[0114] During the voice interaction process, if any one of the driver's reaction time, language clarity and logical judgment ability does not meet the preset standards, it means that the driver's consciousness level is not at a normal level, and the analysis result is determined to be that the driver is not conscious.

[0115] For example, if the driver's average response time exceeds 3 seconds, and / or the driver's speech is unclear when answering questions, and / or the correct answer rate is less than 80%, it is determined that the driver's response time, language clarity and logical judgment ability do not meet the preset standards and the driver is not conscious.

[0116] The present embodiment provides a driving safety monitoring method based on multiple sensors and AI judgment. When the processing result is an abnormal state, a question-and-answer interaction is initiated through voice to obtain question-and-answer data; the driver's reaction time, speech clarity and logical judgment ability are analyzed according to the question-and-answer data; when the reaction time, speech clarity and logical judgment ability meet the preset standards, the analysis result is determined to be that the driver is conscious; when the reaction time, speech clarity and logical judgment ability do not meet the preset standards, the analysis result is determined to be that the driver is not conscious, thereby realizing AI voice judgment of the driver's consciousness level.

[0117] Based on the first embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 4 , step S40 includes steps S41 to S45:

[0118] Step S41, evaluating the danger level of the driver's state according to the processing result and / or the analysis result.

[0119] When the processing result is a normal state, it means that the driver has not drunk alcohol and is in normal physiological state at this time, so there is no need to further judge the driver's level of consciousness, that is, there is no need to conduct a consciousness analysis process, and there is no analysis result. The driver's state can be directly evaluated for danger level through the processing result; when the processing result is an abnormal state, it means that the driver has drunk alcohol and / or is in abnormal physiological state, so it is necessary to further judge the driver's level of consciousness, that is, it is necessary to conduct a consciousness analysis process, and the driver's state is evaluated for danger level through the processing results and the analysis results.

[0120] Step S42, when the processing result is a normal state, determining the danger level is no danger.

[0121] When the processing result is normal, it means that the driver has not drunk alcohol and is in normal physiological condition, then the danger level is determined to be no danger. At this time, the driver meets the requirements of normal driving and can drive the vehicle. There is no need to take other measures to warn the driver of danger.

[0122] Step S43, when the processing result is that the state is abnormal, obtain the first comparison result and the second comparison result.

[0123] When the processing result is an abnormal state, it means that the driver is drinking and / or his physiological state is abnormal at this time, so it is necessary to obtain the first comparison result in the alcohol concentration detection process and the second comparison result in the physiological state detection process, and then determine whether the driver is driving under the influence of alcohol, driving while drunk, driving while fatigued, and driving due to illness.

[0124] It should be noted that drinking and driving will reduce the driver's reaction ability. When faced with emergencies on the road, such as pedestrians suddenly crossing the road or the front car braking suddenly, the driver cannot react as quickly as in normal conditions. The braking reaction time will be prolonged, which is easy to cause accidents such as rear-end collision or collision with pedestrians. At the same time, drinking will also affect the driver's judgment ability, and there will be deviations in the judgment of vehicle speed, distance and road conditions, such as misestimating the curvature of the curve, which will cause the vehicle to lose control; the harm of drunk driving is even more serious, because at this time the driver's brain is deeply paralyzed by alcohol, and the body's sense of balance and coordination are greatly damaged. The driver may have blurred vision and cannot clearly see the traffic lights, road signs or the movements of other vehicles and pedestrians. In addition, his ability to operate the vehicle becomes very poor, the steering wheel may be unstable, and the pedals may be stepped on incorrectly. These factors combined make the accidents caused by drunk driving often serious high-speed collisions, rollovers and other large-scale accidents, which bring huge disasters to the life and property safety of the driver and other road users. The main difference between driving under the influence of alcohol and drunk driving is the different alcohol content in the blood. As the alcohol content increases, the degree of damage to driving ability gradually increases. The possibility and severity of serious consequences caused by drunk driving are far higher than driving under the influence of alcohol.

[0125] In addition, it should be noted that fatigue driving is a very dangerous behavior. When the driver is in a fatigued state, the body's reaction ability will be greatly reduced, just like a slow machine, and it is difficult to respond in time to sudden situations on the road, such as vehicles suddenly changing lanes, pedestrians crossing the road, or traffic lights changing. Attention will also be unable to concentrate, thoughts may be separated from driving, eyes may unconsciously squint or vision will become blurred, observation of road conditions will become rough, and it is easy to ignore important traffic signs and the dynamics of surrounding vehicles. Moreover, fatigue will also reduce the driver's judgment, and there will be serious deviations in the grasp of vehicle speed, following distance, etc., which will increase the probability of rear-end collisions, collisions with obstacles, or driving off the road. Driving while sick is also extremely dangerous. When the driver is ill, the disease consumes the driver's physical strength and the body functions are in an abnormal state. For example, heart disease may cause sudden panic and chest pain, making it impossible for the driver to control the vehicle normally while driving. Vertigo may cause the driver's vision to suddenly go black or the world to spin while driving, and the vehicle may be lost in an instant. Colds and fevers can also make people dazed and slow to react. In this case, driving a vehicle is like walking in the fog. Traffic accidents may occur at any time due to physical discomfort, posing a serious threat to the life and property of the driver and other road users.

[0126] Step S44, when the analysis result is that the driver is conscious, the first comparison result is drunk driving, and the second comparison result is that the driver is in normal physiological state, the danger level is determined to be mild danger, and a danger warning is issued.

[0127] When the analysis result shows that the driver is conscious, the first comparison result is drunk driving, and the second comparison result shows that the driver is in normal physiological state, the driver's alcohol concentration is slightly above the standard but his physiological state is normal, and the AI ​​voice determines that he is conscious and able to think normally, and determines that the danger level in this case is mild danger. Danger warnings can be issued through voice, for example, through voice prompts: "You are not in a suitable state for driving, please pay attention to safety", and text prompts can also be issued through the human-computer interaction interface, for example: displaying classical Chinese on the car screen: "You are not in a suitable state for driving, please pay attention to safety".

[0128] For example, a driver drinks a small amount of alcohol at dinner and then gets in the car to go home. After the vehicle is started, the alcohol concentration sensor detects that the alcohol content in the driver's breath slightly exceeds the standard. The system starts the voice interaction module and asks questions, and detects that his reaction time and consciousness state are normal. Therefore, the danger level is assessed as mild. The system only reminds the driver to pay attention to safety by voice and does not take further measures.

[0129] Step S45, when the analysis result is that the driver is unconscious or the first comparison result is drunk driving or the second comparison result is that the driver is in an abnormal physiological state, the danger level is determined to be severe danger, and the vehicle is controlled to be locked and enter emergency contact mode.

[0130] It should be noted that the emergency contact mode is a functional system that can quickly contact the outside world when the vehicle encounters an emergency. It can call the preset emergency contact for help, and send a distress text message or push notification containing the vehicle's location information and driver's status. At the same time, the emergency contact mode can also directly establish a communication connection with the designated driver platform or public security department (such as the 110 alarm center, 120 emergency center) to ensure the safety of the vehicle and passengers.

[0131] When the analysis result is that the driver is not conscious, or the first comparison result is drunk driving, or the second comparison result is that the driver is in an abnormal physiological state, the driver's alcohol concentration is seriously exceeded, or his physiological state is abnormal, or the AI ​​voice determines that he is not conscious. At this time, the driver can no longer control his limbs and brain normally, cannot think normally, and cannot drive. The danger level in this situation is determined to be severe danger, and the vehicle is automatically locked, driving is prohibited, and the emergency contact mode is entered.

[0132] For example, if a driver drives drunk after drinking, the onboard system detects that his alcohol concentration is seriously above the standard, and his answers in the voice question are vague and his reaction is slow. The system determines that he is in serious danger. However, if the driver tries to drive while drunk, the system immediately activates the alarm system. If automatic contact with family members is ineffective, the system directly calls the police and transmits the drunk driving situation and the camera image in the car to the local traffic police department or the platform background to ensure quick processing.

[0133] In addition, it should be noted that in the emergency contact mode, the system can choose to contact emergency contacts, traffic police departments, designated driver platforms and other personnel or platforms. This embodiment and the following embodiments are described using the system automatically contacting emergency contacts and / or designated driver platforms as an example.

[0134] In a feasible implementation manner, step S45 includes steps S451 to S453:

[0135] Step S451, when the vehicle enters the emergency contact mode, automatically select to contact an emergency contact and / or a designated driver platform.

[0136] It should be noted that the designated driver platform is an Internet platform that connects designated drivers and customers who need designated driver services. When a customer cannot drive due to drinking, physical discomfort or other reasons, he or she can post a designated driver request on the designated driver platform. The platform will quickly match nearby designated drivers based on the customer's location, time of demand and other information to provide customers with proxy driving services.

[0137] After entering the emergency contact mode in the car, the system will automatically choose to contact the emergency contact or the designated driver platform. The system will give priority to contacting the emergency contact. If the emergency contact cannot be contacted and the driver or the emergency contact allows it, the system will automatically contact the designated driver platform and send information such as the vehicle location and driver status to the designated driver platform so that the designated driver platform can assign a designated driver.

[0138] Step S452, when the designated driver platform is selected for contact, the audio and video data of the designated driver process is recorded and the safety status in the car is monitored.

[0139] When the system chooses to contact the designated driver platform to provide assistance to the driver, the system will activate the camera, recording equipment and voice system throughout the process to record the entire process of designated driving, including video and audio recording, and store the recorded audio and video data to the storage module in the system, and monitor the safety conditions in the car in real time, such as real-time monitoring of the designated driver's behavior, and real-time monitoring of the driver's physiological state and consciousness state.

[0140] Step S453, when the safety condition in the vehicle is abnormal, an alarm is automatically sounded and the audio and video data is transmitted to the background server.

[0141] When the safety conditions in the car are abnormal, such as abnormal behavior of the designated driver, abnormal physiological state or abnormal consciousness state of the driver, the system will automatically alarm and transmit the audio and video data recording the designated driving process to the background server to protect data privacy and security.

[0142] For example, when the system determines that the danger level is severe, it automatically locks the vehicle and activates the emergency contact mode, calling the driver's family. At the same time, if the driver chooses the designated driver service, the system automatically records the journey after the designated driver gets on the car, activates the in-car camera and recording equipment for full monitoring, and analyzes the driver's safety in real time. During the designated driver process, if there is no abnormality, the system will record and store the designated driver record normally; if there is an abnormality, the system will directly alarm and upload the recording and video to the official backend.

[0143] Through emergency contact and designated driver services and real-time monitoring of the conditions inside the car, the driver can be provided with safety protection.

[0144] The present embodiment provides a driving safety monitoring method based on multi-sensors and AI judgment, which evaluates the danger level of the driver's state according to the processing result and / or the analysis result; when the processing result is a normal state, the danger level is determined to be no danger; when the processing result is an abnormal state, a first comparison result and a second comparison result are obtained; when the analysis result is that the driver is conscious, the first comparison result is drunk driving, and the second comparison result is that the driver's physiological state is normal, the danger level is determined to be mild danger, and a danger prompt is issued; when the analysis result is that the driver is not conscious, or the first comparison result is drunk driving, or the second comparison result is that the driver's physiological state is abnormal, the danger level is determined to be severe danger, the vehicle is controlled to be locked and enter emergency contact mode, thereby ensuring driving safety.

[0145] Based on the first embodiment of the present application, in the fifth embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 5 , step S10 includes steps S11 to S14:

[0146] Step S11, collecting the driver's facial expression, eye state and head posture through a camera.

[0147] It should be noted that the camera is a vehicle-mounted camera used to identify the driver's facial condition. It is mainly based on computer vision technology and uses high-resolution image sensors to capture the driver's facial images, such as eyes, mouth, eyebrows and other parts. It is generally installed near the rearview mirror inside the car or in the center of the top of the car. Such a position can ensure a good field of view and can fully capture the driver's face. Its field of view is usually focused on the driver's head area, including the forehead, eyes, nose, mouth and chin, to ensure that sufficient facial information can be obtained for accurate status analysis.

[0148] Step S12, collecting the alcohol concentration in the driver's exhaled gas through an alcohol concentration sensor.

[0149] It should be noted that the alcohol concentration sensor is a device for detecting the alcohol content in the environment. A common type of alcohol concentration sensor is a semiconductor alcohol concentration sensor. This sensor mainly utilizes the characteristics of semiconductor materials. When alcohol molecules are adsorbed on the surface of the semiconductor, the electrical properties of the semiconductor will change, such as resistance changes. By detecting the change of this electrical parameter, the concentration of alcohol can be calculated. For example, when alcohol gas contacts the semiconductor sensitive element of the sensor, its resistance will decrease as the alcohol concentration increases. The circuit in the sensor can convert this resistance change into an electrical signal output, and then obtain the value of the alcohol concentration after amplification and processing. There is also an electrochemical alcohol concentration sensor, the working principle of which is based on an electrochemical redox reaction. Inside the sensor, there is a pair of electrodes. When alcohol gas diffuses into the sensor, an oxidation reaction will occur on the electrode surface. In this process, an electric current will be generated, and the magnitude of the current is proportional to the alcohol concentration. By accurately measuring the magnitude of the current, the concentration of alcohol can be determined. This sensor has high sensitivity and accuracy and is commonly used in high-precision alcohol detection equipment. This embodiment does not specifically limit the selection of the alcohol concentration sensor.

[0150] The vehicle-mounted alcohol concentration sensor is generally installed in the vehicle's ventilation system or steering wheel. When the driver enters the vehicle and starts it, the sensor starts working and detects the air exhaled by the driver during breathing. If the driver drinks, the exhaled air will contain alcohol molecules. The alcohol concentration sensor can be used to detect the alcohol concentration and then determine the driver's drinking status.

[0151] Step S13, collecting the driver's heart rate, pulse and blood pressure through a biometric information sensor.

[0152] It should be noted that the biometric information sensor is used to collect physiological status information such as the driver's heart rate, pulse and blood pressure.

[0153] For the collection of heart rate and pulse, photoelectric sensors can be used. They are usually installed on the armrests of the car seat or the steering wheel to facilitate contact with the driver's skin. Based on photoplethysmography (PPG), the sensor contains a light-emitting diode (LED) and a photodetector. The LED emits light of a specific wavelength, usually green or red. The light penetrates the driver's skin tissue, is partially absorbed by the blood, and partially reflected back. As the heart beats, the blood volume in the blood vessels changes periodically. This change causes the intensity of the reflected light to also fluctuate periodically. The photodetector captures the change in the intensity of the reflected light and converts it into an electrical signal. After signal processing and analysis, the heart rate and pulse data can be obtained.

[0154] For collecting blood pressure, a pressure sensor can be used. The pressure sensor can be installed on the seat belt or seat back of the car seat. When the driver sits in the seat and fastens the seat belt, the sensor can sense the pressure applied by the body. During the contraction and relaxation of the heart, changes in blood pressure inside the body will cause tiny changes in the blood vessels on the surface of the body, which in turn causes changes in the pressure felt by the sensor. The sensor converts this pressure change into an electrical signal, analyzes and calibrates it through complex algorithms, and then estimates the blood pressure value.

[0155] In addition, it should be noted that there are various types of sensors used to collect driver's biological information. When different sensors are used, the method and accuracy of obtaining physiological information are slightly different, but it does not affect the system's analysis and processing of physiological information. This embodiment does not specifically limit the selection of biological information sensors.

[0156] Step S14: taking the facial expression, the eye state, the head posture, the heart rate, the pulse and the blood pressure as the driver's physiological information.

[0157] The driver's status information collected by the camera and the sensor is summarized as the driver's physiological information and stored in the system's storage module as part of the hardware information.

[0158] This embodiment provides a driving safety monitoring method based on multiple sensors and AI judgment, which collects the driver's facial expressions, eye conditions and head posture through a camera; collects the alcohol concentration in the driver's exhaled gas through an alcohol concentration sensor; collects the driver's heart rate, pulse and blood pressure through a bio-information sensor; and uses the facial expressions, eye conditions, head posture, heart rate, pulse and blood pressure as the driver's physiological information, thereby realizing efficient collection of the driver's status information.

[0159] Based on the first embodiment of the present application, in the sixth embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 6 , after step S40, steps S01 to S03 are included:

[0160] Step S01, when the processing result is an abnormal state, obtaining the question and answer data of the voice interaction.

[0161] When the processing result is an abnormal status, the system conducts question-and-answer interaction through AI language, and the interaction will generate question-and-answer data. Since this data is generated through AI voice question and answer, it will include the driver's privacy information, and the privacy of this data needs to be protected.

[0162] Step S02, encrypt the hardware information and / or the question and answer data to obtain encrypted information, and store the encrypted information locally.

[0163] Hardware information includes the driver's physiological state information, and question and answer data includes the driver's response information when answering questions. When the questions and answers are about the driver's personal information, the question and answer data may also include the driver's privacy issues. Therefore, it is necessary to encrypt this information and data, obtain encrypted information, and store the encrypted information locally. In non-emergency situations, the encrypted information will not be uploaded to the background server, but only saved in the system's local storage module, so as to enhance user data privacy and security protection.

[0164] It should be noted that non-emergency situations refer to normal vehicle operation without system or component failures.

[0165] Step S03: in an emergency, uploading the encrypted information to a backend server.

[0166] It should be noted that an emergency situation refers to a situation where the vehicle encounters internal failure or external damage, some functions can no longer operate normally, or the system or components fail.

[0167] In an emergency, if the vehicle malfunctions and some functions are no longer able to operate normally, the encrypted information may be leaked. In this case, the encrypted information will be uploaded to the backend server to avoid unnecessary information leakage.

[0168] This embodiment provides a driving safety monitoring method based on multi-sensors and AI judgment. When the processing result is an abnormal state, the question and answer data of the voice interaction is obtained; the hardware information and / or the question and answer data are encrypted to obtain encrypted information, and the encrypted information is stored locally; in an emergency, the encrypted information is uploaded to a background server, thereby achieving data security and privacy protection.

[0169] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the driving safety monitoring method based on multi-sensors and AI judgment of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0170] This application also provides a driving safety monitoring device based on multi-sensor and AI judgment, please refer to Figure 7 , the device comprises:

[0171] The information collection module 10 is used to collect hardware information through sensors, where the hardware information includes alcohol concentration and driver physiological information.

[0172] The data processing module 20 is used to process the hardware information through a preset algorithm to obtain a processing result.

[0173] The state analysis module 30 is used to analyze the driver's sobriety level through voice interaction to obtain an analysis result when the processing result is an abnormal state.

[0174] The evaluation control module 40 is used to obtain a danger level evaluation result and take corresponding processing measures according to the processing result and / or the analysis result to achieve driving safety monitoring.

[0175] In one embodiment, the data processing module 20 is also used to obtain alcohol concentration and driver physiological information from the hardware information; based on a preset algorithm, compare the alcohol concentration with an alcohol concentration threshold to obtain a first comparison result; based on the preset algorithm, compare the driver's physiological information with normal physiological information to obtain a second comparison result; when the first comparison result is no drinking and the second comparison result is that the driver's physiological state is normal, determine that the processing result is a normal state; when the first comparison result is drinking and driving or drunk driving, or the second comparison result is that the driver's physiological state is abnormal, determine that the processing result is an abnormal state.

[0176] In one embodiment, the state analysis module 30 is also used to initiate a question-and-answer interaction through voice to obtain question-and-answer data when the processing result is an abnormal state; analyze the driver's reaction time, speech clarity and logical judgment ability according to the question-and-answer data; when the reaction time, speech clarity and logical judgment ability meet the preset standards, determine the analysis result as the driver is conscious; when the reaction time, speech clarity and logical judgment ability do not meet the preset standards, determine the analysis result as the driver is not conscious.

[0177] In one embodiment, the evaluation control module 40 is also used to evaluate the danger level of the driver's state according to the processing result and / or the analysis result; when the processing result is a normal state, determine the danger level as no danger; when the processing result is an abnormal state, obtain a first comparison result and a second comparison result; when the analysis result is that the driver is conscious, the first comparison result is drunk driving, and the second comparison result is that the driver's physiological state is normal, determine that the danger level is mild danger, and issue a danger warning; when the analysis result is that the driver is not conscious, or the first comparison result is drunk driving, or the second comparison result is that the driver's physiological state is abnormal, determine that the danger level is severe danger, control the vehicle to lock and enter emergency contact mode.

[0178] In one embodiment, the evaluation control module 40 is also used to automatically select an emergency contact and / or a designated driver platform to contact when the vehicle enters the emergency contact mode; when selecting to contact the designated driver platform, it records the audio and video data of the designated driver process and monitors the safety conditions in the vehicle; when the safety conditions in the vehicle are abnormal, it automatically alarms and transmits the audio and video data to the background server.

[0179] In one embodiment, the information acquisition module 10 is also used to collect the driver's facial expressions, eye conditions and head posture through a camera; collect the alcohol concentration in the driver's exhaled gas through an alcohol concentration sensor; collect the driver's heart rate, pulse and blood pressure through a biometric information sensor; and use the facial expressions, the eye conditions, the head posture, the heart rate, the pulse and the blood pressure as the driver's physiological information.

[0180] In one embodiment, the evaluation control module 40 is also used to obtain the question and answer data of the voice interaction when the processing result is an abnormal state; encrypt the hardware information and / or the question and answer data to obtain encrypted information, and store the encrypted information locally; in an emergency, upload the encrypted information to the background server.

[0181] The driving safety monitoring device based on multi-sensors and AI judgment provided by the present application adopts the driving safety monitoring method based on multi-sensors and AI judgment in the above-mentioned embodiment, which can solve the technical problems of how to improve the accuracy and real-time performance of driving safety monitoring and enhance the privacy and security protection of user data. Compared with the prior art, the beneficial effects of the driving safety monitoring device based on multi-sensors and AI judgment provided by the present application are the same as the beneficial effects of the driving safety monitoring method based on multi-sensors and AI judgment provided by the above-mentioned embodiment, and the other technical features of the driving safety monitoring device based on multi-sensors and AI judgment are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0182] The present application provides a driving safety monitoring device based on multiple sensors and AI judgment. The driving safety monitoring device based on multiple sensors and AI judgment includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the driving safety monitoring method based on multiple sensors and AI judgment in the above-mentioned embodiment one.

[0183] Reference below Figure 8 , which shows a schematic diagram of the structure of a driving safety monitoring device based on multi-sensors and AI judgment suitable for implementing the embodiment of the present application. The driving safety monitoring device based on multi-sensors and AI judgment in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The driving safety monitoring device based on multiple sensors and AI judgment shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0184] like Figure 8As shown, the driving safety monitoring device based on multi-sensor and AI judgment may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the driving safety monitoring device based on multi-sensor and AI judgment are also stored in RAM1004. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the driving safety monitoring device based on multi-sensor and AI judgment to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a driving safety monitoring device based on multi-sensor and AI judgment with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or provided alternatively.

[0185] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0186] The driving safety monitoring device based on multi-sensors and AI judgment provided by the present application adopts the driving safety monitoring method based on multi-sensors and AI judgment in the above-mentioned embodiment, which can solve the technical problems of how to improve the accuracy and real-time performance of driving safety monitoring and enhance the privacy and security protection of user data. Compared with the prior art, the beneficial effects of the driving safety monitoring device based on multi-sensors and AI judgment provided by the present application are the same as the beneficial effects of the driving safety monitoring method based on multi-sensors and AI judgment provided by the above-mentioned embodiment, and the other technical features of the driving safety monitoring device based on multi-sensors and AI judgment are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0187] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0188] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0189] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the driving safety monitoring method based on multiple sensors and AI judgment in the above-mentioned embodiment.

[0190] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0191] The above-mentioned computer-readable storage medium may be included in the driving safety monitoring device based on multiple sensors and AI judgment; or it may exist independently without being assembled into the driving safety monitoring device based on multiple sensors and AI judgment.

[0192] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the driving safety monitoring device based on multi-sensors and AI judgment, the driving safety monitoring device based on multi-sensors and AI judgment: collects hardware information, and the hardware information includes alcohol concentration and driver physiological information; processes the hardware information through a preset algorithm to obtain a processing result; when the processing result is an abnormal state, analyzes the driver's sobriety through voice interaction to obtain an analysis result; based on the processing result and / or the analysis result, obtains a danger level assessment result and takes corresponding processing measures to achieve driving safety monitoring.

[0193] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0194] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0195] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0196] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned driving safety monitoring method based on multiple sensors and AI judgment, and can solve the technical problems of how to improve the accuracy and real-time performance of driving safety monitoring and enhance user data privacy and security protection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the driving safety monitoring method based on multiple sensors and AI judgment provided in the above-mentioned embodiments, and will not be repeated here.

[0197] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned driving safety monitoring method based on multiple sensors and AI judgment.

[0198] The computer program product provided by this application can solve the technical problem of how to improve the accuracy and real-time performance of driving safety monitoring and enhance the privacy and security protection of user data. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the driving safety monitoring method based on multi-sensor and AI judgment provided by the above embodiment, which will not be elaborated here.

[0199] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A driving safety monitoring method based on multi-sensor and AI judgment, characterized in that: The method comprises: Collecting hardware information, wherein the hardware information includes alcohol concentration and driver physiological information; Processing the hardware information by a preset algorithm to obtain a processing result; When the processing result is an abnormal state, analyzing the driver's sobriety level through voice interaction to obtain an analysis result; According to the processing result and / or the analysis result, a danger level assessment result is obtained and corresponding processing measures are taken to achieve driving safety monitoring.

2. The method according to claim 1, characterized in that The step of processing the hardware information by a preset algorithm to obtain a processing result comprises: Acquiring alcohol concentration and driver physiological information from the hardware information; Based on a preset algorithm, the alcohol concentration is compared with an alcohol concentration threshold to obtain a first comparison result; Based on the preset algorithm, the driver's physiological information is compared with normal physiological information to obtain a second comparison result; When the first comparison result is that the driver has not drunk alcohol and the second comparison result is that the driver's physiological state is normal, determining that the processing result is a normal state; When the first comparison result is driving under the influence of alcohol or drunk driving, or the second comparison result is that the driver's physiological state is abnormal, the processing result is determined to be an abnormal state.

3. The method according to claim 1, characterized in that When the processing result is an abnormal state, the step of analyzing the driver's sobriety through voice interaction to obtain the analysis result includes: When the processing result is an abnormal state, initiating a question-and-answer interaction through voice to obtain question-and-answer data; Analyze the driver's reaction time, language clarity, and logical judgment ability based on the question and answer data; When the reaction time, language clarity and logical judgment ability meet the preset standards, the analysis result is determined to be that the driver is conscious; When the reaction time, language clarity and logical judgment ability do not meet the preset standards, it is determined that the analysis result is that the driver is not conscious.

4. The method according to claim 1, characterized in that The step of obtaining a hazard level assessment result and taking corresponding treatment measures according to the processing result and / or the analysis result comprises: According to the processing result and / or the analysis result, evaluating the danger level of the driver's state; When the processing result is that the state is normal, determining the danger level is no danger; When the processing result is that the state is abnormal, obtaining a first comparison result; When the analysis result shows that the driver is conscious and the first comparison result shows that the driver is driving under the influence of alcohol, the danger level is determined to be a slight danger and a danger warning is issued; When the analysis result is that the driver is not conscious or the first comparison result is drunk driving, the danger level is determined to be severe danger, and the vehicle is controlled to be locked and enter emergency contact mode.

5. The method according to claim 4, characterized in that When the analysis result shows that the driver is unconscious, determining that the danger level is severe danger, and controlling the vehicle to lock and enter the emergency contact mode, the following steps are included: When the vehicle enters the emergency contact mode, automatically select to contact an emergency contact and / or a designated driver platform; When choosing to contact the designated driving platform, record the audio and video data of the designated driving process and monitor the safety status in the car; When the safety condition in the vehicle is abnormal, an alarm is automatically sounded and the audio and video data is transmitted to the backend server.

6. The method according to claim 1, characterized in that The step of collecting hardware information through sensors, wherein the hardware information includes alcohol concentration and driver physiological information, comprises: The driver's facial expression, eye state and head posture are collected through the camera; collecting the alcohol concentration in the driver's exhaled gas by means of an alcohol concentration sensor; Collecting the driver's heart rate, pulse and blood pressure through a biometric information sensor; The facial expression, the eye state, the head posture, the heart rate, the pulse and the blood pressure are used as the driver's physiological information.

7. The method according to any one of claims 1 to 6, characterized in that The step of obtaining a danger level assessment result and taking corresponding treatment measures according to the processing result and / or the analysis result to realize intelligent monitoring of driving safety includes: When the processing result is an abnormal state, obtaining question and answer data of the voice interaction; Encrypting the hardware information and / or the question-and-answer data to obtain encrypted information, and storing the encrypted information locally; In an emergency, the encrypted information is uploaded to the backend server.

8. A driving safety monitoring device based on multiple sensors and AI judgment, characterized in that: The device comprises: An information collection module, used to collect hardware information through sensors, wherein the hardware information includes alcohol concentration and driver physiological information; A data processing module, used to process the hardware information through a preset algorithm to obtain a processing result; A state analysis module, used to analyze the driver's sobriety through voice interaction to obtain an analysis result when the processing result is an abnormal state; The evaluation control module is used to obtain a danger level evaluation result and take corresponding processing measures according to the processing result and / or the analysis result to achieve driving safety monitoring.

9. A driving safety monitoring device based on multiple sensors and AI judgment, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the driving safety monitoring method based on multi-sensors and AI judgment as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the driving safety monitoring method based on multi-sensors and AI judgment as described in any one of claims 1 to 7 are implemented.

Images