Driver state detection method and device, medium and state detector

Abstract

The embodiment of the invention provides a driver state detection method and device, a medium and a state detector. If the vehicle is in the driving state, collecting a face image of a driver; determining a first state of the driver according to the face image, the first state being a mental state of the driver; acquiring control information of a steering wheel, and confirming a second state of the driver according to the control information; the attention state of the driver is confirmed according to the first state and the second state, at the moment, integral consideration is carried out according to the second state and the first state, interaction of the first state and the second state is compatible, control is carried out in multiple dimensions, the accuracy of the attention state of the driver is guaranteed, and the user experience is improved. Therefore, the vehicle equipment can be adjusted according to the attention state, the safety performance of the driver in the driving process is guaranteed, and the emergency processing capacity of the driver in the driving process is improved.

Description

Technical Field

[0001] The present application relates to the technical field of state detectors, and in particular to a driver's state detection method, device, medium and state detector. Background Art

[0002] With the development of science and technology, drivers drive vehicles and drive them on roads, and the driver's attention state during driving has gradually attracted people's attention. In the existing technology, the vehicle will define the attention state according to the driver's facial image, but it is not enough to define it only by the driver's facial image, which will affect the accuracy of the attention state. At the same time, the vehicle cannot be improved based on the driver's attention state. Summary of the invention

[0003] The embodiments of the present application provide a driver's state detection method, device, medium and state detector. If a vehicle is in a driving state, a facial image of the driver is collected; a first state of the driver is determined based on the facial image, wherein the first state is the driver's mental state; control information of the steering wheel is obtained, and the second state of the driver is confirmed based on the control information; the driver's attention state is confirmed based on the first state and the second state. At this time, the second state and the first state are considered as a whole, and the interaction between the first state and the second state is compatible, so as to facilitate control in multiple dimensions, thereby ensuring the accuracy of the driver's attention state, so as to adjust the vehicle equipment according to the attention state, thereby improving the driver's attention state and vehicle equipment, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a method for detecting a driver's state is provided, comprising:

[0006] If the vehicle is in motion, collect the driver's facial image;

[0007] Determining a first state of the driver according to the facial image, wherein the first state is a mental state of the driver;

[0008] Acquiring control information of the steering wheel, and confirming the second state of the driver according to the control information;

[0009] confirming the driver's attention state according to the first state and the second state;

[0010] Vehicle equipment is adjusted based on the attention state.

[0011] Optionally, adjusting vehicle equipment according to the attention state includes:

[0012] Based on the attention state, a braking response rate of the braking system is adjusted.

[0013] Optionally, if the vehicle is in a driving state, collecting a facial image of the driver includes:

[0014] Collect multiple driving parameters of the vehicle;

[0015] defining a first driving characteristic based on a plurality of driving parameters of the vehicle;

[0016] Acquire an interior image of the vehicle, and determine a driver's area based on the interior image of the vehicle;

[0017] defining a second driving characteristic based on the identification of the region of the driver;

[0018] determining whether the driver is in a driving scenario based on the second driving characteristic and the first driving characteristic;

[0019] When the driver is in a driving scene, dynamic shooting is performed based on the driver's area to dynamically capture the driver's facial image;

[0020] Outputting a preprocessed face image based on the preprocessing of the face image;

[0021] An extraction method is defined according to the preprocessed face image and the face outer contour, and a plurality of facial feature points are extracted based on the extraction method.

[0022] Optionally, determining the first state of the driver according to the facial image, wherein the first state is the mental state of the driver, includes:

[0023] Obtain multiple facial feature points;

[0024] Defining corresponding relative positions based on a plurality of facial feature points;

[0025] Defining distribution types of the plurality of facial feature points based on relative positions of the plurality of facial feature points, and constructing a plurality of facial features of the driver according to the distribution types of the plurality of facial feature points and density coefficients of the plurality of facial feature points;

[0026] Combining multiple facial features, outputting multiple facial feature combinations, and defining corresponding combination coefficients for the facial feature combinations;

[0027] The first state of the driver is determined according to a plurality of combination coefficients and a state association table, wherein the first state is the mental state of the driver, which is a mild fatigue state, a severe fatigue state, a mild distraction state or a severe distraction state.

[0028] Optionally, acquiring control information of the steering wheel and confirming the second state of the driver according to the control information includes:

[0029] Monitor the driver's interaction with the steering wheel;

[0030] Defining the interaction type between the driver and the steering wheel according to the interaction actions between the driver and the steering wheel;

[0031] Defining control information of the steering wheel based on the interaction type between the driver and the steering wheel and the dynamic coefficient of the steering wheel, and collecting the control information of the steering wheel;

[0032] The second state of the driver is determined according to the control information of the steering wheel and the driving operation of the driver.

[0033] Optionally, confirming the driver's attention state according to the first state and the second state includes:

[0034] Dynamically matching the second state with the first state;

[0035] Defining a matching coefficient based on the second state and the dynamic matching of the first state;

[0036] The matching level is defined according to the matching coefficient and the space coefficient of the driving space;

[0037] The driver's attention state is determined based on the matching level, the second state, and the first state.

[0038] Optionally, adjusting vehicle equipment according to the attention state includes:

[0039] Match the corresponding state signal based on the attention state;

[0040] The status signal is transmitted to the screen, audio player, and brake respectively, and the responses of the screen, audio player, and brake are triggered;

[0041] Determine the association of the status signal with the screen, audio player, and brake respectively according to the responses of the screen, audio player, and brake, and dynamically adjust the association level of the status signal;

[0042] If the correlation level of the status signal reaches a preset level, the screen, audio player, and brakes are dynamically controlled based on the attention state to adjust the vehicle equipment.

[0043] According to one aspect of an embodiment of the present application, a driver's state detection device is provided, comprising:

[0044] A collection module, used to collect the driver's face image if the vehicle is in a moving state;

[0045] A first state module, configured to determine a first state of the driver according to the face image, wherein the first state is a mental state of the driver;

[0046] A second state module, used to obtain control information of the steering wheel and confirm the second state of the driver according to the control information;

[0047] an attention state module, configured to confirm the driver's attention state according to the first state and the second state;

[0048] An adjustment module is used to adjust vehicle equipment according to the attention state.

[0049] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the driver's state detection method as described in the above embodiment is implemented.

[0050] According to one aspect of an embodiment of the present application, a state detector is provided, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the driver's state detection method as described in the above embodiments.

[0051] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the driver's state detection method provided in the above embodiment.

[0052] In the technical solutions provided in some embodiments of the present application, if the vehicle is in a driving state, a facial image of the driver is collected; a first state of the driver is determined based on the facial image, wherein the first state is the mental state of the driver; control information of the steering wheel is obtained, and the second state of the driver is confirmed based on the control information; the driver's attention state is confirmed based on the first state and the second state. At this time, the second state and the first state are considered as a whole, and the interaction between the first state and the second state is compatible, so as to facilitate control in multiple dimensions, thereby ensuring the accuracy of the driver's attention state, so as to adjust the vehicle equipment according to the attention state, thereby improving the driver's attention state and vehicle equipment, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0053] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0055] Figure 1 A schematic flow chart of a method for detecting a driver's state according to an embodiment of the present application is shown;

[0056] Figure 2 Shows Figure 1 Schematic diagram of the process of S110;

[0057] Figure 3 Shows Figure 1 Schematic diagram of the process of S120;

[0058] Figure 4 Shows Figure 1 Schematic diagram of the process flow of S130;

[0059] Figure 5 Shows Figure 1 Schematic diagram of the process of S140;

[0060] Figure 6 Shows Figure 1 Schematic diagram of the process flow of S150;

[0061] Figure 7A schematic diagram of the architecture of a method for detecting a driver's state according to an embodiment of the present application is shown;

[0062] Figure 8 A block diagram of a driver's state detection device according to an embodiment of the present application is shown;

[0063] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing the state detector of the embodiment of the present application is shown. DETAILED DESCRIPTION

[0064] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0065] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0066] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0067] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0068] Figure 1 A schematic flow chart of a method for detecting a driver's state according to an embodiment of the present application is shown. The method can be applied to a state detector.

[0069] See also Figures 1 to 9 A method for maintaining the status of a portable printer is applied to a scenario of diagnosing the status of a portable printer. The method for maintaining the status of a portable printer comprises:

[0070] Step S110: if the vehicle is in driving state, collecting the driver's face image;

[0071] Step S120: determining a first state of the driver according to the facial image, wherein the first state is a mental state of the driver;

[0072] Step S130: Acquire control information of the steering wheel, and confirm the second state of the driver according to the control information;

[0073] Step S140: confirming the driver's attention state according to the first state and the second state;

[0074] Step S150: Adjust vehicle equipment according to the attention state.

[0075] The adjusting of vehicle equipment according to the attention state includes: adjusting a braking response rate of a braking system based on the attention state.

[0076] In an embodiment of the present invention, through the method in the embodiment of the present invention, if the vehicle is in a driving state, a facial image of the driver is collected; a first state of the driver is determined based on the facial image, wherein the first state is the mental state of the driver; control information of the steering wheel is obtained, and the second state of the driver is confirmed based on the control information; the driver's attention state is confirmed based on the first state and the second state. At this time, the second state and the first state are considered as a whole, and the interaction between the first state and the second state is compatible, so as to facilitate control in multiple dimensions, thereby ensuring the accuracy of the driver's attention state, so as to adjust the vehicle equipment according to the attention state, thereby improving the driver's attention state and vehicle equipment, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0077] refer to Figure 2 , in step S110, if the vehicle is in a driving state, a facial image of the driver is collected;

[0078] In the specific implementation process of the present invention, the specific steps may be:

[0079] S111: Collect multiple driving parameters of the vehicle;

[0080] S112: defining a first driving feature according to a plurality of driving parameters of the vehicle;

[0081] S113: collecting an interior image of the vehicle, and determining a driver's area according to the interior image of the vehicle;

[0082] S114: defining a second driving characteristic based on the identification of the driver's region;

[0083] S115: Determining whether the driver is in a driving scenario based on the second driving feature and the first driving feature;

[0084] S116: When the driver is in a driving scene, dynamically capture the driver's face image based on the driver's area;

[0085] S117: outputting a preprocessed face image based on the preprocessing of the face image;

[0086] S118: defining an extraction method according to the preprocessed face image and the face outer contour, and extracting a plurality of facial feature points based on the extraction method.

[0087] In an embodiment of the present application, multiple driving parameters of the vehicle are collected and further processed to facilitate multi-dimensional control of the multiple driving parameters. At this time, a first driving feature is defined according to the multiple driving parameters of the vehicle, and the first driving feature is defined through multi-dimensional calculation of the multiple driving parameters, thereby ensuring the accuracy of the first driving feature, and then controlling the first driving feature. Optionally, the first driving feature can be the driving state of the vehicle.

[0088] Furthermore, an interior image of the vehicle is collected and further processed to divide the interior image of the vehicle into regions, thereby determining the driver's region based on the interior image of the vehicle, thereby ensuring the accuracy of the driver's region, and then defining a second driving feature based on the identification of the driver's region; optionally, the second driving feature may be the driver's driving state.

[0089] Therefore, whether the driver is in a driving scene is determined based on the second driving characteristic and the first driving characteristic, and is further controlled by the second driving characteristic and the first driving characteristic, so as to define the driver as being in a driving scene in combination with the second driving characteristic and the first driving characteristic.

[0090] When the driver is in a driving scene, dynamic shooting is performed based on the driver's area to dynamically collect the driver's facial image; based on the preprocessing of the facial image, a preprocessed facial image is output; an extraction method is defined according to the preprocessed facial image and the facial outer contour, and the preprocessed facial image and the facial outer contour are controlled in multiple dimensions to ensure the accuracy of the extraction method. At the same time, multiple facial feature points are extracted based on the extraction method, and multiple facial feature points are introduced to facilitate further processing of the multiple facial feature points, thereby ensuring the accuracy of the multiple facial feature points to avoid the influence of blurring factors of the facial image.

[0091] refer to Figure 3 , in step S120, determining a first state of the driver according to the facial image, wherein the first state is a mental state of the driver;

[0092] In the specific implementation process of the present invention, the specific steps may be:

[0093] S121: Acquire multiple facial feature points;

[0094] S122: defining corresponding relative positions based on a plurality of facial feature points;

[0095] S123: defining distribution types of the plurality of facial feature points based on relative positions of the plurality of facial feature points, and constructing a plurality of facial features of the driver according to the distribution types of the plurality of facial feature points and density coefficients of the plurality of facial feature points;

[0096] S124: combining multiple facial features, outputting multiple facial feature combinations, and defining corresponding combination coefficients for the facial feature combinations;

[0097] S125: Determine a first state of the driver according to a plurality of combination coefficients and a state association table, wherein the first state is the mental state of the driver, which is a mild fatigue state, a severe fatigue state, a mild distraction state or a severe distraction state.

[0098] In an embodiment of the present application, multiple facial feature points are obtained to facilitate definition of corresponding relative positions based on the multiple facial feature points, and the relative positions between the multiple facial feature points are introduced, so as to define the distribution types of the multiple facial feature points based on the relative positions of the multiple facial feature points, and multiple facial features of the driver are constructed according to the distribution types of the multiple facial feature points and the density coefficients of the multiple facial feature points, so as to control in multiple dimensions, and the accuracy of the multiple facial features of the driver is ensured based on the distribution types of the multiple facial feature points and the density coefficients of the multiple facial feature points.

[0099] Furthermore, multiple facial features are combined and multiple facial feature combinations are output, which fully utilizes the combination of multiple facial features and presents the interaction between multiple facial features, so that corresponding combination coefficients are defined for the facial feature combination, so as to determine the driver's first state according to the multiple combination coefficients and the state association table, the first state includes a mild fatigue state, a severe fatigue state, a mild distraction state and a severe distraction state, and the state association table has a mapping relationship between the first state and multiple combination coefficients, so that the driver's first state is determined by multiple combination coefficients and the state association table, thereby ensuring the accuracy of the driver's first state.

[0100] refer to Figure 4, in step S130, obtaining control information of the steering wheel, and confirming the second state of the driver according to the control information;

[0101] In the specific implementation process of the present invention, the specific steps may be:

[0102] S131: monitoring the interaction between the driver and the steering wheel;

[0103] S132: defining an interaction type between the driver and the steering wheel according to an interaction action between the driver and the steering wheel;

[0104] S133: defining control information of the steering wheel based on the interaction type between the driver and the steering wheel and the dynamic coefficient of the steering wheel, and collecting the control information of the steering wheel;

[0105] S134: Determine the second state of the driver according to the control information of the steering wheel and the driving operation of the driver.

[0106] In an embodiment of the present application, the interaction actions between the driver and the steering wheel are monitored in real time, and the interaction actions between the driver and the steering wheel are dynamically controlled so that the interaction type between the driver and the steering wheel can be defined according to the interaction actions between the driver and the steering wheel, thereby introducing the interaction type between the driver and the steering wheel, and then controlling the interaction type between the driver and the steering wheel.

[0107] Therefore, the control information of the steering wheel is defined based on the interaction type between the driver and the steering wheel and the dynamic coefficient of the steering wheel, and the control information of the steering wheel is collected to facilitate further control of the control information of the steering wheel.

[0108] At this time, the second state of the driver is determined according to the control information of the steering wheel and the driving operation of the driver, and the second state of the driver presents the degree of dynamic interaction between the driver and the steering wheel, so as to facilitate consideration of the dynamic interaction between the driver and the steering wheel. Optionally, the dynamic interaction degree is the level of interaction between the driver and the steering wheel, which is a deep interaction, a preliminary interaction, and the like.

[0109] refer to Figure 5 , S140: confirming the driver's attention state according to the first state and the second state;

[0110] In the specific implementation process of the present invention, the specific steps may be:

[0111] S141: Dynamically matching the second state with the first state;

[0112] S142: defining a matching coefficient based on dynamic matching of the second state and the first state;

[0113] S143: defining a matching level according to the matching coefficient and the space coefficient of the driving space;

[0114] S144: Determine the driver's attention state based on the matching level, the second state, and the first state.

[0115] In the embodiment of the present application, the second state and the first state are frozen. At this time, the second state and the first state are introduced, and the second state and the first state are associated, so that the second state and the first state can further interact and operate, thereby making full use of the second state and the first state, thereby defining a matching coefficient based on the matching of the second state and the first state, and presenting the matching degree of the second state and the first state based on the matching coefficient. Optionally, the matching degree refers to the mapping degree of the second state and the first state in the driving scene, and is associated with a preset matching parameter table.

[0116] At this time, the matching level is defined according to the matching coefficient and the spatial coefficient of the driving space, so as to control the matching level in the matching coefficient and the spatial coefficient of the driving space, thereby controlling the matching level in multiple dimensions, and then determining the driver's attention state based on the matching level, the second state and the first state, thereby ensuring the accuracy of the attention state.

[0117] refer to Figure 6 , S150: adjusting vehicle equipment according to the attention state;

[0118] In the specific implementation process of the present invention, the specific steps may be:

[0119] S151: Matching a corresponding state signal based on the attention state;

[0120] S152: transmitting the status signal to the screen, the audio player, and the brake respectively, and triggering responses from the screen, the audio player, and the brake;

[0121] S153: Determine the association of the status signal with the screen, the audio player, and the brake respectively according to the responses of the screen, the audio player, and the brake, and dynamically adjust the association level of the status signal;

[0122] S154: If the correlation level of the status signal reaches a preset level, the screen, audio player, and brakes are dynamically regulated based on the attention state to adjust the vehicle equipment.

[0123] In an embodiment of the present application, the attention state is located and further processed. At this time, the corresponding state signal is matched based on the attention state to facilitate interaction with the screen, audio player, and brake based on the form of the state signal.

[0124] At this time, the status signal is transmitted to the screen, audio player, and brake respectively, and the responses of the screen, audio player, and brake are triggered; the association of the status signal with the screen, audio player, and brake is determined according to the responses of the screen, audio player, and brake respectively, and the association level of the status signal is dynamically adjusted to facilitate the control of the association level of the status signal, thereby responding to the dynamic interaction of the screen, audio player, and brake based on the association level of the status signal. Optionally, the attention state is associated with the screen, audio player, and brake, so that the attention state interacts with the screen, audio player, and brake, thereby presenting the interactive effect between the attention state and the screen, audio player, and brake, thereby ensuring the dynamic interaction of the screen, audio player, and brake, and being compatible with the consideration of the driver's attention state. During the specific implementation of the present invention, the driver's attention state is confirmed according to the first state and the second state, so that it is considered as a whole according to the second state and the first state, and the interaction between the first state and the second state is compatible, so as to facilitate control in multiple dimensions, thereby ensuring the accuracy of the driver's attention state. At the same time, the attention state is associated with the screen and audio player; the vehicle equipment is adjusted according to the attention state, so as to adjust the attention state according to the dynamic reminder scene, thereby improving the driver's attention state, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0125] At this time, if the correlation level of the status signal reaches a preset level, the screen and audio player are dynamically regulated based on the attention state to facilitate dynamic adjustment of the screen and audio player.

[0126] Furthermore, a screen reminder mechanism is constructed based on the attention state and the screen; in the screen reminder mechanism, the brightness or gradient color displayed on the screen is matched based on the attention state; at the same time, an audio player reminder mechanism is constructed based on the attention state and the audio player; in the audio player reminder mechanism, the volume or song played by the audio player is matched based on the attention state.

[0127] Therefore, the screen reminder mechanism and the audio player reminder mechanism are associated, and a dynamic reminder scene is formed according to the integration of the screen reminder mechanism and the audio player reminder mechanism, and an interactive space between the screen and the audio player is formed in the dynamic reminder scene; the adjustment amount of the attention state is matched according to the interactive space between the screen and the audio player, and the adjustment of the attention state is triggered according to the adjustment amount of the attention state. Optionally, the dynamic reminder scene is a music scene, a video scene, etc.

[0128] Therefore, the attention state is associated with the screen and audio player; the vehicle equipment is adjusted according to the attention state, so as to facilitate the regulation of the attention state according to the dynamic reminder scene, thereby improving the driver's attention state, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0129] Specific embodiments:

[0130] The DMS system is introduced. The DMS system can monitor the details of the head, eyes, face, hands, etc. in real time through an infrared camera facing the driver. It can detect the driver's status from eye closure, blinking, gaze direction, yawning and head movement. Specifically, the workflow of the DMS system is as follows:

[0131] The DMS system will pre-process the collected facial images to facilitate subsequent feature extraction. The DMS system will extract facial feature points and feature indicators from the pre-processed facial images. These features will be used as the basis for judging the driver's status. The DMS system will match the extracted features with the existing feature library to identify the driver's status. The first status output by the DMS system: mild fatigue status, severe fatigue status, mild distraction status, severe distraction status.

[0132] At this time, the driver's attention state is confirmed according to the first state and the second state, so as to be considered as a whole according to the second state and the first state, and the interaction between the first state and the second state is compatible, so as to facilitate control in multiple dimensions, thereby ensuring the accuracy of the driver's attention state. At the same time, the attention state is associated with the screen and audio player; the vehicle equipment is adjusted according to the attention state, so as to adjust the attention state according to the dynamic reminder scene, thereby improving the driver's attention state, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0133] Attention state Brake pre-fill status for chassis request No distraction Default level Low distraction, Danger Level 1 High distraction Danger Level 2

[0134] After receiving the brake pre-fill signal, the chassis controller automatically adjusts the master cylinder pressure change rate (bar / s) according to the danger content analyzed by the brake pre-fill signal, so that when the driver steps on the brake, the braking system can respond immediately and shorten the braking distance.

[0135] Default level is Level 0, HBA0=12500*15%;

[0136] Danger Level 1 is Level 1, HBA1=HBA0*0.7;

[0137] Danger Level 2 is Level 2, HBA2=HBA0*0.54.

[0138] Furthermore, the vehicle equipment is adjusted according to the attention state so as to adjust the attention state according to the dynamic reminder scene, thereby improving the driver's attention state, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0139] Receive the driver's attention status and increase the brightness of the existing screen by a certain percentage based on different levels. After the alarm ends, restore it to its original state.

[0140] Attention state Screen brightness No distraction constant Low distraction, Brightness increased by 10% High distraction 20% Brightness Increase

[0141] Receive the driver's attention status and increase the volume by a certain percentage based on the existing volume according to different levels. After the alarm ends, restore it to its original state.

[0142] Attention state Loudness No distraction constant Low distraction, 10% loudness increase High distraction 20% louder

[0143] In an embodiment of the present invention, through the method in the embodiment of the present invention, if the vehicle is in a driving state, a facial image of the driver is collected; a first state of the driver is determined based on the facial image, wherein the first state is the mental state of the driver; control information of the steering wheel is obtained, and the second state of the driver is confirmed based on the control information; the driver's attention state is confirmed based on the first state and the second state. At this time, the second state and the first state are considered as a whole, and the interaction between the first state and the second state is compatible, so as to facilitate control in multiple dimensions, thereby ensuring the accuracy of the driver's attention state, so as to adjust the vehicle equipment according to the attention state, thereby improving the driver's attention state and vehicle equipment, ensuring the driver's safety performance during driving, and improving the driver's emergency handling ability during driving.

[0144] The following describes an embodiment of the device of the present application, which can be used to execute the driver's state detection method in the above embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the driver's state detection method in the above embodiment of the present application.

[0145] Figure 8 A block diagram of a driver's state detection device according to an embodiment of the present application is shown.

[0146] Reference Figure 8 As shown, a driver's state detection device according to an embodiment of the present application includes:

[0147] The acquisition module 210 is used to acquire the driver's facial image if the vehicle is in a driving state;

[0148] A first state module 220, configured to determine a first state of the driver according to the face image, wherein the first state is a mental state of the driver;

[0149] The second state module 230 is used to obtain control information of the steering wheel and confirm the second state of the driver according to the control information;

[0150] an attention state module 240, configured to confirm the driver's attention state according to the first state and the second state;

[0151] The adjustment module 250 is used to adjust vehicle equipment according to the attention state.

[0152] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing the state detector of the embodiment of the present application is shown.

[0153] It should be noted that Fig. 9 The computer system of the state detector shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0154] like Fig. 9 As shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0155] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0156] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 309, and / or installed from a removable medium 311. When the computer program is executed by a central processing unit (CPU) 301, various functions defined in the system of the present application are executed.

[0157] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0158] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0159] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0160] As another aspect, the present application also provides a computer-readable medium, which may be included in the state detector described in the above embodiment; or may exist independently without being assembled into the state detector. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a state detector, the state detector implements the method described in the above embodiment.

[0161] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0162] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0163] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0164] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for detecting a driver's state, characterized in that: include: If the vehicle is in motion, collect the driver's facial image; Determining a first state of the driver according to the facial image, wherein the first state is a mental state of the driver; Acquiring control information of the steering wheel, and confirming the second state of the driver according to the control information; confirming the driver's attention state according to the first state and the second state; Vehicle equipment is adjusted based on the attention state.

2. The method according to claim 1, characterized in that: The adjusting vehicle equipment according to the attention state comprises: Based on the attention state, a braking response rate of the braking system is adjusted.

3. The method according to claim 1, characterized in that If the vehicle is in a driving state, collecting the driver's face image includes: Collect multiple driving parameters of the vehicle; defining a first driving characteristic based on a plurality of driving parameters of the vehicle; Acquire an interior image of the vehicle, and determine a driver's area based on the interior image of the vehicle; defining a second driving characteristic based on the identification of the region of the driver; determining whether the driver is in a driving scenario based on the second driving characteristic and the first driving characteristic; When the driver is in a driving scene, dynamic shooting is performed based on the driver's area to dynamically capture the driver's facial image; Outputting a preprocessed face image based on the preprocessing of the face image; An extraction method is defined according to the preprocessed face image and the face outer contour, and a plurality of facial feature points are extracted based on the extraction method.

4. The method according to claim 1, characterized in that The determining the first state of the driver according to the facial image, wherein the first state is the mental state of the driver, comprises: Obtain multiple facial feature points; Defining corresponding relative positions based on a plurality of facial feature points; Defining distribution types of the plurality of facial feature points based on relative positions of the plurality of facial feature points, and constructing a plurality of facial features of the driver according to the distribution types of the plurality of facial feature points and density coefficients of the plurality of facial feature points; Combining multiple facial features, outputting multiple facial feature combinations, and defining corresponding combination coefficients for the facial feature combinations; The first state of the driver is determined according to a plurality of combination coefficients and a state association table, wherein the first state is the mental state of the driver, which is a mild fatigue state, a severe fatigue state, a mild distraction state or a severe distraction state.

5. The method according to claim 1, characterized in that The acquiring control information of the steering wheel and confirming the second state of the driver according to the control information includes: Monitor the driver's interaction with the steering wheel; Defining the interaction type between the driver and the steering wheel according to the interaction actions between the driver and the steering wheel; Defining control information of the steering wheel based on the interaction type between the driver and the steering wheel and the dynamic coefficient of the steering wheel, and collecting the control information of the steering wheel; The second state of the driver is determined according to the control information of the steering wheel and the driving operation of the driver.

6. The method according to claim 1, characterized in that The confirming the driver's attention state according to the first state and the second state includes: Dynamically matching the second state with the first state; Defining a matching coefficient based on the second state and the dynamic matching of the first state; The matching level is defined according to the matching coefficient and the space coefficient of the driving space; The driver's attention state is determined based on the matching level, the second state, and the first state.

7. The method according to claim 5, characterized in that The adjusting vehicle equipment according to the attention state comprises: Match the corresponding state signal based on the attention state; The status signal is transmitted to the screen, audio player, and brake respectively, and the responses of the screen, audio player, and brake are triggered; Determine the association of the status signal with the screen, audio player, and brake respectively according to the responses of the screen, audio player, and brake, and dynamically adjust the association level of the status signal; If the correlation level of the status signal reaches a preset level, the screen, audio player, and brakes are dynamically controlled based on the attention state to adjust the vehicle equipment.

8. A driver status detection device, characterized in that: include: A collection module, used to collect the driver's face image if the vehicle is in a moving state; A first state module, configured to determine a first state of the driver according to the face image, wherein the first state is a mental state of the driver; A second state module, used to obtain control information of the steering wheel and confirm the second state of the driver according to the control information; an attention state module, configured to confirm the driver's attention state according to the first state and the second state; An adjustment module is used to adjust vehicle equipment according to the attention state.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting a driver's state according to any one of claims 1 to 7 is implemented.

10. A state detector, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the driver's state detection method as described in any one of claims 1 to 7.

Images