Driver state detection method, device, equipment and storage medium

Abstract

The application provides a driving state detection method, device, equipment and storage medium. The method comprises the following steps: acquiring driving scene information of a vehicle at a current time and a current face image frame of a driver in the vehicle; the driving scene information is used to represent an environment where the vehicle is located and a driving state of the vehicle; determining an influence factor corresponding to the driving scene information according to the driving scene information; the influence factor represents a factor influencing a driving state parameter of the driver; the driving state parameter represents a number of closed-eye frames of the driver within a preset judgment time period; determining the driving state parameter of the driver at the current time according to the influence factor; if it is determined that the number of closed-eye frames of the current face image frame within the preset judgment time period is greater than or equal to the number of closed-eye frames of the driving state parameter of the driver at the current time, it is determined that the driver is in a fatigue state, and a prompt information is sent to the driver. The method provided by the application improves the driving state detection precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a driver state detection method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of intelligent driving technology, people have higher and higher requirements for intelligent driving, and pay particular attention to driving safety. When a driver drives a vehicle, it is easy to feel tired, and even fall asleep while driving. Therefore, the state of the driver needs to be detected, and the driver in a tired state needs to be reminded in time.

[0003] In the prior art, the method for judging the fatigue state of a driver mainly judges based on the number of blinks and other facial features of the driver. If the number of blinks of the driver exceeds a preset number threshold, it can be considered that the driver is tired. However, this way cannot accurately detect the state of the driver, and seriously affects the personal safety of the driver. SUMMARY

[0004] The present application provides a driver state detection method, device, equipment and storage medium to improve the state detection accuracy of the driver.

[0005] In a first aspect, the present application provides a driver state detection method, comprising:

[0006] obtaining driving scene information of a vehicle at a current time and a current face image frame of a driver in the vehicle; wherein the driving scene information is used to represent the environment where the vehicle is located and the driving state of the vehicle;

[0007] determining an influence factor corresponding to the driving scene information according to the driving scene information; wherein the influence factor represents a factor that affects a driver state parameter; the driver state parameter represents the number of closed-eye frames of the driver in a preset judgment time period;

[0008] determining the driver state parameter of the driver at the current time according to the influence factor;

[0009] if the number of closed-eye frames of the current face image frame in the preset judgment time period is greater than or equal to the number of closed-eye frames indicated by the driver state parameter at the current time, it is determined that the driver is in a tired state, and a prompt information is sent to the driver.

[0010] In a second aspect, the present application provides a driver state detection device, comprising:

[0011] An information obtaining module is configured to obtain driving scene information of a vehicle at a current time and a current face image frame of a driver in the vehicle, wherein the driving scene information is used to represent an environment where the vehicle is located and a driving state of the vehicle.

[0012] A factor determining module is configured to determine an influence factor corresponding to the driving scene information according to the driving scene information, wherein the influence factor represents a factor that influences a driver state parameter, and the driver state parameter represents a number of closed-eye frames of the driver within a preset judgment time period.

[0013] A parameter determining module is configured to determine the driver state parameter of the driver at the current time according to the influence factor.

[0014] A state determining module is configured to determine that the driver is in a fatigue state and send a prompt information to the driver if it is determined that the number of closed-eye frames of the current face image frame within the preset judgment time period is greater than or equal to the number of closed-eye frames indicated by the driver state parameter at the current time.

[0015] In a third aspect, the present application provides an electronic device, comprising a processor and a memory connected with the processor in communication;

[0016] The memory stores computer execution instructions.

[0017] The processor executes the computer execution instructions stored in the memory to implement the driver state detection method according to the first aspect.

[0018] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the driver state detection method according to the first aspect.

[0019] In a fifth aspect, the present application provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the driver state detection method according to the first aspect.

[0020] This application provides a method, apparatus, device, and storage medium for detecting driver state. During driving, it acquires driving scene information and the driver's current facial image frames. Different driving scene information corresponds to different influencing factors, which are determined based on the acquired driving scene information. The influencing factors are those that affect driver state parameters, which are the number of closed-eye frames within a preset judgment time period when the driver is fatigued. Based on the influencing factors, the driver's state parameters at the current time are determined. Different driver state parameters are used to judge the driver state in different driving scenarios, enabling the driver state parameters to adapt to changes. The number of closed-eye frames within a certain period is acquired; if this number is greater than or equal to the number of closed-eye frames indicated by the driver state parameters, driver fatigue is determined, and the driver is alerted. This solves the problem in existing technologies where targeted judgments cannot be made for different scenarios, effectively improving the accuracy of driver state judgment and ensuring driving safety. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 A schematic flowchart illustrating a driver state detection method provided in an embodiment of this application;

[0023] Figure 2 A schematic flowchart illustrating a driver state detection method provided in an embodiment of this application;

[0024] Figure 3 A schematic flowchart illustrating a driver state detection method provided in an embodiment of this application;

[0025] Figure 4 A structural block diagram of a driver state detection device provided in an embodiment of this application;

[0026] Figure 5 A structural block diagram of a driver state detection device provided in an embodiment of this application;

[0027] Figure 6 A structural block diagram of an electronic device provided in an embodiment of this application;

[0028] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0029] The specific embodiments of the application have been shown by way of example in the above figures, and will be described in more detail hereafter. These figures and written description are not meant to limit the scope of the inventive concept in any way but merely to illustrate the inventive concept to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0030] For the purpose of clarity, technical solutions and advantages of the present application will be further described in detail below with reference to the accompanying drawings.

[0031] It should be noted that the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0032] The following description refers to the accompanying drawings. Unless otherwise indicated, same or similar elements in different drawings are denoted by same or similar reference numerals. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0033] In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not necessarily mean a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise specified. "And / or", which describes the relationship between the associated objects, means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0034] It should be noted that due to the limitation of the length of the specification, all optional embodiments cannot be enumerated in the present application. Those skilled in the art should be able to think of any combination of technical features as long as the technical features do not contradict each other, which can constitute an optional embodiment. The embodiments will be described in detail below.

[0035] When a driver is driving for a long distance or at night, the driver often feels tired, and the driver needs to be reminded to adjust the state in time when the driver is tired to ensure driving safety. Therefore, the state of the driver can be determined during the driving process of the driver to determine whether the driver is in a fatigue state. In the current field of intelligent driving, the method for detecting the state of the driver mainly determines the state based on the eye opening degree value, the number of blinks or the number of yawns of the driver and other facial expression features. For example, the number of blinks is used to determine the state, and if the number of blinks of the driver exceeds a preset number threshold, it is considered that the driver is in a fatigue state.

[0036] In the related technology for detecting the state of the driver, a fatigue determination standard suitable for the public is usually found as the number threshold under a large number of samples, but this standard cannot accurately cover all drivers and cannot cover the determination of the eye opening degree value and the blink frequency of the driver at different times and in different driving scenes. For example, in the case of backlight or strong light, the eye opening degree and the pupil of the driver will instinctively decrease; from 23:00 at night to 5:00 in the morning, the blink frequency of the driver will be high, and if the previous standard is continued to be used for determination at this time, the false alarm of the fatigue state of the driver is easy to occur.

[0037] That is, the preset fatigue determination standard is a fixed standard value, and in fact, because of individual physiological differences, the blink frequency and the eye opening degree value of the driver in different scenes are different, and the fixed standard cannot completely match the individual physiological differences of the driver in different scenes, the accuracy of the detection of the state of the driver is low, and the driving safety is affected.

[0038] The detection method for the state of the driver provided in the present application aims to solve the above technical problems of the prior art.

[0039] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0040] Figure 1 is a flowchart of a detection method for a state of a driver according to an embodiment of the present application. The method provided in the present embodiment can be executed by a detection device for a state of a driver, as shown in Figure 1 The method comprises the following steps:

[0041] S101, acquiring driving scene information of a vehicle at a current time and a current face image frame of a driver in the vehicle; wherein the driving scene information is used to represent the environment where the vehicle is located and the driving state of the vehicle.

[0042] Exemplarily, an information collection device can be installed in the vehicle to collect driving scene information when the vehicle is driven. For example, the information collection device can be an image collection device, such as a camera, and a sensor, such as a rain and light sensor and an acceleration sensor. The driving scene information can be used to represent the environment in which the vehicle is located and the driving state of the vehicle, for example, the driving scene information can include environmental information such as temperature, light intensity and weather of the environment in which the vehicle is located, and driving state information such as vehicle speed, acceleration, gear position, hand brake state and hand torque. For vehicles with intelligent driving function, the driving scene information can also include lane deviation alarm times and lane correction control times. The lane deviation alarm times are the number of alarms when the vehicle deviates from the target lane, and the target lane can be the original lane of the vehicle or the lane planned in the navigation route. The lane correction control times are the number of times the vehicle automatically returns to the target lane when the vehicle deviates from the lane.

[0043] When the driver drives the vehicle, the information collection device on the vehicle can collect driving scene information in real time or at a fixed time. The driving scene information of the vehicle at the current time can be obtained, and the current time can be the current time or a period of time before the current time. For example, the driving scene information within five minutes before the current time can be obtained every five minutes as the driving scene information at the current time. The driving scene information can be obtained by the vehicle terminal or by the cloud of the vehicle.

[0044] An image collection device is installed on the vehicle, which can be used to collect real-time or fixed-time face image frames of the driver in the vehicle. The face image frame can represent the facial features of the driver, for example, the facial features can be the eye opening value of the human eye, and the eye opening value can be the distance between the upper and lower eyelids of the human eye. After the face image frame is collected, the face image frame of the driver in the vehicle at the current time can be obtained in real time or at a fixed time as the current face image frame. The current face image frame can be at least one frame.

[0045] S102, according to the driving scene information, determine the influence factor corresponding to the driving scene information; wherein the influence factor represents the factor affecting the driver state parameter; the driver state parameter represents the number of closed-eye frames of the driver in the preset judgment period.

[0046] Exemplarily, the corresponding relationship between the driving scene information and the influence factor is set in advance, for example, the conversion rule between the driving scene information and the influence factor can be preset. After obtaining the driving scene information, the driving scene information is converted into the influence factor according to the preset conversion rule. For example, the driving scene information can be multiplied by a preset percentage to obtain the influence factor.

[0047] The influence factor can be used to represent a factor influencing a driver state parameter, the driver state parameter being used to represent a number of closed-eye frames of the driver within a preset judgment time period, the number of closed-eye frames of the driver within the preset judgment time period being a minimum value of the number of closed-eye frames of the driver within the preset judgment time when the driver is in a fatigue state. In the fatigue state, the number of eye blinks of a human eye is relatively large, and the closed-eye duration is relatively long. That is, if the actual number of closed-eye frames of the driver within the preset judgment time exceeds the number of closed-eye frames indicated by the driver state parameter, it is considered that the driver is in the fatigue state.

[0048] The influence factor influences the determination of the number of closed-eye frames in the driver state parameter, and the influence factor can represent factors such as time, light, and temperature. For example, for the influence factor representing time, the influence factor values are different at different times, so that the number of closed-eye frames in the driving state parameter is different at different times, that is, the judgment standard of the driver fatigue is different at different times. The driver is more likely to feel fatigue at night, and therefore, the number of closed-eye frames in the driving state parameter in the morning can be less than the number of closed-eye frames in the driving state parameter at night.

[0049] In S103, the driver state parameter of the driver at the current time is determined according to the influence factor.

[0050] Exemplarily, the influence factor can influence the number of closed-eye frames in the driver state parameter. For example, the influence factor represents the influence of the light factor, when the light is strong, the number of eye blinks of a human eye is more frequent, and the number of closed-eye frames in the driver state parameter can be increased to realize adaptive adjustment of the driver state parameter and improve the accuracy of the driver state judgment.

[0051] The relationship between the influence factor and the driver state parameter is preset. A calculation formula between the influence factor and the number of closed-eye frames indicated by the driver state parameter can be preset to represent the relationship between the influence factor and the driver state parameter. The association relationship between the influence factor and the driver state parameter can also be preset. For example, a plurality of driver state parameters are preset, and the value range of the influence factor is associated with each driver state parameter. After the influence factor is determined, the value range in which the value of the influence factor is located is obtained, and according to the preset association relationship, the driver state parameter corresponding to the value range is searched.

[0052] A plurality of influence factors can be determined according to driving scene information, and the driver state parameter is determined by the plurality of influence factors. For example, a first influence factor is determined according to time information, a second influence factor is determined according to light information, and a third influence factor is determined according to vehicle speed information. The first influence factor, the second influence factor, and the third influence factor are multiplied, and then multiplied by a preset fixed value of the number of closed-eye frames to obtain the number of closed-eye frames in the driver state parameter.

[0053] S104, if it is determined that the number of closed-eye frames of the current face image frame in the preset judgment time period is greater than or equal to the number of closed-eye frames indicated by the driver state parameter at the current time, it is determined that the driver is in a fatigue state, and a prompt information is sent to the driver.

[0054] For example, the state of the driver can include a fatigue state and a non-fatigue state, and the number of closed-eye frames in the driver state parameter is a judgment standard for determining whether the driver is in a fatigue state. In the process of driving, the current face image frame at the current time can be obtained in real time or at a fixed time. The face image frame in the preset judgment time period before the current time can be obtained as the current face image frame. For example, the preset judgment time period is one minute, and the face image frame in the one minute before the current time can be obtained. The closed-eye frames are obtained from the current face image frame to obtain the number of closed-eye frames. The closed-eye frames can be determined according to a preset image recognition algorithm. In this embodiment, the image recognition algorithm is not limited.

[0055] The number of closed-eye frames in the preset judgment time period at the current time and the driver state parameter are determined, and the number of closed-eye frames is compared with the number of closed-eye frames indicated by the driver state parameter at the current time. If the number of closed-eye frames in the preset judgment time period at the current time is greater than or equal to the number of closed-eye frames indicated by the driver state parameter at the current time, it is considered that the driver has a large number of current closed-eye times or a long closed-eye time, and the driver is currently in a fatigue state. A prompt information can be sent to the driver to remind the driver to adjust the state. For example, the vehicle terminal can send a prompt information, or a mobile terminal such as a mobile phone bound to the driver can send a prompt information.

[0056] In this embodiment, before determining the number of closed-eye frames of the current face image frame in the preset judgment time period, it further includes: determining the driver eye opening threshold corresponding to the current time as the current eye opening threshold according to the association relationship between the preset time and the driver eye opening threshold; determining the driver eye opening value in the current face image frame in the preset judgment time period; if the driver eye opening value in the current face image frame is less than the current eye opening threshold, the current face image frame is determined as a closed-eye frame.

[0057] Specifically, at different times, the number of eye blinks of the human eye is different, and the eye opening value is also different. For example, in the morning light environment, the driver blinks more, the eye opening value is smaller, and the human eye is usually in a squinting state. The driver eye opening threshold in different environments is set in advance, and the face image frame with an eye opening value less than the driver eye opening threshold is a closed-eye frame.

[0058] The current time is determined, and the current facial image frame collected in the preset judgment section at the current time is acquired. The light intensity in the current environment can also be determined according to a light intensity sensor and the like. A preset correlation between different times and the driver eye opening threshold value can also be preset, and a preset correlation between different light intensities and the driver eye opening threshold value can also be preset. According to the preset correlation between the time and the driver eye opening threshold value, the driver eye opening threshold value corresponding to the current time is determined as the current eye opening threshold value. According to the correlation between the light intensity and the driver eye opening threshold value, the driver eye opening threshold value corresponding to the current light intensity is determined as the current eye opening threshold value.

[0059] For example, a preset correlation between the ambient light intensity and the driver eye opening threshold value is set. In a weak light environment, the upper and lower eyelid opening degree of 2.5 mm can be determined as the driver closing eyes; in a strong light environment, the upper and lower eyelid opening degree can be opened to 1.5 mm, and the driver closing eyes is determined.

[0060] According to the preset image recognition algorithm, the driver eye opening value in each current facial image frame is determined. The driver eye opening value is compared with the current eye opening threshold value to determine whether the driver eye opening value is less than the current eye opening threshold value. If yes, the current facial image frame of the driver eye opening value is determined as the closed eye frame, and the number of closed eye frames of the current facial image frame in the preset judgment time period is obtained.

[0061] The beneficial effect of such a setting is that the driver's eye opening value is different in different environments. By setting different driver eye opening threshold values, it can be avoided that the physiological closing of the driver's eyes is mistaken for fatigue, and the accuracy of the driver state judgment is improved.

[0062] The embodiment of the present application provides a driver state detection method. Driver scene information and a current facial image frame of a driver are acquired during driving of the driver. Different driver scene information corresponds to different influence factors. The corresponding influence factor is determined according to the acquired driver scene information. The influence factor is a factor that influences the driver state parameter. The driver state parameter is the number of closed eye frames of the driver in a preset judgment time period when the driver is tired. The driver state parameter of the driver at the current time is determined according to the influence factor. In different driving scenes, different driver state parameters are used for driver state judgment, so that the driver state parameter can be adaptively changed. The number of closed eye frames of the driver in a period of time is acquired. If the number of closed eye frames is greater than or equal to the number of closed eye frames indicated by the driver state parameter, it is determined that the driver is tired, and the driver is reminded. The problem that the present technology cannot make targeted judgment in each scene is solved, the accuracy of the driver state judgment is effectively improved, and the driving safety is ensured.

[0063] Figure 2This is a flowchart illustrating a method for detecting driver status provided in this application. This embodiment is an optional embodiment based on the above embodiment.

[0064] In this embodiment, determining the influencing factor corresponding to the driving scenario information based on the driving scenario information can be further refined as follows: determining the influencing factor corresponding to the driving scenario information based on the driving scenario information and the preset correlation between the driving scenario information and the influencing factor.

[0065] like Figure 2 As shown, the method includes the following steps:

[0066] S201. Obtain the driving scene information of the vehicle at the current time and the current facial image frame of the driver in the vehicle; wherein, the driving scene information is used to represent the environment in which the vehicle is located and the driving status of the vehicle.

[0067] For example, this step can be referred to step S101, and will not be described again.

[0068] S202. Based on the driving scenario information and the preset correlation between the driving scenario information and the influencing factors, determine the influencing factors corresponding to the driving scenario information.

[0069] For example, the relationship between driving scenario information and influencing factors can be pre-defined. Multiple driving scenario information entries can exist, each corresponding to a specific influencing factor. For instance, driving scenario information might include time information, weather information, and vehicle speed information. Time information corresponds to a time influencing factor, weather information to a weather influencing factor, and vehicle speed information to a vehicle speed influencing factor. The relationship between each driving scenario information entry and its corresponding influencing factor value can be pre-defined. For example, the preset values ​​for the time influencing factor could be 0.2, 0.3, and 0.4. For example, if the time information is 8:00 to 9:00, the corresponding time influencing factor value is 0.2; if the time information is 15:00 to 16:00, the corresponding time influencing factor value is 0.3; and if the time information is 20:00 to 21:00, the corresponding time influencing factor value is 0.4.

[0070] After obtaining the driving scenario information, the influencing factors corresponding to the driving scenario information are determined based on the preset correlation between the driving scenario information and the influencing factors. The determined influencing factors include the type of influencing factor and the value of the influencing factor. For example, if the driving scenario information includes the time information 8:30 and the weather information sunny, the influencing factor for the time information 8:30 can be determined to be the time influencing factor, and the value of the influencing factor is 0.2.

[0071] In this embodiment, the driving scene information includes time information, manual driving information and automatic driving information; according to the driving scene information and a preset correlation between the driving scene information and the influence factors, the influence factors corresponding to the driving scene information are determined, including: according to the time information and a preset correlation between the time information and the time influence factors, the time influence factors corresponding to the time information are determined; according to the manual driving information and a preset correlation between the manual driving information and the manual driving influence factors, the manual driving influence factors corresponding to the manual driving information are determined; and according to the automatic driving information and a preset correlation between the automatic driving information and the automatic driving influence factors, the automatic driving influence factors corresponding to the automatic driving information are determined.

[0072] Specifically, the driving scene information can include time information, manual driving information and automatic driving information. The time information is current time or driving time information; the manual driving information is information of manual operation of the vehicle by the driver, for example, information of gear shifting and speed change; and the automatic driving information is information of intelligent driving of the vehicle, for example, information of automatic lane correction and automatic parking of the vehicle.

[0073] The influence factors can include time influence factors, manual driving influence factors and automatic driving influence factors, the influence factor associated with the time information is the time influence factor, the influence factor associated with the manual driving information is the manual driving influence factor, and the influence factor associated with the automatic driving information is the automatic driving influence factor.

[0074] The correlation between the time information and the time influence factors is preset, after the time information in the driving scene information is obtained, the time influence factors corresponding to the obtained time information are determined according to the correlation between the time information and the time influence factors. For example, the time influence factors corresponding to different time periods can be preset, the time period in which the obtained time information is located is determined, and the value of the time influence factor associated with the time period is determined.

[0075] The correlation between the manual driving information and the manual driving influence factors is preset, after the manual driving information in the driving scene information is obtained, the manual driving influence factors corresponding to the obtained manual driving information are determined according to the correlation between the manual driving information and the manual driving influence factors. For example, the manual driving information is vehicle speed, the manual driving influence factors corresponding to different speed ranges can be preset, the speed range in which the obtained vehicle speed is located is determined, and the value of the manual driving influence factor associated with the speed range is determined.

[0076] The association between the automatic driving information and the automatic driving influence factor is preset, and after the automatic driving information in the driving scene information is obtained, the automatic driving influence factor corresponding to the obtained automatic driving information is determined according to the association between the automatic driving information and the automatic driving influence factor. For example, the automatic driving information is the number of lane deviation alarm times, and the automatic driving influence factor corresponding to different lane deviation alarm times can be preset, the number of lane deviation alarm times in the obtained automatic driving information is determined, and the value of the automatic driving influence factor associated with the number of lane deviation alarm times is determined.

[0077] The beneficial effect of such setting is that according to the driving scene information, multiple influence factors can be obtained, ensuring that various factors are comprehensively considered when judging the driver state, and improving the accuracy of driver state detection. Through the preset association, the value of the influence factor can be quickly obtained, and the efficiency of driver state detection is improved.

[0078] In this embodiment, the time information includes driving duration and driving time; according to the time information and the preset association between the time information and the time influence factor, the time influence factor corresponding to the time information is determined, including: according to the driving duration and the preset association between the duration and the duration influence factor, the duration influence factor corresponding to the driving duration is determined; according to the driving time and the preset association between the time and the time influence factor, the time influence factor corresponding to the driving time is determined; according to the duration influence factor and the time influence factor, the time influence factor corresponding to the time information is obtained.

[0079] Specifically, the time information can include driving duration and driving time, the driving duration is the duration of driving after the driver starts the vehicle this time, and the driving time is the current time point. For example, the driver starts driving at 9 am, stops at 9:30 am, and maintains the driving state from 10 am, and the current driving time is 11 am, then the driving duration is 1 hour, not including the half hour from 9 am to 9:30 am.

[0080] A shutdown duration can be preset, and if the duration from the shutdown to the restart of the driver exceeds the shutdown duration, it is considered that the driver starts a new start. For example, the preset shutdown duration is 15 minutes, and if the driver restarts within 15 minutes after shutdown, the driving duration before shutdown needs to be calculated when determining the driving duration. When the gear is in a non-forward gear or the electronic handbrake is pulled up for more than 15 minutes, the system considers that the driver has obtained rest, clears the duration influence factor, and performs the operation according to the initial vehicle ignition, and the driving duration starts to count from 0.

[0081] The time influence factor can include a time length influence factor and a time point influence factor. The time length influence factor and the time point influence factor are respectively associated with a time length and a time point. After obtaining the driving time length, the value of the time length influence factor is determined according to the association between the time length and the time length influence factor; after obtaining the driving time point, the value of the time point influence factor is determined according to the association between the time point and the time point influence factor. For example, when the driving time length is within 0-2 hours, the time length influence factor is 1; when the driving time length is within 2-4 hours, the time length influence factor decreases from 1 to 0.95; when the driving time length is more than 4 hours, the time length influence factor decreases to 0.9; when the driving time point is within the time period of 23:00-05:00, the time point influence factor is 0.95; and when the driving time point is other time points, the time point influence factor is 1.

[0082] According to the time length influence factor and the time point influence factor, a total time influence factor can be obtained as the time influence factor corresponding to the time information. The calculation formula of the time influence factor can be preset, for example, the time influence factor can be equal to the product of the time length influence factor and the time point influence factor.

[0083] The beneficial effect of such setting is that the influence of time on the driver state is comprehensively considered through the driving time length and the driving time point, which is beneficial to adaptively adjusting the driver state parameter according to the time information, and effectively improves the flexibility and accuracy of the driver state detection.

[0084] In this embodiment, the manual driving information includes vehicle speed and gear position; according to the manual driving information and the preset association between the manual driving information and the manual driving influence factor, the manual driving influence factor corresponding to the manual driving information is determined, including: according to the vehicle speed and the preset association between the vehicle speed and the vehicle speed influence factor, the vehicle speed influence factor corresponding to the vehicle speed is determined; according to the gear position and the preset association between the gear position and the gear position influence factor, the gear position influence factor corresponding to the gear position is determined; and according to the vehicle speed influence factor and the gear position influence factor, the manual driving influence factor corresponding to the manual driving information is obtained.

[0085] Specifically, the manual driving information can include vehicle speed, gear position, and manual torque, etc. The vehicle speed can be the speed of the vehicle at the current time, and the gear position can be the gear position of the vehicle at the current time and the duration of the gear position. For example, the current speed of the vehicle is 60 kilometers per hour, the gear position is forward gear, and the forward gear has been maintained for 30 minutes. The vehicle speed can also represent the change value of the speed within a period of time before the current time, for example, the speed at the current time and the speed one minute before the current time can be obtained to obtain the change value of the vehicle speed within one minute.

[0086] The manual driving influence factors can include a vehicle speed influence factor, a gear influence factor, a hand torque influence factor, etc. The association between the vehicle speed and the vehicle speed influence factor, the association between the gear and the gear influence factor, and the association between the hand torque and the hand torque influence factor are pre-set. After obtaining the vehicle speed, the value of the vehicle speed influence factor is determined according to the association between the vehicle speed and the vehicle speed influence factor; after obtaining the gear, the value of the gear influence factor is determined according to the association between the gear and the gear influence factor; and after obtaining the hand torque, the value of the hand torque influence factor is determined according to the association between the hand torque and the hand torque influence factor. For example, in the non-automatic driving mode, when the vehicle speed is higher than 60 kph (kilometers per hour), if the absolute value of the vehicle speed changes less than 3 kph within 1 minute, the vehicle speed influence factor is 0.95; and when the hand torque of the driver is less than 3.5 Nm (Newton meter) for 5 seconds, or the absolute value of the change of the hand torque of the driver exceeds 6 Nm within 2 seconds, the hand torque influence factor can be changed to 0.95.

[0087] According to the vehicle speed influence factor, the gear influence factor, and the hand torque influence factor, a total manual driving influence factor can be obtained as the manual driving influence factor corresponding to the manual driving information. The calculation formula of the manual driving influence factor can be pre-set. For example, when the manual driving influence factor includes the vehicle speed influence factor and the gear influence factor, the manual driving influence factor can be equal to the product of the vehicle speed influence factor and the gear influence factor.

[0088] The beneficial effects of such setting are that the influence of the manual driving information on the driver state is comprehensively considered, which is conducive to the adaptive adjustment of the driver state parameter according to the manual driving information, and effectively improves the flexibility and accuracy of the driver state detection.

[0089] In this embodiment, the automatic driving information includes the number of lane deviation alarm and the number of lane correction control in a preset time period; and the automatic driving influence factor corresponding to the automatic driving information is determined according to the automatic driving information and the pre-set association between the automatic driving information and the automatic driving influence factor, including: the automatic driving influence factor corresponding to the automatic driving information is determined according to the number of lane deviation alarm and / or the number of lane correction control and the pre-set association between the number and the automatic driving influence factor.

[0090] Specifically, the automatic driving information can include the number of lane deviation alarm and the number of lane correction control, etc. In the adaptive cruise driving mode, the vehicle can detect whether the lane is deviated, and automatically corrects when the lane is deviated.

[0091] The preset association relationship between the automatic driving information and the automatic driving influence factor is, for example, an association relationship between the number of lane deviation alarm and the automatic driving influence factor, and an association relationship between the number of lane deviation control and the automatic driving influence factor. The number of lane deviation alarm and the number of lane deviation control at the current time are determined, and the value of the automatic driving influence factor is determined according to the association relationship between the automatic driving information and the automatic driving influence factor. For example, if the number of lane deviation alarm is more than 3 times or the number of lane deviation control is more than 3 times within one minute, the automatic driving influence factor is 0.95.

[0092] The beneficial effect of such setting is that the influence of the automatic driving information on the driver state is comprehensively considered, which is beneficial to adaptively adjusting the driver state parameter according to the automatic driving information, and effectively improves the flexibility and accuracy of the driver state detection.

[0093] S203, determining the driver state parameter of the driver at the current time according to the influence factor.

[0094] For example, this step can refer to step S103, which will not be repeated here.

[0095] S204, if it is determined that the number of closed-eye frames of the current face image frame within the preset judgment time period is greater than or equal to the number of closed-eye frames indicated by the driver state parameter at the current time, it is determined that the driver is in a fatigue state, and a prompt information is sent to the driver.

[0096] For example, this step can refer to step S104, which will not be repeated here.

[0097] The embodiment of the present application provides a driver state detection method. The driving scene information and the current face image frame of the driver are obtained during driving of the driver. Different driving scene information corresponds to different influence factors. The corresponding influence factor is determined according to the obtained driving scene information. The influence factor is a factor that influences the driver state parameter. The number of closed-eye frames within a preset judgment time period is the driver state parameter when the driver is tired. The driver state parameter of the driver at the current time is determined according to the influence factor. In different driving scenes, different driver state parameters are used for driver state judgment, so that the driver state parameter can be adaptively changed. The number of closed-eye frames of the driver within a period of time is obtained. If the number of closed-eye frames is greater than or equal to the number of closed-eye frames indicated by the driver state parameter, it is determined that the driver is tired, and the driver is reminded. The problem that the prior art cannot make targeted judgment in each scene is solved, the accuracy of driver state judgment is effectively improved, and driving safety is ensured.

[0098] Figure 3A flowchart of a driver state detection method provided by an embodiment of the present application is shown in the figure. The embodiment is an optional embodiment based on the above embodiment.

[0099] In the embodiment, before acquiring the driving scene information of the vehicle at the current time, the following can be additionally performed: in response to a vehicle starting operation of the driver, determining the number of closed-eye frames in the driver face image frames in a preset initial time period; and determining the driver state parameter in the initial time period according to the number of closed-eye frames in the preset initial time period.

[0100] As shown in the figure, the method comprises the following steps: Figure 3

[0101] S301, in response to a vehicle starting operation of the driver, determining the number of closed-eye frames in the driver face image frames in a preset initial time period.

[0102] For example, the driver gets into the vehicle and makes a vehicle starting operation, for example, the vehicle starting operation can be a starting operation. In response to the vehicle starting operation of the driver, it is determined that the vehicle is woken up. A preset initial time period is set, for example, the initial time period can be three minutes. Driver face image frames in the initial time period after the vehicle starts are acquired, for example, driver face image frames in the three minutes after the vehicle starts are acquired. An image frame number threshold can also be set, and driver face image frames of the image frame number threshold are acquired after the vehicle starts, for example, the first 7200 frames of driver face image frames are acquired. Image recognition is performed on the acquired driver face image frames to obtain the number of closed-eye frames in the driver face image frames.

[0103] In the embodiment, in response to the vehicle starting operation of the driver, the number of closed-eye frames in the driver face image frames in the preset initial time period is determined, which comprises: in response to the vehicle starting operation of the driver, acquiring the starting time of the vehicle, determining the driver eye opening threshold corresponding to the starting time as the initial eye opening threshold; acquiring the driver face image frames in the initial time period, determining the driver eye opening values of the driver face image frames in the initial time period; and according to the driver eye opening values of the driver face image frames in the initial time period and the initial eye opening threshold, determining the number of closed-eye frames in the driver face image frames in the preset initial time period.

[0104] ​Specifically, in response to a vehicle start operation of the driver, the vehicle is started, and a start time of the vehicle is determined. A correlation between a time and a driver eye opening threshold value is preset, and the driver eye opening threshold value associated with the start time is determined as an initial eye opening threshold value. An initial light intensity of an environment when the vehicle is started can also be obtained, a correlation between the light intensity and the driver eye opening threshold value is preset, and the driver eye opening threshold value associated with the initial light intensity is determined as the initial eye opening threshold value, so that the eye opening threshold value of the closed-eye frame can be adjusted in real time according to the current light intensity.

[0105] The driver face image frames in an initial time period after the vehicle is started are obtained, and eye opening values in each driver face image frame are determined. The eye opening values are compared with the initial eye opening threshold value. If the eye opening value is less than the initial eye opening threshold value, the driver face image frame is a closed-eye frame. If the eye opening value is equal to or greater than the initial eye opening threshold value, the driver face image frame is an open-eye frame. The number of closed-eye frames in the driver face image frames in the preset initial time period is determined.

[0106] The beneficial effect of such a setting is that when the user starts to start the vehicle, the initial eye opening threshold value is first determined, the number of closed-eye frames of the driver in a non-fatigue state is obtained, and the number of closed-eye frames of the driver in a fatigue state is determined. When the driver starts the vehicle at different times or in different environments, the initial number of closed-eye frames is different, the comprehensive consideration of time and environment is realized, the number of closed-eye frames of the current driver is self-learned and collected within three minutes after the vehicle is started, and the accuracy of subsequent driver state judgment is improved.

[0107] S302, according to the number of closed-eye frames in the preset initial time period, determine the driver state parameter in the initial time period.

[0108] Exemplarily, when the vehicle is started, the driver is not yet tired, the number of closed-eye frames in the initial time period is the number of closed-eye frames of the driver in a non-fatigue state, a correlation between the number of closed-eye frames in a non-fatigue state and the number of closed-eye frames in a fatigue state is preset, and the number of closed-eye frames of the driver in a fatigue state in the initial time period is obtained according to the number of closed-eye frames in the driver face image frames in the initial time period. The number of closed-eye frames in a fatigue state in the initial time period is the driver state parameter in the initial time period.

[0109] A default driver state parameter is preset, and if the number of closed-eye frames of the driver in the initial time period cannot be obtained, the default driver state parameter is determined as the driver state parameter in the initial time period. For example, after the driver starts the vehicle, the driver covers the camera with his hand, and the camera cannot collect the driver face image frame, so the default driver state parameter can be used to determine the driver state parameter in the initial time period.

[0110] In this embodiment, the driver state parameter in the initial time period is determined according to the number of closed-eye frames in the initial time period, including: determining the driver state parameter in the initial time period according to the number of closed-eye frames in the driver face image frame in the initial time period and the preset driver state judgment coefficient.

[0111] Specifically, a driver state judgment coefficient can be preset, and after obtaining the number of closed-eye frames in the driver face image frame in the initial time period, the number of closed-eye frames can be multiplied by the preset driver state judgment coefficient to obtain the driver state parameter in the initial time period. The number of closed-eye frames when tired is greater than the number of closed-eye frames when not tired, therefore, the driver state judgment coefficient can be greater than 1. For example, the driver state judgment coefficient is 1.5, and the number of closed-eye frames in the driver face image frame in the initial time period is 320 frames, then the number of closed-eye frames in the initial time period when tired is 480 frames, that is, the number of closed-eye frames indicated by the driver state parameter in the initial time period is 480 frames.

[0112] The beneficial effect of such setting is that through the preset driver state judgment coefficient, the number of closed-eye frames of the driver in the tired state can be inferred, which is convenient for subsequent determination of the driver state parameter of the driver at different times.

[0113] S303, obtaining driving scene information of the vehicle at the current time and a current face image frame of the driver in the vehicle; wherein the driving scene information is used to represent the environment where the vehicle is located and the driving state of the vehicle.

[0114] Exemplarily, this step can refer to step S101, and will not be repeated here.

[0115] S304, determining an influence factor corresponding to the driving scene information according to the driving scene information; wherein the influence factor represents a factor affecting the driver state parameter; and the driver state parameter represents the number of closed-eye frames of the driver in the preset judgment time period.

[0116] Exemplarily, this step can refer to step S102, and will not be repeated here.

[0117] S305, determining the driver state parameter of the driver at the current time according to the influence factor.

[0118] Exemplarily, the driver state parameter of the driver at the current time can be determined according to the time influence factor, the manual driving influence factor and the automatic driving influence factor at the current time.

[0119] In the embodiment, the driver state parameter of the driver at the current time is determined according to the influence factor, including: determining the driver state parameter of the driver at the current time according to the driver state parameter in the initial time period and the influence factor.

[0120] Specifically, the driver state parameter in the initial time period and the influence factor of the current time are determined, the driver state parameter in the initial time period and the influence factor of the current time are multiplied to obtain the driver state parameter of the driver at the current time. For example, the driver state parameter in the initial time period is multiplied by the time influence factor, the manual driving influence factor and the automatic driving influence factor at the current time to obtain the closed-eye frame number of the fatigue state of the driver at the current time, and the closed-eye frame number of the fatigue state is the driver state parameter.

[0121] The beneficial effect of such setting is that the driver state parameter at the current time can be determined according to the initial driver state parameter. When detecting the state of different drivers, the initial driver state parameter used is not the same. The initial time period of the driver state parameter of each driver is determined when the driver starts the vehicle, reducing the physiological differences of different individuals. When determining the driver state parameter of the driver at the current time, the driver state parameter in the initial time period of the driver can be determined according to the driver, realizing the targeted state detection of each driver and improving the state detection accuracy.

[0122] S306, if the closed-eye frame number of the current face image frame in the preset judgment time period is greater than or equal to the closed-eye frame number indicated by the driver state parameter at the current time, it is determined that the driver is in a fatigue state, and a prompt information is sent to the driver.

[0123] Exemplarily, Xiaoming drives the vehicle for 3 hours from 9:00, at 12:00, the hand torque value is 3 Nm for 8 seconds, the vehicle speed is maintained at 75 kph within 1 minute, and 3 lane deviation warnings are triggered during the period. It can be determined that the driving time of 3 hours corresponds to a time influence factor of 0.975, and the current driving time of 12:00 corresponds to a time influence factor of 1, and the total time influence factor is 0.975*1=0.975. The hand torque is 3 Nm for 8 seconds, and the hand torque influence factor is 0.95. The vehicle speed is maintained at 75 kph within 1 minute, and the vehicle speed influence factor is 0.95. The total manual driving influence factor is 0.95*0.95=0.9025. Within 1 minute, 3 lane deviation warnings are triggered, and the automatic driving influence factor is 0.95. The number of eye-closed frames indicated by the driver state parameter in the initial time period is 480 frames, so the driver state parameter at the current time is 480*0.975*0.9025*0.95=401, and the minimum number of eye-closed frames per minute in the driver fatigue state is 401 frames. At this time, Xiaoming's cumulative eye-closed frame number in a 1-minute time period exceeds 401 frames, so a fatigue warning prompt message can be sent.

[0124] If Xiaoming receives the prompt message and stops for a break for half an hour, the weather turns cloudy, and the light sensor reports that it is a weak light environment at this time. The eye opening threshold of the driver in the weak light environment is 2.5 mm. Within ten minutes of restarting the vehicle, Xiaoming's hand position blocks the camera, so the default driver state parameter of 430 frames is used as the driver state parameter in the initial time period. If the influence factors at the current time are all 1 during driving, the driver state parameter at the current time is 430 frames. At this time, Xiaoming's cumulative eye-closed frame number in a 1-minute time period is 350 frames, which does not exceed 430 frames, so no prompt message is sent.

[0125] The application provides a driver state detection method. Driver scene information and a current face image frame of the driver are obtained during driving of the driver. Different driving scene information corresponds to different influence factors. The influence factor corresponding to the obtained driving scene information is determined. The influence factor is a factor that influences the driver state parameter. The driver state parameter is the number of eye-closed frames of the driver in a preset judgment time period when the driver is fatigued. The driver state parameter of the driver at the current time is determined according to the influence factor. In different driving scenes, different driver state parameters are used to judge the driver state, so that the driver state parameter can be adaptively changed. The number of eye-closed frames of the driver in a period of time is obtained. If the number of eye-closed frames is greater than or equal to the number of eye-closed frames indicated by the driver state parameter, it is determined that the driver is fatigued, and the driver is reminded. The problem that the prior art cannot make targeted judgments in various scenes is solved, the accuracy of driver state judgment is effectively improved, and driving safety is ensured.

[0126] Figure 4 A structural block diagram of a driver state detection device is provided for an embodiment of the present application. For ease of illustration, only parts related to the embodiments of the present disclosure are shown. Referring to Figure 4 , the device comprises an information acquisition module 401, a factor determination module 402, a parameter determination module 403, and a state determination module 404.

[0127] The information acquisition module 401 is configured to acquire driving scene information of a vehicle at a current time and a current face image frame of a driver in the vehicle; wherein the driving scene information is used to represent the environment where the vehicle is located and the driving state of the vehicle;

[0128] The factor determination module 402 is configured to determine an influence factor corresponding to the driving scene information according to the driving scene information; wherein the influence factor represents a factor that affects the driver state parameter; and the driver state parameter represents the number of closed-eye frames of the driver within a preset judgment time period.

[0129] The parameter determination module 403 is configured to determine the driver state parameter of the driver at the current time according to the influence factor.

[0130] The state determination module 404 is configured to determine that the driver is in a fatigue state and send a prompt information to the driver if it is determined that the number of closed-eye frames of the current face image frame within the preset judgment time period is greater than or equal to the number of closed-eye frames indicated by the driver state parameter at the current time.

[0131] In one example, the factor determination module 402 is specifically configured to:

[0132] determine the influence factor corresponding to the driving scene information according to the driving scene information and a preset association relationship between the driving scene information and the influence factor.

[0133] The embodiment of the present application provides a kind of driver state detection device, obtain the driving scene information and the current face image frame of driver in the process that driver drives vehicle, different driving scene information corresponds to different influence factor, according to the driving scene information obtained to determine corresponding influence factor.Influence factor is the factor that influences driver state parameter, and driver state parameter is the number of eye-closing frame in the preset judgment period when driver is tired.According to influence factor, determine the driver state parameter of driver at current time.In different driving scene, different driver state parameter is used to judge driver state, realize that driver state parameter can be self-adapting change.The number of eye-closing frame of driver in a period of time is obtained, if the number of eye-closing frame is greater than or equal to the number of eye-closing frame indicated by driver state parameter, then determine that driver is tired, and the driver is reminded.The problem that cannot be judged in each scene in prior art is solved, the accuracy of driver state judgment is effectively improved, and driving safety is guaranteed.

[0134] Figure 5 The structure block diagram of a kind of driver state detection device provided by the embodiment of the present application is shown in Figure 4 As shown in the embodiment, driving scene information includes time information, manual driving information and automatic driving information. Figure 5

[0135] In one example, factor determination module 402 includes:

[0136] First factor determination unit 4021 is used to determine the time influence factor corresponding to the time information according to the time information and the association relationship between preset time information and time influence factor;

[0137] Second factor determination unit 4022 is used to determine the manual driving influence factor corresponding to the manual driving information according to the manual driving information and the association relationship between preset manual driving information and manual driving influence factor;

[0138] Third factor determination unit 4023 is used to determine the automatic driving influence factor corresponding to the automatic driving information according to the automatic driving information and the association relationship between preset automatic driving information and automatic driving influence factor.

[0139] In one example, time information includes driving duration and driving time;

[0140] First factor determination unit 4021 is specifically used to:

[0141] Determine the time influence factor corresponding to the driving duration according to the driving duration and the association relationship between preset duration and duration influence factor;

[0142] ​According to the driving time and a preset association between a time and a time influence factor, a time influence factor corresponding to the driving time is determined;

[0143] According to the time length influence factor and the time influence factor, a time influence factor corresponding to the time information is obtained.

[0144] In one example, the manual driving information includes a vehicle speed and a gear position;

[0145] The second factor determination unit 4022 is specifically configured to:

[0146] According to the vehicle speed and a preset association between a vehicle speed and a vehicle speed influence factor, a vehicle speed influence factor corresponding to the vehicle speed is determined;

[0147] According to the gear position and a preset association between a gear position and a gear position influence factor, a gear position influence factor corresponding to the gear position is determined;

[0148] According to the vehicle speed influence factor and the gear position influence factor, a manual driving influence factor corresponding to the manual driving information is obtained.

[0149] In one example, the automatic driving information includes a number of lane deviation alarm times and a number of lane deviation control times in a preset time period;

[0150] The third factor determination unit 4023 is specifically configured to:

[0151] According to the automatic driving information and a preset association between automatic driving information and an automatic driving influence factor, an automatic driving influence factor corresponding to the automatic driving information is determined, including:

[0152] According to the number of lane deviation alarm times and / or the number of lane deviation control times and a preset association between a number and an automatic driving influence factor, an automatic driving influence factor corresponding to the automatic driving information is determined.

[0153] In one example, the device further includes:

[0154] The threshold determination module is configured to, before determining the number of closed-eye frames of the current face image frame in a preset judgment time period, determine a driver eye opening threshold value corresponding to a current time as a current eye opening threshold value according to a preset association between a time and a driver eye opening threshold value.

[0155] The opening value determination module is configured to determine a driver eye opening value in the current face image frame in a preset judgment time period.

[0156] The closed-eye frame determination module is configured to determine the current face image frame as a closed-eye frame if the driver eye opening value in the current face image frame is less than the current eye opening threshold value.

[0157] In one example, the apparatus further includes:

[0158] The number determination module is configured to determine the number of closed-eye frames in the driver face image frames in the preset initial time period in response to a vehicle starting operation of the driver before acquiring the driving scene information of the vehicle at the current time.

[0159] The initial parameter determination module is configured to determine the driver state parameter in the initial time period according to the number of closed-eye frames in the preset initial time period.

[0160] In one example, the number determination module is specifically configured to:

[0161] In response to a vehicle starting operation of the driver, the starting time of the vehicle is acquired, and the driver eye opening threshold value corresponding to the starting time is determined as an initial eye opening threshold value.

[0162] The driver face image frames in the initial time period are acquired, and the driver eye opening values of the driver face image frames in the initial time period are determined.

[0163] The number of closed-eye frames in the driver face image frames in the preset initial time period is determined according to the driver eye opening values of the driver face image frames in the initial time period and the initial eye opening threshold value.

[0164] In one example, the initial parameter determination module is specifically configured to:

[0165] The driver state parameter in the initial time period is determined according to the number of closed-eye frames in the driver face image frames in the preset initial time period and a preset driver state judgment coefficient.

[0166] In one example, the parameter determination module 403 is specifically configured to:

[0167] The driver state parameter of the driver at the current time is determined according to the driver state parameter in the initial time period and the influence factor.

[0168] Figure 6 A structural block diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1. Figure 6 As shown in FIG. 1, the electronic device includes a memory 61, a processor 62, a memory 61, and a memory for storing executable instructions of the processor 62.

[0169] The processor 62 is configured to perform the method provided by the above-described embodiments.

[0170] The electronic device further includes a receiver 63 and a transmitter 64. The receiver 63 is configured to receive instructions and data transmitted from other devices, and the transmitter 64 is configured to transmit instructions and data to external devices.

[0171] Figure 7 is a block diagram of an electronic device according to an exemplary embodiment. The device can be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, a vehicle, etc.

[0172] The device 700 can include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0173] The processing component 702 usually controls overall operations of the device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 can include one or more processors 720 to execute instructions to complete all or part of steps of the above-described methods. In addition, the processing component 702 can include one or more modules to facilitate interaction between the processing component 702 and other components. For example, the processing component 702 can include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0174] The memory 704 is configured to store various types of data to support operations of the device 700. Examples of these data include instructions for any application or method operating on the device 700, contact data, phonebook data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0175] The power supply component 706 supplies power for various components of the device 700. The power supply component 706 can include a power supply management system, one or more power sources, and other components associated with generating, managing and distributing power for the device 700.

[0176] The multimedia component 708 includes a screen providing an output interface between the device 700 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the device 700 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0177] The audio component 710 is configured to output and / or input an audio signal. For example, the audio component 710 includes a microphone (MIC) configured to receive an external audio signal when the device 700 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 further includes a speaker for outputting an audio signal.

[0178] The I / O interface 712 provides an interface between the processing component 702 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0179] The sensor component 714 includes one or more sensors to provide various state assessments for the device 700. For example, the sensor component 714 can detect an open / closed state of the device 700, relative positioning of components, such as a display and a keypad of the device 700, a change in position of the device 700 or a component of the device 700, presence or absence of user contact with the device 700, a change in orientation of the device 700 or acceleration / deceleration of the device 700, and a temperature change of the device 700. The sensor component 714 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 714 can further include a light sensor, such as a CMOS or CCD image sensor, for use in an imaging application. In some embodiments, the sensor component 714 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0180] The communication component 716 is configured to facilitate wired or wireless communication between the device 700 and other devices. The device 700 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-WideBand (UWB) technology, Bluetooth (BT) technology and other technologies.

[0181] In an exemplary embodiment, the device 700 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic elements, for executing the above-described methods.

[0182] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 704 including instructions, is also provided, which can be executed by the processor 720 of the device 700 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0183] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a terminal device, enables the terminal device to perform the above-described method for detecting a driver state of the terminal device.

[0184] The present application also discloses a computer program product, including a computer program, which, when executed by a processor, implements the method as described in the present embodiment.

[0185] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip systems (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0186] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or electronic device.

[0187] In the context of the present application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0188] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0189] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0190] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system, and solves the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server, or VPS for short) services. The server can also be a distributed system server or a server combined with a blockchain. It should be understood that the various forms of procedures shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved, and this document does not limit this.

[0191] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0192] It is to be understood that the application is not limited to the precise construction herein disclosed and shown in the drawings, and that various changes in shape, size and arrangements of parts can be made without departing from the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A method of detecting a state of a driver, characterized by, The method comprises: obtaining driving scene information of a vehicle at a current time and a current face image frame of a driver in the vehicle; wherein the driving scene information is used to represent an environment where the vehicle is located and a driving state of the vehicle; determining an influence factor corresponding to the driving scene information according to the driving scene information and a preset association relationship between driving scene information and influence factors; wherein the driving scene information comprises time information, manual driving information and automatic driving information, the automatic driving information comprises a lane deviation alarm frequency and a lane deviation correction control frequency in a preset time period; the influence factor represents a factor influencing a driver state parameter; the driver state parameter represents a number of eye-closed frames of the driver in a preset judgment time period; the number of eye-closed frames of the driver in the preset judgment time period refers to a minimum value of the number of eye-closed frames of the driver in a preset judgment time when the driver is in a fatigue state; determining the driver state parameter of the driver at the current time according to the influence factor; if it is determined that the number of eye-closed frames of the current face image frame in the preset judgment time period is greater than or equal to the number of eye-closed frames indicated by the driver state parameter at the current time, it is determined that the driver is in a fatigue state, and a prompt information is sent to the driver; determining the influence factor corresponding to the driving scene information according to the driving scene information and a preset association relationship between driving scene information and influence factors comprises: determining a time influence factor corresponding to the time information according to the time information and a preset association relationship between time information and time influence factors; determining a manual driving influence factor corresponding to the manual driving information according to the manual driving information and a preset association relationship between manual driving information and manual driving influence factors; determining an automatic driving influence factor corresponding to the automatic driving information according to the lane deviation alarm frequency and / or the lane deviation correction control frequency and a preset association relationship between the frequency and the automatic driving influence factor.

2. The method of claim 1, wherein, The time information comprises a driving duration and a driving time; determining the time influence factor corresponding to the time information according to the time information and a preset association relationship between time information and time influence factors comprises: determining a duration influence factor corresponding to the driving duration according to the driving duration and a preset association relationship between duration and duration influence factors; determining a time influence factor corresponding to the driving time according to the driving time and a preset association relationship between time and time influence factors; determining the time influence factor corresponding to the time information according to the duration influence factor and the time influence factor.

3. The method of claim 1, wherein, The manual driving information comprises a vehicle speed and a gear position; determining the manual driving influence factor corresponding to the manual driving information according to the manual driving information and a preset association relationship between manual driving information and manual driving influence factors comprises: determining a vehicle speed influence factor corresponding to the vehicle speed according to the vehicle speed and a preset association relationship between vehicle speed and vehicle speed influence factors; According to the gear and a preset correlation between the gear and a gear influencing factor, a gear influencing factor corresponding to the gear is determined; According to the vehicle speed influencing factor and the gear influencing factor, a manual driving influencing factor corresponding to the manual driving information is obtained.

4. The method of claim 1, wherein, Before determining the number of closed-eye frames of the current face image frame within a preset judgment time period, the method further comprises: According to a preset correlation between time and a driver eye opening threshold, a driver eye opening threshold corresponding to the current time is determined as a current eye opening threshold; A driver eye opening value in the current face image frame within the preset judgment time period is determined; If the driver eye opening value in the current face image frame is less than the current eye opening threshold, the current face image frame is determined as a closed-eye frame.

5. The method according to any one of claims 1 to 4, characterized in that, Before obtaining the driving scene information of the vehicle at the current time, the method further comprises: In response to a vehicle starting operation of the driver, the number of closed-eye frames in the driver face image frame within a preset initial time period is determined; According to the number of closed-eye frames within the preset initial time period, a driver state parameter within the initial time period is determined.

6. The method of claim 5, wherein, In response to a vehicle starting operation of the driver, the number of closed-eye frames in the driver face image frame within a preset initial time period is determined, comprising: In response to a vehicle starting operation of the driver, a starting time of the vehicle is obtained, a driver eye opening threshold corresponding to the starting time is determined as an initial eye opening threshold; The driver face image frame within the initial time period is obtained, and a driver eye opening value of the driver face image frame within the initial time period is determined; According to the driver eye opening value of the driver face image frame within the initial time period and the initial eye opening threshold, the number of closed-eye frames in the driver face image frame within the preset initial time period is determined.

7. The method of claim 5, wherein, According to the number of closed-eye frames within the preset initial time period, a driver state parameter within the initial time period is determined, comprising: According to the number of closed-eye frames in the driver face image frame within the preset initial time period and a preset driver state judgment coefficient, a driver state parameter within the initial time period is determined.

8. The method of claim 5, wherein, According to the influencing factor, the driver state parameter of the driver at the current time is determined, comprising: According to the driver state parameter within the initial time period and the influencing factor, the driver state parameter of the driver at the current time is determined.

9. A driver state detection device characterized by comprising: Comprise: An information acquisition module is configured to obtain driving scene information of a vehicle at a current time and a current face image frame of a driver in the vehicle; wherein the driving scene information is used to represent the environment and the driving state of the vehicle; a factor determination module, configured to determine an influence factor corresponding to the driving scene information according to the driving scene information and a preset association relationship between driving scene information and influence factors; wherein the driving scene information comprises time information, manual driving information and automatic driving information, the automatic driving information comprises a number of lane deviation alarm times and a number of lane deviation control times in a preset time period; the influence factor represents a factor influencing a driver state parameter; the driver state parameter represents a number of eye-closed frames of the driver in a preset judgment time period; a parameter determination module, configured to determine the driver state parameter of the driver at a current time according to the influence factor; a state determination module, configured to determine that the driver is in a fatigue state and send a prompt information to the driver if it is determined that the number of eye-closed frames of the current face image frame in the preset judgment time period is greater than or equal to the number of eye-closed frames indicated by the driver state parameter at the current time. The factor determination module is specifically configured to determine a time influence factor corresponding to the time information according to the time information and a preset association relationship between time information and time influence factors; determine a manual driving influence factor corresponding to the manual driving information according to the manual driving information and a preset association relationship between manual driving information and manual driving influence factors; and determine an automatic driving influence factor corresponding to the automatic driving information according to the number of lane deviation alarm times and / or the number of lane deviation control times and a preset association relationship between times and automatic driving influence factors.

10. An electronic device, comprising: comprise: a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the driver state detection method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the driver state detection method in any one of claims 1-8.

12. A computer program product, characterised in that, comprise a computer program, which is executed by the processor to implement the driver state detection method in any one of claims 1-8.

Images