Driving state detection method and apparatus, electronic device, and readable storage medium

Abstract

Provided in the present disclosure are a driving state detection method and apparatus, an electronic device, and a readable storage medium. The method comprises: acquiring an image to be subjected to detection that includes the head of a driver; detecting a head image of the driver from the image to be subjected to detection, and on the basis of the head image, determining a head pose of the driver; detecting a facial region of the driver from the image to be subjected to detection; determining the number of occluded eye feature points and an eye image from the facial region; on the basis of the number of occluded eye feature points, determining whether the eye image can be used for gaze direction estimation; when the eye image can be used for gaze direction estimation, inputting the eye image into a gaze direction estimation model to determine a gaze direction of the driver; on the basis of the gaze direction, head position information and a head rotation angle of the driver, and position information of each region in a vehicle, determining a gaze region of the driver; and on the basis of the head rotation angle and the gaze region, determining a driving state of the driver. By means of the method, the accuracy of detecting a distracted state is improved.

Description

A driving state detection method and device, electronic equipment and readable storage medium

[0001] Cross-reference to Related Applications

[0002] The present application claims priority to the Chinese patent application No. 202411252748.0, filed on September 9, 2024, and entitled "A driving state detection method and device, electronic equipment and readable storage medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of vehicle driving, and in particular to a driving state detection method and device, electronic equipment and readable storage medium. BACKGROUND

[0004] Distraction detection (also known as driver attention detection) is crucial in vehicle safety driving technology. By performing real-time distraction detection during the driver driving the vehicle, the driving safety of the vehicle can be significantly improved, the traffic accidents caused by the driver's distraction (lack of concentration) can be reduced, and the driving safety can be ensured.

[0005] Currently, the distraction detection usually uses the solvPnP algorithm (an algorithm for estimating the camera pose) to detect whether the driver appears in a distraction state. However, this approach relies on the facial feature point information of the driver. When the horizontal face yaw angle of the driver is too large (for example, the driver looks back), or the vertical face pitch angle of the driver is too large (for example, the driver looks down), so that the camera cannot capture the face image of the driver, the facial feature point information of the driver cannot be recognized, and at this time, the driver cannot be detected for distraction, which undoubtedly increases the safety risk during driving.

[0006] In addition, the distraction detection can also use the gaze direction of the driver to determine whether a distraction state occurs. However, in some cases, for example, when the driver wears sunglasses or the surface of the eyes reflects light, the gaze direction recognition becomes unreliable, which may lead to errors in gaze direction estimation and affect the accuracy of distraction detection.

[0007] SUMMARY

[0008] Therefore, the purpose of the present disclosure is to provide a driving state detection method and device, electronic equipment and readable storage medium to improve the accuracy of detecting the distraction state of the driver.

[0009] In a first aspect, the present disclosure provides a driving state detection method, comprising:

[0010] In the process that the driver drives the vehicle, a to-be-detected image containing the head of the driver at the current time is acquired in real time;

[0011] A head image of the driver is detected from the to-be-detected image, the head image is input into a trained head posture estimation model, and a head posture of the driver is output by the head posture estimation model; the head posture includes a head rotation angle and head position information;

[0012] When the to-be-detected image contains a face region of the driver, the face region of the driver is detected from the to-be-detected image;

[0013] Face feature point position information is determined from the face region, and eye feature point position information is determined from the face feature point position information;

[0014] According to the eye feature point position information, the number of occluded eye feature points is determined; and an eye region is determined from the face region according to the eye feature point position information, so as to determine an eye image according to the eye region;

[0015] According to the number of occluded eye feature points, it is judged whether the eye image can be used for gaze direction estimation;

[0016] When the eye image can be used for gaze direction estimation, the eye image is input into a trained gaze direction estimation model, and the gaze direction of the driver is determined by the gaze direction estimation model;

[0017] According to the gaze direction of the driver, the head position information, the head rotation angle, and the position information of each region in the vehicle, a gaze region of the driver is determined;

[0018] According to the head rotation angle and the gaze region, it is judged whether the driving state of the driver is a distracted driving state.

[0019] With reference to the first aspect, embodiments of the present disclosure provide a first possible implementation manner of the first aspect, and the method further includes:

[0020] When the to-be-detected image does not contain the face region of the driver, it is judged according to the head rotation angle of the driver whether the driving state of the driver is a distracted driving state.

[0021] With reference to the first aspect, embodiments of the present disclosure provide a second possible implementation manner of the first aspect, and the occluded eye feature point is an eye feature point of a single eye; and the judging whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points includes:

[0022] calculating, for each eye, a first ratio of a number of the occluded eye feature points corresponding to the eye to a preset total number of single eye feature points;

[0023] when the first ratio is greater than a first preset threshold, indicating that the eye image corresponding to the eye cannot be used for gaze direction estimation;

[0024] when the first ratio is not greater than the first preset threshold, inputting the eye image corresponding to the eye into the trained glasses-wearing discrimination model to determine whether the eye wears glasses through the glasses-wearing discrimination model;

[0025] if the eye does not wear glasses, indicating that the eye can be used for gaze direction estimation;

[0026] if the eye wears glasses, determining a type of the glasses;

[0027] when the type of the glasses is a light-blocking glasses, indicating that the eye image corresponding to the eye cannot be used for gaze direction estimation;

[0028] when the type of the glasses is a transparent glasses, determining whether a light spot on a lens corresponding to the eye in the glasses meets a condition of affecting gaze direction estimation;

[0029] when the light spot on the lens corresponding to the eye in the glasses meets the condition of affecting gaze direction estimation, indicating that the eye image corresponding to the eye cannot be used for gaze direction estimation; when the light spot on the lens corresponding to the eye in the glasses does not meet the condition of affecting gaze direction estimation, indicating that the eye image corresponding to the eye can be used for gaze direction estimation.

[0030] With reference to the second possible implementation manner of the first aspect, the third possible implementation manner of the first aspect is provided, wherein the determining the type of the glasses if the eye wears glasses comprises:

[0031] determining an inner eyelid region and an outer eyelid region from the eye image of the eye;

[0032] calculating a pixel gray mean value of the inner eyelid region according to a gray value of each pixel point in the inner eyelid region, and calculating a pixel gray mean value of the outer eyelid region according to a gray value of each pixel point in the outer eyelid region;

[0033] calculating a second ratio between the pixel gray mean value of the inner eyelid region and the pixel gray mean value of the outer eyelid region;

[0034] When the second ratio is greater than a second preset threshold, the category of the glasses is determined as sunglasses; and when the second ratio is not greater than the second preset threshold, the category of the glasses is determined as transparent glasses.

[0035] With reference to the second possible implementation manner of the first aspect, the present disclosure provides a fourth possible implementation manner of the first aspect, and when the category of the glasses is transparent glasses, the method further includes:

[0036] When the category of the glasses is transparent glasses, a third ratio of a number of pixel points with a gray value greater than a third preset threshold to a total number of pixel points in the eye image of the eye is calculated according to the gray values of the pixel points in the eye image of the eye.

[0037] If the third ratio is greater than a fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses satisfies the condition of affecting the estimation of the gaze direction.

[0038] If the third ratio is not greater than the fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses does not satisfy the condition of affecting the estimation of the gaze direction.

[0039] With reference to the second possible implementation manner of the first aspect, the present disclosure provides a fifth possible implementation manner of the first aspect, and the gaze direction is the gaze direction of a single eye; and the method further includes:

[0040] For each time between the current time and a target historical time, a first historical eye image of the driver at a first historical time and a second historical eye image of the driver at a second historical time are obtained, the first historical time being the previous time of the current time and the second historical time being the next time of the first historical time.

[0041] For each eye, a first gaze direction of the eye at the first historical time is determined according to the first historical eye image of the eye, and a second gaze direction of the eye at the second historical time is determined according to the second historical eye image of the eye.

[0042] A predicted gaze direction of the eye at the current time is predicted according to the first gaze direction and the second gaze direction.

[0043] calculate an included angle between the predicted line-of-sight direction of the eye at the moment and the line-of-sight direction of the eye at the moment, retain the eye image corresponding to the eye of the driver at the moment when the included angle is not greater than a third preset angle, and delete the eye image corresponding to the eye of the driver at the moment when the included angle is greater than the third preset angle;

[0044] calculate a mean value of the line-of-sight directions corresponding to the retained eye images according to the line-of-sight directions corresponding to the retained eye images of each eye of the driver at each moment, to obtain the line-of-sight directions of the eyes of the driver;

[0045] determine the gaze region of the driver according to the line-of-sight directions of the eyes of the driver, the head position information, the head rotation angle, and the position information of each region in the vehicle.

[0046] In combination with the first aspect, the embodiments of the present disclosure provide a sixth possible implementation manner of the first aspect, wherein the head rotation angle includes a head yaw angle in a horizontal direction and a head pitch angle in a vertical direction; and the determining whether the driving state of the driver is the distracted driving state according to the head rotation angle and the gaze region includes:

[0047] determine that the driving state of the driver is the distracted driving state when any one or more of the following distracted situations is continuously met within a preset time length; the distracted situations include that the head yaw angle is greater than a first preset angle, the head pitch angle is greater than a second preset angle, and the gaze region is a region other than the front windshield region in the vehicle.

[0048] In the second aspect, the embodiments of the present disclosure further provide a driving state detection device, which includes:

[0049] a first acquisition module, configured to acquire, in real time, a to-be-detected image containing a head of a driver at a current moment during driving of the driver;

[0050] a first detection module, configured to detect a head image of the driver from the to-be-detected image, input the head image into a trained head pose estimation model, and output a head pose of the driver by the head pose estimation model; the head pose includes a head rotation angle and head position information;

[0051] a second detection module, configured to detect a face region of the driver from the to-be-detected image when the to-be-detected image contains the face region of the driver;

[0052] a first determination module, configured to determine face feature point position information from the face region, and determine eye feature point position information from the face feature point position information;

[0053] a second determining module, configured to determine a number of occluded eye feature points according to the eye feature point position information, and determine an eye region from the face region according to the eye feature point position information, so as to determine an eye image according to the eye region;

[0054] a first judging module, configured to judge whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points;

[0055] an input module, configured to input the eye image into a trained gaze direction estimation model when the eye image can be used for gaze direction estimation, so as to determine a gaze direction of the driver by the gaze direction estimation model;

[0056] a third determining module, configured to determine a gaze region of the driver according to the gaze direction of the driver, the head position information, the head rotation angle and position information of each region in the vehicle;

[0057] a second judging module, configured to judge whether a driving state of the driver is a distracted driving state according to the head rotation angle and the gaze region.

[0058] With reference to the second aspect, embodiments of the present disclosure provide a first possible implementation manner of the second aspect, and the apparatus further includes:

[0059] a third judging module, configured to judge whether a driving state of the driver is a distracted driving state according to the head rotation angle of the driver when the face region of the driver is not included in the to-be-detected image.

[0060] With reference to the second aspect, embodiments of the present disclosure provide a second possible implementation manner of the second aspect, wherein the occluded eye feature points are occluded eye feature points of a single eye, and the first judging module, when judging whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points, is specifically configured to:

[0061] calculate, for each eye, a first ratio of the number of occluded eye feature points corresponding to the eye to a preset total number of eye feature points of a single eye;

[0062] when the first ratio is greater than a first preset threshold, it is indicated that the eye image corresponding to the eye cannot be used for gaze direction estimation;

[0063] when the first ratio is not greater than the first preset threshold, inputting the eye image corresponding to the eye into a trained glasses-wearing discrimination model to determine whether the eye wears glasses through the glasses-wearing discrimination model;

[0064] if the eye does not wear glasses, indicating that the eye image corresponding to the eye can be used for gaze direction estimation;

[0065] if the eye wears glasses, determining a type of the glasses;

[0066] when the type of the glasses is a light-blocking glasses, indicating that the eye image corresponding to the eye cannot be used for gaze direction estimation;

[0067] when the type of the glasses is a transparent glasses, determining whether a light spot on a lens corresponding to the eye in the glasses meets a condition of affecting gaze direction estimation;

[0068] when the light spot on the lens corresponding to the eye in the glasses meets the condition of affecting gaze direction estimation, indicating that the eye image corresponding to the eye cannot be used for gaze direction estimation; when the light spot on the lens corresponding to the eye in the glasses does not meet the condition of affecting gaze direction estimation, indicating that the eye image corresponding to the eye can be used for gaze direction estimation.

[0069] With reference to the second possible implementation manner of the second aspect, the present disclosure provides a third possible implementation manner of the second aspect, wherein the first determining module, when used for determining the type of the glasses if the eye wears glasses, is specifically used for:

[0070] determining an inner eyelid region and an outer eyelid region from the eye image of the eye;

[0071] calculating a pixel gray mean value of the inner eyelid region according to a gray value of each pixel point in the inner eyelid region, and calculating a pixel gray mean value of the outer eyelid region according to a gray value of each pixel point in the outer eyelid region;

[0072] calculating a second ratio between the pixel gray mean value of the inner eyelid region and the pixel gray mean value of the outer eyelid region;

[0073] when the second ratio is greater than a second preset threshold, determining that the type of the glasses is a light-blocking glasses; when the second ratio is not greater than the second preset threshold, determining that the type of the glasses is a transparent glasses.

[0074] With reference to the second possible implementation manner of the second aspect, the present disclosure provides a fourth possible implementation manner of the second aspect. In the fourth possible implementation manner, when the category of the glasses is transparent glasses, the first determining module is specifically configured to:

[0075] when the category of the glasses is transparent glasses, calculate a third ratio of a number of pixel points with a gray value greater than a third preset threshold to a total number of pixel points in the eye image of the eye according to the gray values of the pixel points in the eye image of the eye;

[0076] if the third ratio is greater than a fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses meets the condition of affecting the estimation of the gaze direction;

[0077] if the third ratio is not greater than the fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses does not meet the condition of affecting the estimation of the gaze direction.

[0078] With reference to the second possible implementation manner of the second aspect, the present disclosure provides a fifth possible implementation manner of the second aspect. In the fifth possible implementation manner, the gaze direction is the gaze direction of a single eye; and the third determining module is specifically configured to:

[0079] for each time between the current time and a target historical time, acquire a first historical eye image of the driver at a first historical time of the time and a second historical eye image of the driver at a second historical time of the time; the first historical time is the previous time of the time, and the second historical time is the next time of the first historical time;

[0080] for each eye, determine a first gaze direction of the eye at the first historical time according to the first historical eye image of the eye, and determine a second gaze direction of the eye at the second historical time according to the second historical eye image of the eye;

[0081] predict a predicted gaze direction of the eye at the time according to the first gaze direction and the second gaze direction;

[0082] calculate an included angle between the predicted gaze direction of the eye at the time and the gaze direction of the eye at the time, retain the eye image of the eye at the time when the included angle is not greater than a third preset angle, and delete the eye image of the eye at the time when the included angle is greater than the third preset angle;

[0083] According to the mean value of the gaze directions corresponding to the eye images retained by each eye of the driver at each of the time points, the gaze directions of the two eyes of the driver are obtained.

[0084] According to the gaze directions of the two eyes of the driver, the head position information, the head rotation angle, and the position information of each region in the vehicle, the gaze region of the driver is determined.

[0085] With reference to the second aspect, a sixth possible implementation manner of the second aspect is provided, wherein the head rotation angle includes a head yaw angle in a horizontal direction and a head pitch angle in a vertical direction; and the second determining module, when used to determine whether the driving state of the driver is the distracted driving state according to the head rotation angle and the gaze region, is specifically used to:

[0086] When any one or more of the following distracted situations is continuously met within a preset time length, it is determined that the driving state of the driver is the distracted driving state; the distracted situations include that the head yaw angle is greater than a first preset angle, the head pitch angle is greater than a second preset angle, and the gaze region is a region other than the front windshield region in the vehicle.

[0087] In a third aspect, the embodiments of the present disclosure further provide an electronic device, including a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps in any one of the possible implementation manners of the first aspect.

[0088] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the steps in any one of the possible implementation manners of the first aspect.

[0089] The driving state detection method and device, the electronic device, and the computer readable storage medium provided by the embodiments of the present disclosure, wherein the head image of the driver is input into a head pose estimation model to determine the head pose of the driver, and then the head pose is used to determine whether the driving state of the driver is a distracted driving state. Compared with the method relying on facial feature point information, the method in the present disclosure can identify the head pose of the driver regardless of the rotation angle of the face of the driver or the pitch angle of the face of the driver, and can detect the distraction of the driver, which is beneficial to reduce the safety risk of the driver during driving.

[0090] Meanwhile, when the face region of the driver is included in the to-be-detected image, it is determined whether the eye image can be used for the gaze direction estimation, and when the eye image can be used for the gaze direction estimation, the gaze direction is determined by using the eye image, and then the gaze region of the driver is determined according to the gaze direction. Then, whether the driving state of the driver is the distraction driving state is determined by combining the gaze region and the head rotation angle in the head posture. In this embodiment, it is determined whether the eye image can be used before the gaze direction is determined by using the eye image, and the gaze direction is determined by using the eye image only when the eye image can be used, so that the determined gaze direction is more accurate, thereby improving the accuracy of the distraction state detection of the driver.

[0091] In order to make the above objectives, characteristics and advantages of the present disclosure more apparent, the following will specifically describe a preferred embodiment in conjunction with the accompanying drawings, and make a detailed description as follows. BRIEF DESCRIPTION OF DRAWINGS

[0092] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained from these drawings without creative labor.

[0093] FIG. 1 shows a flowchart of a driving state detection method provided by an embodiment of the present disclosure;

[0094] FIG. 2 shows a schematic diagram of eye feature point position information of an eye provided by an embodiment of the present disclosure;

[0095] FIG. 3 shows a flowchart of another driving state detection method provided by an embodiment of the present disclosure;

[0096] FIG. 4 shows a structural schematic diagram of a driving state detection device provided by an embodiment of the present disclosure;

[0097] FIG. 5 shows a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0098] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the following will be combined with the accompanying drawings of the embodiments of the present disclosure to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, but not all the embodiments. The components of the embodiments of the present disclosure generally described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0099] In view of the prior art, when detecting the distraction of the driver, the solvPnP algorithm (an algorithm for estimating the camera pose) is used to detect whether the driver is in a state of distraction. The way of detecting whether the driver is in a state of distraction needs to rely on the facial feature point information of the driver. When the camera cannot capture the facial image of the driver, the facial feature point information of the driver cannot be recognized, and at this time, the driver cannot be detected for distraction, which undoubtedly increases the safety risk in the driving process. In the way of using the line-of-sight direction of the driver to judge whether the driver is in a state of distraction, if the driver wears sunglasses or glasses with surface reflection, the line-of-sight direction recognition becomes unreliable, and at this time, the line-of-sight direction estimation is prone to error, thereby affecting the accuracy of the distraction detection. Based on this, the embodiments of the present disclosure provide a driving state detection method, device, electronic equipment and readable storage medium, which are described below through embodiments.

[0100] In order to facilitate the understanding of the present embodiment, first, a driving state detection method disclosed by the embodiments of the present disclosure is described in detail. FIG. 1 shows a flowchart of a driving state detection method provided by the embodiments of the present disclosure, as shown in FIG. 1, including the following steps S101-S109:

[0101] S101: In the process of driving a vehicle by a driver, a to-be-detected image containing the head of the driver at the current time is acquired in real time.

[0102] In this embodiment, the vehicle driven by the driver includes a car, a new energy vehicle, etc. A camera is arranged in the cockpit of the vehicle, and the camera is used to collect a video stream containing the head of the driver in real time in the process of driving the vehicle by the driver.

[0103] The video stream is acquired from the camera, and the video stream is decomposed into a single-frame to-be-detected image containing the head of the driver. Generally, the to-be-detected image at the current time is the latest to-be-detected image decomposed from the video stream.

[0104] S102: Detect the head image of the driver from the to-be-detected image, input the head image into the trained head posture estimation model, and output the head posture of the driver through the head posture estimation model; the head posture includes a head rotation angle and head position information.

[0105] In this embodiment, the to-be-detected image not only includes the head region of the driver, but also includes a background region. In order to remove the background interference, whether the head region of the driver is included in the to-be-detected image is detected through the head detection model. When the head region of the driver is included, a head detection frame is output through the head detection model, and then the head image of the driver is cropped from the to-be-detected image along the head detection frame.

[0106] The head image is input into the pre-trained head posture estimation model, and the head posture of the driver is estimated through the head posture estimation model. The head posture estimation model is trained using sample images containing heads.

[0107] The head posture includes a head rotation angle and head position information. The head rotation angle includes a head yaw angle in the horizontal direction and a head pitch angle in the vertical direction. The head rotation angle refers to the angle of turning the head left and right. For example, when the driver looks back, the head rotation angle of the driver can be 180 degrees. The head pitch angle refers to the angle of tilting the head up and down. For example, the driver looks up or looks down. The head position information is the head position coordinates in a three-dimensional coordinate system centered on the camera.

[0108] S103: When the to-be-detected image includes the face region of the driver, the face region of the driver is detected from the to-be-detected image.

[0109] In this embodiment, since the to-be-detected image containing the head of the driver is obtained, the to-be-detected image can include the face region of the driver or can not include the face region of the driver.

[0110] When the to-be-detected image includes the face region of the driver, steps S104-S109 are continued.

[0111] Specifically, whether the face region of the driver is included in the to-be-detected image is detected through the face detection model. When the face region of the driver is included, a face region frame is output through the face detection model, and then the region framed by the face detection frame is taken as the face region.

[0112] In one possible implementation, when the to-be-detected image does not include the face region of the driver, whether the driving state of the driver is a distracted driving state is determined according to the head rotation angle of the driver.

[0113] In the embodiment, when the face region of the driver is not included in the image to be detected, the subsequent steps S104-S109 are stopped, and the head rotation angle of the driver is directly used to determine whether the driving state of the driver is the distracted driving state.

[0114] In the determination of whether the driving state of the driver is the distracted driving state according to the head rotation angle of the driver, specifically, when the head yaw angle is greater than a first preset angle, or the head pitch angle is greater than a second preset angle, it is determined that the driving state of the driver is the distracted driving state. When the head yaw angle is not greater than the first preset angle, and the head pitch angle is not greater than the second preset angle, it is determined that the driving state of the driver is the undistracted driving state.

[0115] S104: Determine the face feature point position information from the face region, and determine the eye feature point position information from the face feature point position information.

[0116] The face feature point position information includes eye feature point position information, mouth feature point position information, nose feature point position information, eyebrow feature point position information, and face contour feature point position information. As shown in FIG. 2, the eye feature point position information can be eye contour feature point position information.

[0117] S105: Determine the number of occluded eye feature points according to the eye feature point position information, and determine the eye region from the face region according to the eye feature point position information, so as to determine the eye image according to the eye region.

[0118] In the embodiment, the total number of eye feature points is fixed, and the difference between the total number of eye feature points and the number of identifiable (i.e., unoccluded) eye feature points can be used as the number of occluded eye feature points.

[0119] S106: Determine whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points.

[0120] In one possible implementation, the eye image in step S105 is a single eye image, and the occluded eye feature point is also a single eye occluded eye feature point. When step S106 is performed, it can be performed according to the following steps S1061-S1068:

[0121] S1061: For each eye, calculate a first ratio of the number of occluded eye feature points corresponding to the eye to the total number of single eye feature points.

[0122] In this embodiment, when the eye is not occluded, the number of occluded eye feature points of the single eye should be 0, and the total number of preset single eye feature points should be fixed. As shown in FIG. 2, the total number of preset single eye feature points is 6.

[0123] However, in reality, due to wearing glasses or head rotation, the number of eye feature points of the single eye identified may be less than the total number of preset single eye feature points, that is, there are occluded eye feature points at this time.

[0124] In this embodiment, by calculating the first ratio of the number of occluded eye feature points corresponding to each eye to the total number of preset single eye feature points, it is determined whether the eye image corresponding to each eye can be used for gaze direction estimation through the first ratio.

[0125] S1062: When the first ratio is greater than a first preset threshold, it is indicated that the eye image corresponding to the eye cannot be used for gaze direction estimation.

[0126] In this embodiment, taking one eye of the driver as an example, as shown in FIG. 3, when the first ratio of the eye is greater than the first preset threshold, it is indicated that the occluded part of the eye is more, and therefore the eye image corresponding to the eye cannot be used for gaze direction estimation. Exemplarily, the first preset threshold is 1 / 2.

[0127] S1063: When the first ratio is not greater than the first preset threshold, the eye image corresponding to the eye is input into the trained glasses-wearing discrimination model to determine whether the eye wears glasses through the glasses-wearing discrimination model.

[0128] As shown in FIG. 3, when the first ratio of the eye is not greater than (i.e., less than or equal to) the first preset threshold, it is indicated that the occluded part of the eye is less, and therefore it is necessary to further determine whether each eye wears glasses.

[0129] S1064: If the eye does not wear glasses, it is indicated that the eye can be used for gaze direction estimation.

[0130] S1065: If the eye wears glasses, the type of the glasses is determined.

[0131] In this embodiment, the type of the glasses includes light-shielding glasses and transparent glasses, wherein the light-shielding glasses are, for example, sunglasses, and the transparent glasses are, for example, myopia glasses, presbyopia glasses, and plain glasses. Generally, the light transmittance of the light-shielding glasses is less than that of the transparent glasses.

[0132] In a possible implementation, when step S1065 is performed, the following steps can be specifically performed: determining an inner eyelid region and an outer eyelid region from the eye image of the eye; calculating a pixel gray mean value of the inner eyelid region according to a gray value of each pixel point in the inner eyelid region; and calculating a pixel gray mean value of the outer eyelid region according to a gray value of each pixel point in the outer eyelid region. A second ratio between the pixel gray mean value of the inner eyelid region and the pixel gray mean value of the outer eyelid region is calculated. When the second ratio is greater than a second preset threshold, it is determined that the type of the glasses is the sunglasses; and when the second ratio is not greater than the second preset threshold, it is determined that the type of the glasses is the transparent glasses.

[0133] In this embodiment, as shown in FIG. 2, the inner eyelid region in the eye image refers to a region inside an eye contour formed by the eye feature points, and the outer eyelid region refers to other regions in the eye image except the inner eyelid region.

[0134] S1066: When the type of the glasses is the sunglasses, it is indicated that the eye image corresponding to the eye cannot be used for the gaze direction estimation.

[0135] S1067: When the type of the glasses is the transparent glasses, it is determined whether a light spot on a lens corresponding to the eye in the glasses satisfies a condition affecting the gaze direction estimation.

[0136] In a possible implementation, when step S1067 is performed, the following steps can be specifically performed: when the type of the glasses is the transparent glasses, a third ratio between a number of pixel points with a gray value greater than a third preset threshold and a total number of pixel points in the eye image is calculated according to the gray value of each pixel point in the eye image of the eye; if the third ratio of the eye is greater than a fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses satisfies the condition affecting the gaze direction estimation; and if the third ratio of the eye is not greater than the fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses does not satisfy the condition affecting the gaze direction estimation.

[0137] S1068: When the light spot on the lens corresponding to the eye in the glasses satisfies the condition affecting the gaze direction estimation, it is indicated that the eye image corresponding to the eye cannot be used for the gaze direction estimation; and when the light spot on the lens corresponding to the eye in the glasses does not satisfy the condition affecting the gaze direction estimation, it is indicated that the eye image corresponding to the eye can be used for the gaze direction estimation.

[0138] S107: When the eye image can be used for the gaze direction estimation, the eye image is input into the trained gaze direction estimation model, and the gaze direction of the driver is determined through the gaze direction estimation model.

[0139] In this embodiment, when the eye image cannot be used for the gaze direction estimation, the subsequent steps S107-S109 are stopped from being executed, and the driving state of the driver is directly judged according to the head rotation angle of the driver.

[0140] As shown in FIG. 1, in this embodiment, the execution sequence of step S102 and steps S103-S107 can be that step S102 is executed first, and then steps S103-S107 are executed; or steps S103-S107 can be executed first, and then step S102 is executed; or steps S102 and steps S103-S107 can be executed synchronously, and the present disclosure does not limit this.

[0141] S108: determining the gaze area of the driver according to the gaze direction of the driver, the head position information, the head rotation angle, and the position information of each region in the vehicle.

[0142] In this embodiment, in order to improve the accuracy of the determination of the gaze area of the driver, the stability of the gaze direction of the driver needs to be judged. When the gaze direction of the driver has a significant jump, it is considered that the gaze direction is unstable, and the accuracy of the gaze area determined using the unstable gaze direction will also be affected. Therefore, when the gaze area of the driver is determined, the stability of the gaze direction can be judged first.

[0143] In a possible implementation, the gaze direction of the driver is the gaze direction of a single eye; when step S108 is executed, it can be executed according to the following steps:

[0144] S1081: for each time between the current time and the target historical time, the first historical eye image of each eye of the driver at the first historical time of the time and the second historical eye image of each eye of the driver at the second historical time of the time are obtained; wherein the first historical time is the previous time of the time, and the second historical time is the next time of the first historical time.

[0145] S1082: for each eye, the first gaze direction of the eye at the first historical time is determined according to the first historical eye image of the eye, and the second gaze direction of the eye at the second historical time is determined according to the second historical eye image of the eye.

[0146] S1083: the predicted gaze direction of the eye at the time is predicted according to the first gaze direction and the second gaze direction.

[0147] S1084: the included angle between the predicted gaze direction of the eye at the time and the gaze direction of the eye at the time is calculated, when the included angle is not greater than a third preset angle, the eye image corresponding to the eye at the time is retained, and when the included angle is greater than the third preset angle, the eye image corresponding to the eye at the time is deleted.

[0148] S1085: Calculate the mean value of the gaze directions corresponding to the retained eye images of the driver at each time, to obtain the gaze directions of the driver's eyes.

[0149] S1086: Determine the gaze area of the driver according to the gaze directions of the driver's eyes, the head position information, the head rotation angle, and the position information of each area in the vehicle.

[0150] In step S1081, the target historical time refers to a historical time before the current time, and the length of the time period between the current time and the target historical time is the specified length. For example, when the current time is t0 and the target historical time is tn, and the specified length is 6, n is 6, that is, the time points between the current time t0 and the target historical time t6 are t0, t1, t2, t3, t4, t5, t6, respectively, and the time of each time point is in the order of t6, t5, t4, t3, t2, t1, t0.

[0151] Taking time t4 as an example, the first historical eye image of each eye of the driver at time t4 and the second historical eye image of each eye of the driver at time t4 are obtained.

[0152] In step S1082, taking one eye as an example, the first historical eye image of the eye is input into the gaze direction estimation model, and the first gaze direction of the eye at the first historical time t5 is output by the gaze direction estimation model. The second historical eye image of the eye is input into the gaze direction estimation model, and the second gaze direction of the eye at the second historical time t6 is output by the gaze direction estimation model.

[0153] In step S1083, the predicted gaze direction of the eye at time t4 is predicted according to the first gaze direction and the second gaze direction of the eye.

[0154] In step S1084, the included angle between the predicted gaze direction of the eye at time t4 determined in step S1083 and the gaze direction of the eye at time t4 determined in step S107 is calculated, and when the included angle is not greater than the third preset angle, the eye image corresponding to the eye at time t4 is retained; when the included angle is greater than the third preset angle, the eye image corresponding to the eye at time t4 is deleted.

[0155] In step S1085, for example, for the left eye of the driver, if the eye images corresponding to time t3 and time t2 are deleted, and the eye images corresponding to time t6, time t5, time t4, time t1 and time t0 are retained, and for the right eye of the driver, if the eye images corresponding to time t3 and time t4 are deleted, and the eye images corresponding to time t6, time t5, time t2, time t1 and time t0 are retained, the average of the gaze directions of the left eye at time t6, time t5, time t4, time t1 and time t0 determined in step S105, and the average of the gaze directions of the right eye at time t6, time t5, time t2, time t1 and time t0 determined in step S105 is calculated to obtain the gaze directions of the eyes of the driver.

[0156] In step S1086, the head position information and the position information of each region in the vehicle are position coordinates in a three-dimensional rectangular coordinate system centered on the camera, so that the gaze region of the driver can be determined according to the gaze directions of the eyes of the driver, the head position information, the head rotation angle and the position information of each region in the vehicle.

[0157] S109: determining whether the driving state of the driver is a distracted driving state according to the head rotation angle and the gaze region.

[0158] In a possible implementation, the head rotation angle includes a head rotation angle in the horizontal direction and a head pitch angle in the vertical direction. When any one or more of the following distraction conditions is continuously met within a preset time length, it is determined that the driving state of the driver is a distracted driving state; the distraction conditions include that the head yaw angle is greater than a first preset angle, the head pitch angle is greater than a second preset angle, and the gaze region is a region other than the front windshield region in the vehicle.

[0159] Based on the same technical concept, the disclosure also provides a driving state detection device. FIG. 4 shows a structural schematic diagram of a driving state detection device provided by an embodiment of the disclosure. As shown in FIG. 4, the device includes:

[0160] A first acquisition module 401 is configured to acquire, in real time, a to-be-detected image containing a head of a driver at a current time during driving of the vehicle by the driver.

[0161] A first detection module 402 is configured to detect a head image of the driver from the to-be-detected image, input the head image to a trained head pose estimation model, and output a head pose of the driver by the head pose estimation model; the head pose includes a head rotation angle and head position information.

[0162] The second detection module 403 is configured to detect a face region of the driver from the image to be detected when the image to be detected contains the face region of the driver.

[0163] The first determination module 404 is configured to determine face feature point position information from the face region, and determine eye feature point position information from the face feature point position information.

[0164] The second determination module 405 is configured to determine a number of occluded eye feature points according to the eye feature point position information, and determine an eye region from the face region according to the eye feature point position information, so as to determine an eye image according to the eye region.

[0165] The first judgment module 406 is configured to judge whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points.

[0166] The input module 407 is configured to input the eye image into a trained gaze direction estimation model when the eye image can be used for gaze direction estimation, so as to determine a gaze direction of the driver by using the gaze direction estimation model.

[0167] The third determination module 408 is configured to determine a gaze region of the driver according to the gaze direction of the driver, the head position information, the head rotation angle and position information of each region in the vehicle.

[0168] The second judgment module 409 is configured to judge whether a driving state of the driver is a distracted driving state according to the head rotation angle and the gaze region.

[0169] Optionally, the device further comprises:

[0170] The third judgment module is configured to judge whether a driving state of the driver is a distracted driving state according to the head rotation angle of the driver when the image to be detected does not contain the face region of the driver.

[0171] Optionally, the occluded eye feature points are occluded eye feature points of a single eye, and the first judgment module 406 is specifically configured to:

[0172] calculate, for each eye, a first ratio of a number of the occluded eye feature points of the eye to a preset total number of eye feature points of a single eye;

[0173] when the first ratio is greater than a first preset threshold, it is indicated that the eye image corresponding to the eye cannot be used for gaze direction estimation;

[0174] when the first ratio is not greater than the first preset threshold, the eye image corresponding to the eye is input into a trained glasses discrimination model to determine whether the eye wears glasses through the glasses discrimination model;

[0175] if the eye does not wear glasses, it is indicated that the eye can be used for gaze direction estimation;

[0176] if the eye wears glasses, the type of the glasses is determined;

[0177] when the type of the glasses is a light-blocking glasses, it is indicated that the eye image corresponding to the eye cannot be used for gaze direction estimation;

[0178] when the type of the glasses is a transparent glasses, it is determined whether a light spot on a lens corresponding to the eye in the glasses meets a condition of affecting gaze direction estimation;

[0179] when the light spot on the lens corresponding to the eye in the glasses meets the condition of affecting gaze direction estimation, it is indicated that the eye image corresponding to the eye cannot be used for gaze direction estimation; when the light spot on the lens corresponding to the eye in the glasses does not meet the condition of affecting gaze direction estimation, it is indicated that the eye image corresponding to the eye can be used for gaze direction estimation.

[0180] Optionally, the first determination module 406 is specifically configured to, when used for determining the type of the glasses if the eye wears glasses:

[0181] determine an inner eyelid region and an outer eyelid region from the eye image of the eye;

[0182] calculate a pixel gray mean value of the inner eyelid region according to a gray value of each pixel point in the inner eyelid region, and calculate a pixel gray mean value of the outer eyelid region according to a gray value of each pixel point in the outer eyelid region;

[0183] calculate a second ratio between the pixel gray mean value of the inner eyelid region and the pixel gray mean value of the outer eyelid region;

[0184] when the second ratio is greater than a second preset threshold, it is determined that the type of the glasses is a light-blocking glasses; when the second ratio is not greater than the second preset threshold, it is determined that the type of the glasses is a transparent glasses.

[0185] Optionally, the first determining module 406 is specifically used for:

[0186] when the category of the glasses is transparent glasses, calculating a third ratio of a number of pixel points with a gray value greater than a third preset threshold to a total number of pixel points in the eye image of the eye according to the gray values of the pixel points in the eye image of the eye;

[0187] if the third ratio is greater than a fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses meets the condition of affecting the estimation of the gaze direction;

[0188] if the third ratio is not greater than the fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses does not meet the condition of affecting the estimation of the gaze direction.

[0189] Optionally, the gaze direction is the gaze direction of a single eye; and the third determining module 408 is specifically used for:

[0190] for each time between the current time and a target historical time, acquiring a first historical eye image of the driver at a first historical time of the time and a second historical eye image of the driver at a second historical time of the time; the first historical time is the previous time of the time, and the second historical time is the next time of the first historical time;

[0191] for each eye, determining a first gaze direction of the eye at the first historical time according to the first historical eye image of the eye, and determining a second gaze direction of the eye at the second historical time according to the second historical eye image of the eye;

[0192] predicting a predicted gaze direction of the eye at the time according to the first gaze direction and the second gaze direction;

[0193] calculating an included angle between the predicted gaze direction of the eye at the time and the gaze direction of the eye at the time, retaining the eye image corresponding to the eye at the time when the included angle is not greater than a third preset angle, and deleting the eye image corresponding to the eye at the time when the included angle is greater than the third preset angle;

[0194] calculating a mean value of the gaze directions corresponding to the retained eye images of each eye of the driver, to obtain the gaze directions of the eyes of the driver.

[0195] determine the gazing area of the driver according to the line-of-sight direction of the driver's two eyes, the head position information, the head rotation angle, and the position information of each area in the vehicle.

[0196] Optionally, the head rotation angle includes a head yaw angle in a horizontal direction and a head pitch angle in a vertical direction; and the second determining module 409 is specifically configured to determine whether the driving state of the driver is the distracted driving state according to the head rotation angle and the gazing area.

[0197] When any one or more of the following distracted situations is continuously met within a preset time length, it is determined that the driving state of the driver is the distracted driving state; the distracted situations include that the head yaw angle is greater than a first preset angle, the head pitch angle is greater than a second preset angle, and the gazing area is an area other than the front windshield area in the vehicle.

[0198] FIG. 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure, which includes a processor 501, a memory 502, and a bus 503. The memory 502 stores machine readable instructions executable by the processor 501. When the electronic device runs the information processing method described above, the processor 501 and the memory 502 communicate through the bus 503. The processor 501 executes the machine readable instructions to perform the method steps described in Embodiment I.

[0199] The present disclosure also provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the method steps described in Embodiment I.

[0200] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described apparatus, electronic device, and computer readable storage medium can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0201] In several embodiments provided by the present disclosure, it should be understood that the disclosed apparatus and method can be implemented by other manners. The apparatus embodiments described above are merely schematic, for example, the division of the modules is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed ones can be indirect coupling or communication connection through some communication interfaces, apparatuses or units, which can be electrical, mechanical or other forms.

[0202] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0203] In addition, each functional unit in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0204] If the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various program code storage media.

[0205] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them, the protection scope of the present disclosure is not limited thereto, although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: any skilled person in the art within the technical range disclosed by the present disclosure, they can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and all should be covered in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims. Industrial applicability

[0206] The driving state detection method, device, electronic equipment and readable storage medium provided by the present disclosure can improve the accuracy of the distraction state detection by analyzing the head posture and the line-of-sight direction of the driver and judging whether the driving state is a distraction driving state. When judging whether the driver is in a distraction state according to the head posture, the head posture of the driver is input into a head posture estimation model to judge whether the driving state is a distraction driving state, so that no matter how large the rotation angle of the face of the driver is or how large the pitch angle of the face of the driver is, the head posture of the driver can be recognized to detect the distraction of the driver. When judging whether the driver is in a distraction state according to the line-of-sight direction, it is first judged whether the eye image is available, and the line-of-sight direction is determined using the eye image only when the eye image is available; when the eye image can be used for line-of-sight direction estimation, the stability of the line-of-sight direction of the driver needs to be judged, so as to improve the reliability when confirming the gaze area of the driver. The driving state detection method, device, electronic equipment and readable storage medium provided by the present disclosure improve the accuracy of the distraction state detection, reduce traffic accidents caused by the distraction of the driver, and significantly improve the driving safety of the vehicle.

Claims

1. A driving state detection method characterized by comprising: The method comprises: acquiring, in real time, a to-be-detected image containing a head of a driver during driving of the driver; detecting a head image of the driver from the to-be-detected image, inputting the head image into a trained head posture estimation model, and outputting a head posture of the driver by the head posture estimation model; the head posture comprises a head rotation angle and head position information; when the to-be-detected image contains a face region of the driver, detecting the face region of the driver from the to-be-detected image; determining human face feature point position information from the face region, and determining eye feature point position information from the human face feature point position information; determining a number of occluded eye feature points according to the eye feature point position information; determining an eye region from the face region according to the eye feature point position information, and determining an eye image according to the eye region; judging whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points; when the eye image can be used for gaze direction estimation, inputting the eye image into a trained gaze direction estimation model, and determining a gaze direction of the driver by the gaze direction estimation model; determining a gaze region of the driver according to the gaze direction of the driver, the head position information, the head rotation angle, and position information of each region in the vehicle; judging whether a driving state of the driver is a distracted driving state according to the head rotation angle and the gaze region.

2. The method of claim 1, wherein, The method further comprises: when the to-be-detected image does not contain the face region of the driver, judging whether the driving state of the driver is the distracted driving state according to the head rotation angle of the driver.

3. The method of claim 1, wherein, The occluded eye feature points are occluded eye feature points of a single eye; and the judging whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points comprises: calculating, for each eye, a first ratio of a number of the occluded eye feature points corresponding to the eye to a preset total number of single eye feature points; when the first ratio is greater than a first preset threshold, it is indicated that the eye image corresponding to the eye cannot be used for gaze direction estimation; when the first ratio is not greater than the first preset threshold, inputting the eye image corresponding to the eye into a trained glasses discrimination model to judge whether the eye wears glasses by the glasses discrimination model; if the eye does not wear glasses, it is indicated that the eye can be used for gaze direction estimation; if the eye wears glasses, a type of the glasses is determined; when the type of the glasses is a shading glass, it is indicated that the eye image corresponding to the eye cannot be used for gaze direction estimation; when the type of the glasses is a transparent glass, it is judged whether a light spot on a lens corresponding to the eye in the glass meets a condition of affecting gaze direction estimation. When the light spot on the lens corresponding to the eye in the glasses meets the condition affecting the gaze direction estimation, the eye image corresponding to the eye is not capable of being used for the gaze direction estimation; when the light spot on the lens corresponding to the eye in the glasses does not meet the condition affecting the gaze direction estimation, the eye image corresponding to the eye is capable of being used for the gaze direction estimation.

4. The method of claim 3, wherein, If the eye wears glasses, the type of the glasses is determined, including: An inner eyelid region and an outer eyelid region are determined from the eye image of the eye; A pixel gray mean value of the inner eyelid region is calculated according to a gray value of each pixel point in the inner eyelid region, and a pixel gray mean value of the outer eyelid region is calculated according to a gray value of each pixel point in the outer eyelid region; A second ratio between the pixel gray mean value of the inner eyelid region and the pixel gray mean value of the outer eyelid region is calculated; When the second ratio is greater than a second preset threshold, the type of the glasses is determined as dark glasses; when the second ratio is not greater than the second preset threshold, the type of the glasses is determined as transparent glasses.

5. The method of claim 3, wherein, When the type of the glasses is transparent glasses, whether the light spot on the lens corresponding to the eye in the glasses meets the condition affecting the gaze direction estimation is determined, including: When the type of the glasses is transparent glasses, a third ratio between a number of pixel points with a gray value greater than a third preset threshold and a total number of pixel points in the eye image is calculated according to the gray value of each pixel point in the eye image of the eye; If the third ratio is greater than a fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses meets the condition affecting the gaze direction estimation; If the third ratio is not greater than the fourth preset threshold, it is indicated that the light spot on the lens corresponding to the eye in the glasses does not meet the condition affecting the gaze direction estimation.

6. The method of claim 3, wherein, The gaze direction is a gaze direction of a single eye; and the gaze region of the driver is determined according to the gaze direction of the driver, the head position information, the head rotation angle and the position information of each region in the vehicle, including: For each time between the current time and a target historical time, a first historical eye image of the eye at a first historical time and a second historical eye image of the eye at a second historical time at the time are acquired; the first historical time is a previous time of the time, and the second historical time is a next time of the first historical time; For each eye, a first gaze direction of the eye at the first historical time is determined according to the first historical eye image of the eye, and a second gaze direction of the eye at the second historical time is determined according to the second historical eye image of the eye; A predicted gaze direction of the eye at the time is predicted according to the first gaze direction and the second gaze direction. calculate an included angle between the predicted gaze direction of the eye at the time and the gaze direction of the eye at the time, retain the eye image corresponding to the eye of the driver at the time when the included angle is not greater than a third preset angle, and delete the eye image corresponding to the eye of the driver at the time when the included angle is greater than the third preset angle; calculate a mean value of the gaze directions corresponding to the retained eye images according to the gaze directions corresponding to the retained eye images of each eye of the driver at each time, and obtain the gaze directions of the eyes of the driver; determine the gaze region of the driver according to the gaze directions of the eyes of the driver, the head position information, the head rotation angle, and the position information of each region in the vehicle.

7. The method of claim 1, wherein, The head rotation angle includes a head yaw angle in a horizontal direction and a head pitch angle in a vertical direction; and the determination of whether the driving state of the driver is a distracted driving state according to the head rotation angle and the gaze region includes: determine that the driving state of the driver is the distracted driving state when any one or more of the following distracted conditions is continuously met within a preset time length; the distracted conditions include that the head yaw angle is greater than a first preset angle, the head pitch angle is greater than a second preset angle, and the gaze region is a region other than the front windshield region in the vehicle.

8. A driving state detection device characterized by comprising: include: a first acquisition module, configured to acquire, in real time, a to-be-detected image containing a head of a driver at a current time during driving of the driver; a first detection module, configured to detect a head image of the driver from the to-be-detected image, input the head image into a trained head pose estimation model, and output a head pose of the driver by the head pose estimation model; the head pose includes a head rotation angle and head position information; a second detection module, configured to detect a face region of the driver from the to-be-detected image when the to-be-detected image contains the face region of the driver; a first determination module, configured to determine face feature point position information from the face region, and determine eye feature point position information from the face feature point position information; a second determination module, configured to determine a number of occluded eye feature points according to the eye feature point position information, and determine an eye region from the face region according to the eye feature point position information, and determine an eye image according to the eye region; a first judgment module, configured to judge whether the eye image can be used for gaze direction estimation according to the number of occluded eye feature points; an input module, configured to input the eye image into a trained gaze direction estimation model when the eye image can be used for gaze direction estimation, and determine a gaze direction of the driver by the gaze direction estimation model; a third determination module, configured to determine a gaze region of the driver according to the gaze direction of the driver, the head position information, the head rotation angle, and position information of each region in the vehicle. ​ A second judging module is configured to judge whether the driving state of the driver is a distracted driving state according to the head rotation angle and the gaze region.

9. An electronic device, comprising: The application further provides a computer readable storage medium, which stores a computer program. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the method according to any one of claims 1 to 7.

Images