Driver state recognition method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of intelligent driver recognition, in particular to a driver state recognition method and device, computer equipment and a storage medium. Extracting feature point data from the driving state image, wherein the feature point data comprises hand feature points; inputting the hand feature points into a driving behavior classification model for driving behavior classification to obtain a behavior category of the driver; and if the behavior category belongs to the preset target distraction behavior, determining that the driver is in a distraction state. Therefore, whether the driver is in the distraction state or not is determined based on the hand feature points extracted from the driving state image of the driver, and compared with the prior art, the state recognition result of the driver is enriched, and the safety of vehicle driving is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driver identification technology, and in particular to a driver status identification method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, the demand for intelligent and safe driving is increasing. Intelligent driver recognition technology has emerged in this context and has quickly become a key technology for improving vehicle safety. This technology integrates cutting-edge technologies such as machine vision, artificial intelligence, and sensors. It uses sensors such as cameras to capture real-time information such as the driver's facial features, eye movements, and head posture. It then uses advanced algorithms to analyze this data to effectively identify the driver's condition and provide timely reminders to drive safely if the driver is fatigued.

[0003] At present, the existing intelligent driver recognition technology is mainly used to identify whether the driver is in a fatigued state, that is, fatigue driving. However, if the driver is not driving fatigued but the driver's attention is not focused, it will still reduce the safety of vehicle driving. Therefore, there is a problem that the driver status recognition results are relatively single. Summary of the Invention

[0004] Embodiments of the present invention provide a driver status recognition method, apparatus, computer equipment, and storage medium to solve the problem of relatively single driver status recognition results.

[0005] A driver status recognition method, comprising: Acquiring a driving status image of the driver; extracting feature point data from the driving state image, the feature point data including hand feature points; Inputting the hand feature points into a driving behavior classification model to perform driving behavior classification to obtain the driver's behavior category; If the behavior category belongs to a preset target distraction behavior, it is determined that the driver is in a distracted state.

[0006] In the above driver status recognition method, optionally, the feature point data further includes facial feature points; After extracting feature point data from the driving state image, the method further includes: Extracting a preset first feature point from the hand feature points, and extracting a preset second feature point from the facial feature points; Calculating a target distance between a hand and a face based on a first coordinate corresponding to the first feature point and a second coordinate corresponding to the second feature point; Determining whether the target distance is greater than a preset distance threshold; If the target distance is greater than the distance threshold, determining that the driver is in a normal state; If the target distance is less than or equal to the distance threshold, it is determined that the driver is in a distracted state.

[0007] The above driver status recognition method may, optionally, further comprise: after extracting feature point data from the driving status image; Calculating the driver's head posture data based on the facial feature points; determining the driver's head posture according to the posture data; If the head posture is a preset target fatigue posture, it is determined that the driver is in a fatigue state.

[0008] In the above driver status recognition method, optionally, the posture data includes an eye aspect ratio, and the eye aspect ratio is calculated as follows: Extracting eyelid feature points and eye corner feature points from the facial feature points; Calculating the eye height according to the eyelid coordinates corresponding to the eyelid feature points; Calculating the eye width according to the eye corner coordinates corresponding to the eye corner feature points; The eye aspect ratio is determined according to the eye height and the eye width.

[0009] In the above driver status recognition method, optionally, the posture data includes a head turning parameter, and the head turning parameter is calculated as follows: Extracting nose tip feature points, left cheek feature points, and right cheek feature points from the facial feature points; Calculating a first parameter based on the nose tip coordinates corresponding to the nose tip feature point and the left cheek coordinates corresponding to the left cheek feature point; Calculating a second parameter based on the nose tip coordinates and the right cheek coordinates corresponding to the right cheek feature points; The rotor parameter is obtained by calculation according to the first parameter and the second parameter.

[0010] In the above driver status recognition method, optionally, the posture data includes a head-down parameter, and the head-down parameter is calculated as follows: Extracting nose tip feature points and chin feature points from the facial feature points; The head-lowering parameter is calculated according to the nose tip coordinates corresponding to the nose tip feature points and the chin coordinates corresponding to the chin feature points.

[0011] In the above driver status recognition method, optionally, the posture data includes a mouth width-to-height ratio, and the mouth width-to-height ratio is calculated as follows: Extracting lip feature points and mouth corner feature points from the facial feature points; Calculating the mouth height according to the lip coordinates corresponding to the lip feature points; Calculating the mouth width according to the mouth corner coordinates corresponding to the mouth corner feature points; The mouth aspect ratio is determined according to the mouth height and the mouth width.

[0012] A driver status recognition device, comprising: An image acquisition module, used to acquire an image of the driver's driving status; A feature point extraction module, configured to extract feature point data from the driving state image, wherein the feature point data includes hand feature points; A behavior classification module, configured to input the hand feature points into a driving behavior classification model to classify the driver's behavior and obtain the driver's behavior category; The first state determination module is configured to determine that the driver is in a distracted state when the behavior category belongs to a preset target distracting behavior.

[0013] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the above-mentioned driver status identification methods when executing the computer program.

[0014] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements any of the above-mentioned driver status recognition methods.

[0015] The driver state recognition method, apparatus, computer device, and storage medium described above acquire a driver's driving state image; extract feature point data from the driving state image, including hand feature points; input the hand feature points into a driving behavior classification model to classify the driver's driving behavior and determine a driver behavior category; and determine that the driver is in a distracted state if the behavior category falls within a preset target distraction behavior. This method, based on hand feature points extracted from the driver's driving state image, enriches driver state recognition results and improves vehicle driving safety compared to existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 This is a flowchart of a method for identifying a driver's state according to an embodiment of the present invention; Figure 2 is a schematic diagram of hand feature points in one embodiment of the present invention; Figure 3 is another implementation flow chart of the driver status identification method according to one embodiment of the present invention; Figure 4 is a schematic diagram of facial feature points in one embodiment of the present invention; Figure 5 is another implementation flow chart of the driver status identification method according to one embodiment of the present invention; Figure 6 is another implementation flow chart of the driver status identification method according to one embodiment of the present invention; Figure 7 is another implementation flow chart of the driver status identification method according to one embodiment of the present invention; Figure 8 is another implementation flow chart of the driver status identification method according to one embodiment of the present invention; Figure 9 is another implementation flow chart of the driver status identification method according to one embodiment of the present invention; Figure 10 is a schematic diagram of a driver status recognition device according to an embodiment of the present invention; Figure 11 FIG. 1 is a schematic diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0019] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0020] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0021] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0022] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0023] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically stated. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically stated.

[0024] The present invention discloses a driver state recognition method, apparatus, computer device, and storage medium. These methods include obtaining a driver's driving state image; extracting feature point data from the driving state image, including hand feature points; inputting the hand feature points into a driving behavior classification model to classify the driver's driving behavior and determine the driver's behavior category; and determining that the driver is in a distracted state if the behavior category belongs to a preset target distraction behavior. This method, based on hand feature points extracted from the driver's driving state image, determines whether the driver is in a distracted state. Compared to existing technologies, the present invention enriches driver state recognition results and improves vehicle driving safety. This will be described below using specific embodiments.

[0025] In one embodiment, if Figure 1As shown, a driver status recognition method is provided. The method is applicable to vehicles with specific driver image acquisition capabilities and image processing capabilities, including but not limited to sedans, SUVs, and trucks. The method includes the following steps: S101: Acquire a driving status image of the driver.

[0026] In a specific implementation, this embodiment can capture images of the driver's driving status using a camera installed in the vehicle. Cameras include, but are not limited to, driver monitoring system (DMS) cameras. DMS cameras can be either visible light cameras, infrared cameras, or a combination of both. When the vehicle starts, the DMS camera is activated simultaneously and continuously captures images of the driver's driving status at a preset capture frequency. Alternatively, in this embodiment, the DMS camera can be activated when the vehicle's real-time speed exceeds a preset speed and continuously capture images of the driver's driving status at a preset capture frequency. The DMS camera's capture frequency can be 1fps, 2fps, etc., and the preset speed can be 10km / h, 15km / h, etc. In this embodiment, the specific values of the DMS camera's capture frequency and preset speed are not limited.

[0027] S102: Extracting feature point data from the driving state image, where the feature point data includes hand feature points.

[0028] Among them, the hand feature points are the key points of the hand in the driving state image, such as the key points corresponding to the wrist point, each joint and fingertip position of the thumb, index finger, middle finger, ring finger and little finger. The key points of the hand can be as follows Figure 2 As shown, it includes 20 key points of the human hand.

[0029] Each time the DMS camera acquires a frame of driving status image, it extracts the driver's feature point data from the driving status image, and the feature point data includes the driver's hand feature points.

[0030] In a specific implementation, this embodiment can use a feature point extraction model to extract feature point data, including hand feature points, from the driving state image. Feature point extraction models include, but are not limited to, the YOLOv8-Pose model and the OpenPose model. This embodiment does not limit the feature point extraction model.

[0031] In one embodiment, the feature point extraction model in this embodiment can be trained in the following manner: obtaining a training image that has been marked with feature points, performing image enhancement on the training image through random rotation, affine transformation, color transformation, etc. to increase the data volume of the training data, and then pre-training the feature point extraction model based on the training data. After the pre-training is completed, the feature points in the collected infrared training image are predicted, and the feature points in the infrared training image are manually corrected, and the image enhancement is performed on the corrected infrared training image to increase the data volume of the infrared training image. Finally, the pre-trained feature point extraction model is fine-tuned based on the infrared training image, thereby obtaining a trained feature point extraction model.

[0032] S103: Inputting the hand feature points into a driving behavior classification model to classify the driving behavior and obtain the driver's behavior category.

[0033] In a specific implementation, the driving behavior classification model in this embodiment includes, but is not limited to, an SVM classifier model, a multi-layer perceptron (MLP), and a convolutional neural network (CNN). The hand feature points are input into the driving behavior classification model to obtain a probability value corresponding to each driving behavior, and the driving behavior with the largest probability value is selected as the driver's behavior type. It should be noted that the hand feature points in this embodiment include their initial coordinates. The coordinates of the hand feature points are converted from the image coordinate system to a coordinate system with the thumb as the origin to obtain the transformed coordinates of the hand feature points. The hand feature points and the corresponding transformed coordinates are then input into the driving behavior classification model to perform driving behavior classification and obtain the driver's behavior category.

[0034] In one embodiment, the driving behavior classification model in this embodiment can be trained in the following manner: obtaining a training image; marking the hand feature points and the corresponding feature point coordinates in the training image, and marking the classification category label corresponding to the training image; the feature point coordinates are the coordinates corresponding to the coordinate system with the thumb feature point as the coordinate origin; the training image marked with the classification label, the hand feature points and the corresponding feature point coordinates is input into the driving behavior classification model to obtain the predicted category label of the training image; based on the predicted category label and the classification category label, the classification loss value is calculated; when the classification loss value reaches the preset training requirement, it is determined that the driving behavior classification model is fully trained.

[0035] S104: If the behavior category belongs to the preset target distraction behavior, it is determined that the driver is in a distracted state.

[0036] If the behavior category belongs to the preset target distraction behavior, it is determined that the driver is in a distracted state; if the behavior category does not belong to the preset target distraction behavior, it is determined that the driver is in a normal driving state.

[0037] In a specific implementation, the preset target avatar behaviors in this embodiment include, but are not limited to, any one or more of smoking, playing with a phone, drinking water, and eating. If the driver's behavior category falls into any of these categories, the driver is determined to be distracted. If the driver's behavior category does not fall into any of the target distracting behaviors, the driver is considered to be driving normally. This allows for accurate driver status identification.

[0038] It should be noted that if no hand feature points are extracted from the driving status image, it can be assumed that the driver is in a normal state, that is, the driver is not distracted. If hand feature points are extracted from the driving status image, the driving behavior is classified based on the hand feature points to obtain the driver's behavior category, and whether the driver is distracted is determined based on the driver's behavior category.

[0039] As an extension, this embodiment determines whether the driver is distracted based on each captured driving state image. If, among a small number of consecutively captured driving state images, the driver's behavior is classified as the target distracting behavior during driver state recognition, the driver is not considered distracted. Only when the driver's behavior is classified as the target distracting behavior during driver state recognition for a large number of driving state images is the driver considered distracted. This avoids misjudgments that could affect the driver's driving experience. Specifically, within a preset time period, if the number of driver state images with the target distracting behavior exceeds a preset threshold, the driver is considered distracted. If the number of driver state images with the target distracting behavior is less than or equal to the preset threshold, the driver is considered not distracted.

[0040] For example, taking a preset duration of 3 seconds, 6 driving status images can be acquired within 3 seconds. The timing starts from the time when the first image that recognizes the driver's behavior category as the target distraction behavior is acquired. If 6 driving status images are acquired within 3 seconds, if only 1 of the 6 driving status images recognizes the driver's behavior category as the target distraction behavior, the driver is not considered to be in a distracted state. If 4 or more of the 6 driving status images recognize the driver's behavior category as the target distraction behavior, the driver is considered to be in a distracted state. In this way, the accuracy of driving status recognition can be effectively improved.

[0041] In summary, the driver status recognition method disclosed in this embodiment extracts the driver's hand feature points from the acquired driving image and determines whether the driver is in a distracted state based on the hand feature points. Compared with identifying whether the driver is in a fatigue state only through facial recognition, this method achieves the purpose of enriching the driver status recognition results and effectively improves the safety of vehicle driving.

[0042] In one embodiment, based on Figure 1 In the specific implementation, the feature point data in this embodiment also includes facial feature points. After step S102, this embodiment may further include the following steps: Figure 3 As shown: S105: extracting a preset first feature point from the hand feature points, and extracting a preset second feature point from the facial feature points.

[0043] Among them, facial feature points are key points of the face in the driving state image, including key points corresponding to key areas such as eyebrows, eyes, nose, mouth, and facial contours. The key points of the face can be as follows Figure 4 As shown, it includes 68 key points of the face.

[0044] The first feature point can be any feature point of the hand, such as the wrist point or the feature point corresponding to the thumb vertex, which is not limited in this embodiment. The second feature point can be any feature point of the face, such as the feature point corresponding to the chin point or the nose tip point.

[0045] In a specific implementation, in this embodiment, the first feature point and the second feature point can be pre-specified. When necessary, the preset first feature point is directly extracted from the hand feature points, and the preset second feature point is extracted from the facial feature points.

[0046] S106: Calculate the target distance between the human hand and the human face according to the first coordinate corresponding to the first feature point and the second coordinate corresponding to the second feature point.

[0047] In a specific implementation, the first coordinate corresponding to the first feature point in this embodiment can be the pixel coordinate of the first feature point in the driving status image, and the second coordinate corresponding to the second feature point can be the pixel coordinate of the second feature point in the driving status image. The Euclidean distance between the first coordinate and the second coordinate is calculated as the target distance between the hand and the face.

[0048] The calculation formula of Euclidean distance is as follows: in, represents the first coordinate, represents the second coordinate, Indicates the target distance.

[0049] S107: Determine whether the target distance is greater than a preset distance threshold.

[0050] S108: If the target distance is greater than the distance threshold, it is determined that the driver is in a normal state.

[0051] S109: If the target distance is less than or equal to the distance threshold, it is determined that the driver is in a distracted state.

[0052] If the target distance is greater than the distance threshold, it means that the human hand is far away from the face. At this time, it is considered that the driver is not engaged in distracting behaviors such as smoking, drinking water, or eating, and the driver is determined to be in a normal state. If the target distance is less than or equal to the distance threshold, it means that the human hand is close to the face. The driver is likely to be engaged in distracting behaviors such as smoking, drinking water, or eating, and the driver is determined to be in a distracted state.

[0053] To sum up, in this embodiment, by calculating the distance between the human hand and the face, it is further determined whether the driver is in a distracted state. Combined with the driver state recognition based on the human hand feature points, the accuracy and rationality of driver recognition can be further improved, which is conducive to improving the driver's driving experience.

[0054] In one embodiment, after step S102, the present embodiment may further include the following steps: Figure 5 As shown: S110: Calculating the driver's head posture data based on the facial feature points.

[0055] In a specific implementation, the head posture data in this embodiment includes but is not limited to any one or more of eye aspect ratio, head turning parameter, head lowering parameter, and mouth aspect ratio.

[0056] S111: Determine the driver's head posture based on the posture data.

[0057] In a specific implementation, the head posture in this embodiment includes but is not limited to any one or more of closing eyes, turning head, lowering head and yawning. When the driver's eye width-to-height ratio is less than a preset first threshold, the driver's head posture is determined to be closing eyes. Otherwise, if the eye width-to-height ratio is greater than the preset first threshold, the driver's head posture is determined to be normal; when the driver's head turning parameter is greater than a preset second threshold, the driver's head posture is determined to be turning head. Otherwise, when the head turning parameter is less than or equal to the second threshold, the driver's head posture is determined to be normal; when the driver's head lowering parameter is less than a preset third threshold, the driver's head posture is determined to be lowering head. Otherwise, when the head turning parameter is greater than or equal to the third threshold, the driver's head posture is determined to be normal; when the driver's mouth width-to-height ratio is greater than a preset fourth threshold, the driver's head posture is determined to be yawning. Otherwise, when the head turning parameter is less than or equal to the fourth threshold, the driver's head posture is determined to be normal. Wherein, the specific values of the first threshold, the second threshold, the third threshold and the fourth threshold in this embodiment are set according to actual needs and are not limited in this embodiment.

[0058] S112: If the head posture is a preset target fatigue posture, it is determined that the driver is in a fatigue state.

[0059] When the driver's head posture is any one of closing eyes, turning head, lowering head and yawning, the driver is determined to be in a fatigue state. That is to say, only when all the posture data of the driver's head, namely the eye aspect ratio, head turning parameters, head lowering parameters and mouth aspect ratio are all normal, the driver is considered to be in a normal state.

[0060] As an extension, this embodiment determines whether the driver is fatigued based on each captured driving status image. If, among continuously captured driving status images, the driver's head posture matches the preset target fatigue posture only when driver status recognition is performed for a small number of driving status images, the driver is not considered fatigued. Only when the driver's head posture matches the target fatigue posture for a large number of driving status images is the driver considered distracted. This avoids misjudgments that could affect the driver's driving experience. For example, within a preset time period, if the number of driver status images with the target fatigue posture identified exceeds a preset threshold, the driver is considered distracted. If the number of driver status images with the target fatigue posture identified is less than or equal to the preset threshold, the driver is considered not distracted.

[0061] In summary, a driver state recognition method disclosed in this embodiment extracts the driver's facial feature points and hand feature points from the acquired driving image, and judges whether the driver is in a fatigue state based on the facial feature points, judges whether the driver is in a distracted state based on the hand feature points, and judges whether the driver is in a distracted state based on the facial feature points and the hand feature points at the same time, thereby realizing the recognition of the driver's driving state. Compared with identifying whether the driver is in a fatigue state only through facial recognition, the purpose of enriching the driver state recognition results is achieved, and the safety of vehicle driving is effectively improved.

[0062] In one embodiment, the eye aspect ratio in this embodiment is calculated as follows: Figure 6 As shown: S601: Extracting eyelid feature points and eye corner feature points from facial feature points.

[0063] It should be noted that the eye aspect ratio in this embodiment may include the right eye aspect ratio and the left eye aspect ratio. The right eye aspect ratio and the left eye aspect ratio may be calculated separately. In this embodiment, only the calculation process of the left eye aspect ratio is used as an example for explanation.

[0064] In a specific implementation, the eyelid feature points in this embodiment include upper eyelid feature points and lower eyelid feature points, and the eye corner feature points include left eye corner feature points and right eye corner feature points. Figure 4 As shown, the upper eyelid feature point and the lower eyelid feature point may be P38 and P42, or the upper eyelid feature point and the lower eyelid feature point may be P39 and P41; the left eye corner feature point and the right eye corner feature point may be P37 and P40.

[0065] S602: Calculate the eye height according to the eyelid coordinates corresponding to the eyelid feature points.

[0066] The eyelid coordinates corresponding to the eyelid feature points refer to the pixel coordinates of the eyelid feature points in the driving state image.

[0067] In a specific implementation, the eyelid coordinates corresponding to the eyelid feature points in this embodiment include upper eyelid coordinates corresponding to upper eyelid feature points and lower eyelid coordinates corresponding to lower eyelid feature points. The Euclidean distance between the upper eyelid coordinates and the lower eyelid coordinates is calculated to obtain the eye width. For example, the Euclidean distance between P38 and P42 is calculated to obtain the eye width, or the Euclidean distance between P42 and P41 is calculated to obtain the eye height.

[0068] S603: Calculate the eye width according to the eye corner coordinates corresponding to the eye corner feature points.

[0069] The eye corner coordinates corresponding to the eye corner feature points refer to the pixel coordinates of the eye corner feature points in the driving state image.

[0070] In a specific implementation, the eye corner coordinates corresponding to the eye corner feature points in this embodiment include the left eye corner coordinates corresponding to the left eye corner feature point and the right eye corner coordinates corresponding to the right eye corner feature point. The Euclidean distance between the left eye corner coordinates and the right eye corner coordinates is calculated to obtain the eye width. For example, the Euclidean distance between P37 and P40 is calculated to obtain the eye width.

[0071] S604: Determine the eye aspect ratio according to the eye height and the eye width.

[0072] In a specific implementation, the eye aspect ratio refers to the ratio of eye width to eye height. The greater the eye height, the smaller the eye aspect ratio. It is understandable that when a driver is fatigued, the eye height will decrease, and thus the eye aspect ratio will increase. Conversely, when a driver is alert, the eye height will increase, and thus the eye aspect ratio will decrease. Therefore, the eye aspect ratio can be used to determine whether the driver is fatigued.

[0073] It should be noted that when both the left eye aspect ratio and the right eye aspect ratio indicate that the driver is in a fatigued state, the driver is considered to be in a fatigued state. When either the left eye aspect ratio or the right eye aspect ratio indicates that the driver is in a fatigued state, the driver is not considered to be in a fatigued state.

[0074] To sum up, in this embodiment, the distance between facial feature points is calculated to determine whether the driver is in a fatigue state. Compared with the classification model used for classification to determine whether the driver is in a fatigue state, this can reduce the device's rigid demand for computing power resources.

[0075] In one embodiment, the rotor parameters in this embodiment are calculated as follows: Figure 7 As shown: S701: Extracting nose tip feature points, left cheek feature points, and right cheek feature points from facial feature points.

[0076] refer to Figure 4 As shown, the nose tip feature point may be P34, the left cheek feature point may be P3, and the right cheek feature point may be P15.

[0077] S702: Calculate a first parameter based on the nose tip coordinates corresponding to the nose tip feature point and the left cheek coordinates corresponding to the left cheek feature point.

[0078] The nose tip coordinates corresponding to the nose tip feature point are the pixel coordinates of the nose tip feature point in the driving state image, and the left cheek coordinates corresponding to the left cheek feature point are the pixel coordinates of the left cheek feature point in the driving state image.

[0079] In a specific implementation, the first parameter in this embodiment may be the Euclidean distance between the nose tip coordinate and the left cheek coordinate.

[0080] S703: Calculate a second parameter based on the nose tip coordinates and the right cheek coordinates corresponding to the right cheek feature points.

[0081] The right cheek coordinates corresponding to the right cheek feature point are the pixel coordinates of the right cheek feature point in the driving state image.

[0082] In a specific implementation, the second parameter in this embodiment may be the Euclidean distance between the nose tip coordinate and the right cheek coordinate.

[0083] S704: Calculate the head rotation parameter according to the first parameter and the second parameter.

[0084] In a specific implementation, the head-turn parameter in this embodiment is the absolute value of the difference between the first parameter and the second parameter. When the driver does not turn their head, the first parameter and the second parameter are nearly equal, and thus the head-turn parameter is smaller. Conversely, when the driver turns their head, the difference between the first parameter and the second parameter increases, and thus the head-turn parameter is larger. Therefore, based on the size of the head-turn parameter, it is possible to determine whether the driver has turned their head.

[0085] To sum up, in this embodiment, the distance between the nose tip feature point, the left cheek feature point and the right cheek feature point is calculated to determine whether the driver is in a fatigue state. Compared with the classification model used for classification to determine whether the driver is in a fatigue state, the device's rigid demand for computing power resources can be reduced.

[0086] In one embodiment, the head-down parameter in this embodiment is calculated as follows: Figure 8 As shown: S801: Extracting nose tip feature points and chin feature points from facial feature points.

[0087] Among them, reference Figure 4 As shown, the nose tip feature point may be P34, and the chin feature point may be P9.

[0088] S802: Calculate the head-down parameters according to the nose tip coordinates corresponding to the nose tip feature points and the chin coordinates corresponding to the chin feature points.

[0089] The nose tip coordinates corresponding to the nose tip feature point are the pixel coordinates of the nose tip feature point in the driving state image, and the chin coordinates corresponding to the chin feature point are the pixel coordinates of the chin feature point in the driving state image.

[0090] In a specific implementation, the head-down parameter in this embodiment is the Euclidean distance between the nose tip coordinates and the chin coordinates. When a driver is fatigued, they will lower their head, causing the Euclidean distance between the nose tip and chin coordinates in the driving state image to decrease. When a driver is not fatigued, they generally do not lower their head. In this case, the Euclidean distance between the nose tip and chin coordinates in the driving state image will be larger than when they are lowering their head. Therefore, the head-down parameter can be used to determine whether the driver is fatigued.

[0091] To summarize, in this embodiment, the Euclidean distance between the nose tip feature point and the chin feature point is calculated to determine whether the driver is in a fatigue state. Compared with using a classification model to classify whether the driver is in a fatigue state, this can reduce the device's rigid demand for computing resources.

[0092] In one embodiment, the width-to-height ratio of the mouth in this embodiment is calculated as follows: Figure 9 As shown: S901: Extracting lip feature points and mouth corner feature points from facial feature points.

[0093] In a specific implementation, the lip feature points in this embodiment include upper lip feature points and lower lip feature points, and the mouth corner feature points include left mouth corner feature points and right mouth corner feature points. Figure 4 As shown, the upper lip feature point and the lower lip feature point may be P63 and P67; the left mouth corner feature point and the right mouth corner feature point may be P61 and P65.

[0094] S902: Calculate the mouth height based on the lip coordinates corresponding to the lip feature points.

[0095] The lip coordinates corresponding to the lip feature points refer to the pixel coordinates of the lip feature points in the driving state image.

[0096] In a specific implementation, the lip coordinates corresponding to the lip feature points in this embodiment include the upper lip coordinates corresponding to the upper lip feature points and the lower lip coordinates corresponding to the lower lip feature points. The Euclidean distance between the upper lip coordinates and the lower lip coordinates is calculated to obtain the mouth height.

[0097] S903: Calculate the mouth width according to the mouth corner coordinates corresponding to the mouth corner feature points.

[0098] The mouth corner coordinates corresponding to the mouth corner feature points refer to the pixel coordinates of the mouth corner feature points in the driving state image.

[0099] In a specific implementation, the mouth corner coordinates corresponding to the mouth corner feature points in this embodiment include the left mouth corner coordinates corresponding to the left mouth corner feature points and the right mouth corner coordinates corresponding to the right mouth corner feature points. The Euclidean distance between the left mouth corner coordinates and the right mouth corner coordinates is calculated to obtain the mouth width.

[0100] S904: Determine the width-to-height ratio of the mouth according to the mouth height and the mouth width.

[0101] In practice, the mouth aspect ratio refers to the ratio of mouth width to mouth height. The greater the mouth height, the smaller the ratio. As you can see, when a driver is fatigued, they may yawn. At this point, the mouth height increases, resulting in a smaller mouth aspect ratio. Conversely, when a driver is alert, the mouth height decreases, resulting in a larger eye aspect ratio. Therefore, the mouth aspect ratio can be used to determine driver fatigue.

[0102] To sum up, in this embodiment, the distance between facial feature points is calculated to determine whether the driver is in a fatigue state. Compared with the classification model used for classification to determine whether the driver is in a fatigue state, this can reduce the device's rigid demand for computing power resources.

[0103] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0104] In one embodiment, a driver status recognition device is provided, which corresponds one-to-one with the driver status recognition method in the above embodiment. Figure 10 As shown, the driver status recognition device includes an image acquisition module, a feature point extraction module, a behavior classification module, and a status determination module. The functional modules are described in detail as follows: An image acquisition module 1001 is used to acquire an image of the driver's driving status; A feature point extraction module 1002 is used to extract feature point data from the driving state image, where the feature point data includes hand feature points; The behavior classification module 1003 is used to input the hand feature points into the driving behavior classification model to classify the driver's behavior and obtain the driver's behavior category; The first state determination module 1004 is configured to determine that the driver is in a distracted state when the behavior category belongs to a preset target distracting behavior.

[0105] In one embodiment, the feature point data further includes facial feature points, and further includes a second state recognition module for: Extracting a preset first feature point from the hand feature points, and extracting a preset second feature point from the facial feature points; Calculate the target distance between the hand and the face based on the first coordinate corresponding to the first feature point and the second coordinate corresponding to the second feature point; Determine whether the target distance is greater than a preset distance threshold; If the target distance is greater than the distance threshold, it is determined that the driver is in a normal state; If the target distance is less than or equal to the distance threshold, it is determined that the driver is in a distracted state.

[0106] In one embodiment, a third state recognition module is further included, which is configured to: Calculate the driver's head posture data based on facial feature points; Determine the driver's head posture based on the posture data; If the head posture is a preset target fatigue posture, it is determined that the driver is in a fatigue state.

[0107] In one embodiment, the posture data includes the eye aspect ratio, which is calculated as follows: Extract eyelid feature points and eye corner feature points from facial feature points; Calculate the eye height based on the eyelid coordinates corresponding to the eyelid feature points; Calculate the eye width based on the eye corner coordinates corresponding to the eye corner feature points; Determine the eye aspect ratio based on eye height and eye width.

[0108] In one embodiment, the posture data includes a head turning parameter, which is calculated as follows: Extract nose tip feature points, left cheek feature points and right cheek feature points from facial feature points; Calculate the first parameter based on the nose tip coordinates corresponding to the nose tip feature point and the left cheek coordinates corresponding to the left cheek feature point; Calculate the second parameter based on the coordinates of the nose tip and the right cheek coordinates corresponding to the right cheek feature points; The head rotation parameter is calculated according to the first parameter and the second parameter.

[0109] In one embodiment, the posture data includes a head-down parameter, which is calculated as follows: Extract nose tip feature points and chin feature points from facial feature points; The head-down parameters are calculated based on the nose tip coordinates corresponding to the nose tip feature points and the chin coordinates corresponding to the chin feature points.

[0110] In one embodiment, the posture data includes the mouth width-to-height ratio, which is calculated as follows: Extract lip feature points and mouth corner feature points from facial feature points; Calculate the mouth height based on the lip coordinates corresponding to the lip feature points; The mouth width is calculated based on the coordinates of the mouth corners corresponding to the mouth corner feature points; Determine the width-to-height ratio of the mouth based on its height and width.

[0111] In summary, the driver state recognition device disclosed in this embodiment extracts the driver's facial feature points and hand feature points from the acquired driving image to determine whether the driver is in a fatigue state based on the facial feature points, determines whether the driver is in a distracted state based on the hand feature points, and simultaneously determines whether the driver is in a distracted state based on the facial feature points and the hand feature points. In this way, the driver's driving state can be recognized. Compared with identifying whether the driver is in a fatigue state only through facial recognition, the purpose of enriching the driver state recognition results is achieved, and the safety of vehicle driving is effectively improved.

[0112] The specific definitions of the driver status recognition device can be found in the definitions of the driver status recognition method above and will not be repeated here. Each module in the aforementioned driver status recognition device may be implemented in whole or in part via software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in hardware form, or may be stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0113] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 11 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a driver status recognition method is implemented.

[0114] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the driver status identification method in the above embodiment is implemented, for example Figure 1 The driver status recognition method shown, or Figures 2 to 9Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the driver status recognition device are realized, for example Figure 10 The functions of the driver status recognition shown are not described here in detail to avoid repetition.

[0115] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the driver status recognition method in the above embodiment is implemented, for example Figure 1 Driver status identification shown, or Figures 2 to 9 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the driver status recognition device are realized, for example Figure 10 The functions of the driver status recognition shown are not described here in detail to avoid repetition.

[0116] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0117] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0118] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A driver status recognition method, characterized in that: include: Acquiring a driving status image of the driver; extracting feature point data from the driving state image, the feature point data including hand feature points; Inputting the hand feature points into a driving behavior classification model to perform driving behavior classification to obtain the driver's behavior category; If the behavior category belongs to a preset target distraction behavior, it is determined that the driver is in a distracted state.

2. The driver status recognition method according to claim 1, characterized in that: The feature point data also includes facial feature points; After extracting feature point data from the driving state image, the method further includes: Extracting a preset first feature point from the hand feature points, and extracting a preset second feature point from the facial feature points; Calculating a target distance between a hand and a face based on a first coordinate corresponding to the first feature point and a second coordinate corresponding to the second feature point; Determining whether the target distance is greater than a preset distance threshold; If the target distance is greater than the distance threshold, determining that the driver is in a normal state; If the target distance is less than or equal to the distance threshold, it is determined that the driver is in a distracted state.

3. The driver status recognition method according to claim 2, characterized in that: After extracting feature point data from the driving state image, the method further includes: Calculating the driver's head posture data based on the facial feature points; determining the driver's head posture according to the posture data; If the head posture is a preset target fatigue posture, it is determined that the driver is in a fatigue state.

4. The driver status recognition method according to claim 3, characterized in that: The posture data includes the eye aspect ratio, which is calculated as follows: Extracting eyelid feature points and eye corner feature points from the facial feature points; Calculating the eye height according to the eyelid coordinates corresponding to the eyelid feature points; Calculating the eye width according to the eye corner coordinates corresponding to the eye corner feature points; The eye aspect ratio is determined according to the eye height and the eye width.

5. The driver status recognition method according to claim 3, characterized in that: The posture data includes head turning parameters, which are calculated as follows: Extracting nose tip feature points, left cheek feature points, and right cheek feature points from the facial feature points; Calculating a first parameter based on the nose tip coordinates corresponding to the nose tip feature point and the left cheek coordinates corresponding to the left cheek feature point; Calculating a second parameter based on the nose tip coordinates and the right cheek coordinates corresponding to the right cheek feature points; The rotor parameter is obtained by calculation according to the first parameter and the second parameter.

6. The driver status recognition method according to claim 3, characterized in that: The posture data includes a head-down parameter, which is calculated as follows: Extracting nose tip feature points and chin feature points from the facial feature points; The head-lowering parameter is calculated according to the nose tip coordinates corresponding to the nose tip feature points and the chin coordinates corresponding to the chin feature points.

7. The driver status recognition method according to claim 3, characterized in that: The posture data includes the width-to-height ratio of the mouth, which is calculated as follows: Extracting lip feature points and mouth corner feature points from the facial feature points; Calculating the mouth height according to the lip coordinates corresponding to the lip feature points; Calculating the mouth width according to the mouth corner coordinates corresponding to the mouth corner feature points; The mouth aspect ratio is determined according to the mouth height and the mouth width.

8. A driver status recognition device, characterized in that: include: An image acquisition module, used to acquire an image of the driver's driving status; A feature point extraction module, configured to extract feature point data from the driving state image, wherein the feature point data includes hand feature points; A behavior classification module, configured to input the hand feature points into a driving behavior classification model to classify the driver's behavior and obtain the driver's behavior category; The first state determination module is configured to determine that the driver is in a distracted state when the behavior category belongs to a preset target distracting behavior.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the driver status identification method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the driver status identification method according to any one of claims 1 to 7 is implemented.

Images