Fatigue driving detection method, electronic device and medium thereof

Abstract

The application relates to the field of intelligent driving, and discloses a fatigue driving detection method, an electronic device and a medium. The fatigue driving detection method comprises the following steps: acquiring a face image of a driver in a vehicle; determining key part data of the face of the driver in the face image; determining whether feature information of each key part meets a first fatigue state condition based on the key part data; and determining that the driver is in fatigue driving when the feature information corresponding to at least one key part meets the first fatigue state condition. Whether the driver is in fatigue driving can be determined according to changes in a local area corresponding to a key part of the face of the driver, the algorithm for fatigue driving detection is simple and efficient, the recognition rate is high, and the inconvenience and misjudgment caused by fatigue driving detection through special equipment for collecting physiological characteristics of the driver or through driving data of the vehicle are avoided.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and in particular to a fatigue driving detection method and its electronic equipment and medium. Background Technology

[0002] As people's living standards improve, cars have become an essential means of transportation for most families. With the increase in the number of cars, traffic accidents are also increasing year by year. Among them, accidents caused by driver fatigue account for a very high proportion. Therefore, helping drivers reduce driver fatigue and develop correct safe driving behaviors is of great significance to ensuring road traffic safety and protecting personal life and property.

[0003] Existing methods for detecting driver fatigue include those based on the driver's physiological characteristics and those based on vehicle driving data. These methods often require specialized equipment and are prone to misjudgment under special circumstances (such as a driver with a specific medical condition or complex road conditions), resulting in low reliability. Therefore, a simple and highly accurate method for detecting driver fatigue is needed. Summary of the Invention

[0004] This application provides a fatigue driving detection method, its electronic equipment, and a medium.

[0005] In a first aspect, this application provides a fatigue driving detection method applied to electronic devices, comprising:

[0006] Acquire facial images of the driver in the vehicle;

[0007] Determine key feature data of a driver's face in a face image, wherein the key feature data is used to define a local region that includes at least a portion of the key features of the driver's face;

[0008] Based on the data of key parts, determine whether the characteristic information of each key part meets the conditions of the first fatigue state.

[0009] If the feature information corresponding to at least one key part satisfies the first fatigue state condition, it is determined that the driver is driving while fatigued.

[0010] In this application, it is possible to determine whether a driver is driving while fatigued based on changes in local areas corresponding to key facial features. The fatigue detection algorithm is simple, efficient, and has a high recognition rate, avoiding the inconvenience and misjudgment issues associated with requiring drivers to wear special equipment to collect physiological characteristics or using vehicle driving data for fatigue detection.

[0011] On the other hand, key parts include the driver's eyes, the local area corresponding to the eyes is the eye region, and the characteristic information of the eyes is the proportion of the white area or the black area.

[0012] On the other hand, based on data from key components, it is determined whether the characteristic information of each key component meets the conditions for the first fatigue state, including:

[0013] Obtain the first pixel value of a pixel within a local area of ​​the eye;

[0014] The obtained first pixel value is binarized to obtain the second pixel value;

[0015] Based on the second pixel value, determine the proportion of the white area or the black area of ​​the eye;

[0016] If the proportion of sclera or iris area meets the proportion condition, and the number of historical face images that meet the proportion condition is greater than the number threshold within the first historical time period, then the eye feature information is confirmed to meet the first fatigue state condition.

[0017] On the other hand, the proportion conditions include: the proportion of the white area of ​​the eye is greater than the first proportion threshold or the proportion of the black area of ​​the eye is less than the second proportion threshold.

[0018] On the other hand, the key part includes the driver's mouth, the local area corresponding to the mouth is the mouth area, and the characteristic information corresponding to the mouth is the outline curvature of the mouth.

[0019] On the other hand, based on data from key components, it is determined whether the characteristic information of each key component meets the conditions for the first fatigue state, including:

[0020] Obtain the position information of pixels on the outline of the mouth region;

[0021] Based on the obtained location information, the contour curvature of the user's mouth is determined;

[0022] If the contour curvature is determined to be greater than the curvature threshold, the feature information of the mouth is determined to satisfy the first fatigue state condition.

[0023] On the other hand, the key part includes the driver's nose; and the corresponding local area is the nose area.

[0024] On the other hand, based on data from key components, it is determined whether the characteristic information of each key component meets the conditions for the first fatigue state, including:

[0025] Obtain the position information of pixels in the nose region, wherein the position information includes first position information, second position information, third position information and fourth position information;

[0026] Based on the obtained location information, the first line segment between the first location information and the second location information, the second line segment between the third location information and the second location information, and the third line segment between the fourth location information and the second location information are determined;

[0027] If the ratio of the first included angle between the first line segment and the second line segment to the second included angle between the first line segment and the third line segment is less than the first angle ratio or greater than the second angle ratio, then the feature information of the nose satisfies the first fatigue state condition.

[0028] On the other hand, the first location information includes the location information of the pixels at the bridge of the nose in the nose region, the second location information includes the location information of the pixels at the tip of the nose in the nose region, the third location information includes the location information of the pixels at the left wing of the nose in the nose region, and the fourth location information includes the location information of the pixels at the right wing of the nose in the nose region.

[0029] Secondly, this application provides a readable medium storing instructions that, when executed by an electronic device, cause the electronic device to perform the fatigue driving detection method of the first aspect.

[0030] Thirdly, this application provides an electronic device, comprising:

[0031] Memory, which stores instructions and

[0032] A processor for reading and executing instructions from memory to cause electronic devices to perform any of the fatigue driving detection methods of the first aspect. Attached Figure Description

[0033] Figure 1 According to some embodiments of this application, a schematic diagram of a fatigue driving detection scenario is shown;

[0034] Figure 2 According to some embodiments of this application, a flowchart of a fatigue driving detection method is shown;

[0035] Figure 3 According to some embodiments of this application, a schematic diagram of key points of key parts of a driver's face is shown;

[0036] Figure 4 According to some embodiments of this application, a flowchart of a fatigue driving detection method is shown;

[0037] Figure 5 According to some embodiments of this application, a flowchart of a fatigue driving detection method is shown;

[0038] Figure 6According to some embodiments of this application, a flowchart of a fatigue driving detection method is shown;

[0039] Figure 7 According to some embodiments of this application, a structural diagram of a fatigue driving detection system is shown;

[0040] Figure 8 According to some embodiments of this application, a structural diagram of an electronic device is shown. Specific Implementation

[0041] To make the objectives, embodiments, and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0042] Figure 1 This diagram illustrates an application scenario for fatigue driving detection according to some embodiments of this application. Terminal device 2 may include a device with a camera function, such as a mobile phone, tablet, camera, or in-vehicle infotainment system. Taking a camera positioned near the driver's seat as an example, while driver 1 is driving, terminal device 2 can capture images 101 of the driver's face in real time. Terminal device 2 can send images 101 to server 3. Server 3 can identify key facial features of driver 1 from the facial features in image 101, such as eyes, nose, and mouth, and determine whether fatigue driving is present. If so, for example, if eyes are closed or the mouth is yawning, the vehicle's infotainment system will alert driver 1 to pay attention to driving safety.

[0043] To address the complexity and potential for misjudgment in fatigue driving detection mentioned above, this application provides a fatigue driving detection method. The method includes: acquiring a facial image of a driver in a vehicle; determining key facial feature data from the facial image, wherein the key feature data defines a local region including at least a portion of the key facial features of the driver; based on the key feature data, determining whether the feature information of each key feature satisfies a first fatigue state condition; if the feature information corresponding to at least one key feature satisfies the first fatigue state condition, determining that the driver is fatigued; and issuing an alarm message. The key feature data here can be pixels constituting the key features in the facial image.

[0044] In some embodiments, the key parts may include: eyes, mouth, and nose, etc. For the eyes, the left and right eyes can be identified, and the local regions formed by the pixels (key points) corresponding to the contours of the left and right eyes can be determined. That is, the local regions of the key points in the face image. Further, the area occupied by black or white pixels (white or black area of ​​the eye) in the local region is determined. If the area meets a preset condition, for example, the ratio between the area occupied by black pixels and the local region is less than a preset ratio, then it is determined that the driver is in a state of fatigued driving with eyes closed. For the mouth, the local region formed by the key points corresponding to the contour of the mouth can be determined. Further, the contour curvature of the local region is determined to determine the extent to which the driver's mouth is open. If the contour curvature is greater than a preset roundness threshold, then it is determined that the driver is in a state of fatigued driving with yawning. For the nose, the local area formed by the key points from the bridge of the nose to the tip of the nose and from the tip of the nose to the sides of the nostrils can be determined. The ratio of the angle formed by the sides of the nostrils and the tip of the nose can be further determined. If the ratio of the angle is less than the first preset angle ratio or greater than the second preset angle ratio, it is determined that the driver is in a state of fatigue driving with his eyes deviating from straight ahead.

[0045] As can be seen, the fatigue driving detection method of this application can determine whether the driver is fatigued based on the changes in the local area corresponding to the key parts of the driver's face. The fatigue driving detection algorithm is simple, efficient, and has a high recognition rate. It avoids the inconvenience and misjudgment caused by requiring the driver to wear special equipment to collect the driver's physiological characteristics or to use vehicle driving data for fatigue driving detection.

[0046] The fatigue driving detection method provided in this application is described in detail below. Please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating a fatigue driving detection method provided in an embodiment of this application. This fatigue driving detection method can be used with various terminal devices that have shooting capabilities. The terminal devices may include mobile phones, in-vehicle systems, tablets, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other devices with displays. This application embodiment does not impose any restrictions on the specific type of terminal device.

[0047] In some embodiments, the terminal device can also send the captured facial image of the driver to a server, which performs fatigue driving detection on the facial image. This server may include an application server, a cloud server, etc., and this application embodiment does not limit the specific type of server. The following description uses a vehicle's infotainment system as an example to illustrate the fatigue driving detection method. Figure 2 As shown, fatigue driving detection methods may include:

[0048] S201: Obtain the facial image of the driver in the vehicle.

[0049] For example, a vehicle-mounted infotainment system can capture the driver's facial image in real time via a camera.

[0050] S202: Determine key facial features of the driver in a facial image.

[0051] For example, after acquiring a face image, facial landmark detection technology can be used to determine the key facial features of the driver in the face image. These key features can be pixels (key points) in the face image, such as… Figure 3 As shown, key feature data is used to define a local area including at least a portion of key features of the driver's face. Key features may include: the driver's eyes, mouth, and nose, etc.

[0052] In some embodiments, facial landmark detection technology may include the Dlib library, an open-source machine learning library for face detection and facial landmark detection from images.

[0053] S203: Based on the data of key parts, determine whether the characteristic information of each key part meets the conditions of the first fatigue state.

[0054] For example, the key feature information here may include: the proportion of the black or white area of ​​the eye region, the contour curvature of the mouth region, and the ratio of the angles between the two sides of the nostrils and the bridge and tip of the nose region, etc. The conditions for satisfying the first fatigue state may include: whether the driver's eyes are closed, whether the driver is yawning, whether the driver is distracted, etc. If these conditions are met, step S204 is executed to determine that the driver is fatigued and an alarm message is issued. Otherwise, the process returns to step S201.

[0055] S204: The feature information corresponding to at least one key part satisfies the first fatigue state condition, and it is determined that the driver is driving while fatigued.

[0056] For example, once it is determined that the driver is driving while fatigued, a warning message can be played through the vehicle's speaker to remind the driver to avoid traffic accidents caused by fatigued driving.

[0057] In some embodiments, different fatigue driving detection methods can be performed on different key parts of the driver. The fatigue driving detection methods corresponding to each key part are described below.

[0058] For example, such as Figure 4 As shown, the driver's eye-related fatigue detection process may include:

[0059] S401: Obtain the first pixel value of a pixel within a local area of ​​the eye.

[0060] For example, the local area corresponding to the eye is called the eye region, and the feature information of the eye is the proportion of the sclera (white area) or iris (black area). The key data of the eye here can be the pixels of the eye region of the driver's eyes (including the left and right eyes), also known as key points. The key points of the eyes are extracted from all facial key points in the face image. (Continue to refer to...) Figure 3 The key points of the eyes can correspond to Figure 3 The index numbers in the text are key points 37, 38, 39, 40, 41, 42 for the left eye and 43, 44, 45, 46, 47, 48 for the right eye.

[0061] S402: Binarize the obtained first pixel value to obtain the second pixel value.

[0062] For example, taking the left eye as an example, based on the position information of the six key points 37, 38, 39, 40, 41, and 42, the local area of ​​the left eye, that is, the left eye region, can be obtained. The shape of the left eye region can be binarized by taking the minimum bounding rectangle of the above six key points.

[0063] S403: Based on the second pixel value, determine the proportion of the white area or the black area of ​​the eye.

[0064] For example, the second pixel value here can be the pixel value of a black pixel, for example, the pixel value can be 255. This application embodiment does not specifically limit this. Taking the proportion of the black area of ​​the left eye as an example, the proportion of the black area here can be the number of black pixels, counting the number of black pixels in the binarized left eye region. Conversely, the proportion of the white area here can be the number of white pixels, counting the number of white pixels in the binarized left eye region.

[0065] In some embodiments, the proportion of the white area of ​​the eye or the black area of ​​the eye can also be determined by the ratio of the number of black pixels or white pixels to the total number of pixels in the eye region.

[0066] S404: If the proportion of the white area or the black area of ​​the eye meets the proportion condition, and the number of historical face images that meet the proportion condition is greater than the number threshold within the first historical time period, then the feature information of the eye is confirmed to meet the first fatigue state condition.

[0067] For example, if the number of black pixels is greater than the quantity threshold, it means that the proportion of the black area of ​​the eyes meets the proportion condition, and the driver is considered to be in an open-eye state; otherwise, the driver is considered to be in a closed-eye state. Step S401 continues to obtain the next face image, and the proportion of the white or black area of ​​the eyes is judged to meet the proportion condition. If, after a period of time, i.e., within the first historical time period, the number of times both the left and right eyes close reaches the quantity threshold, then the characteristic information of the driver's eyes is determined to meet the first fatigue state condition, i.e., the driver is driving while fatigued. For example, if the number of times both the left and right eyes close reaches 10 times within the first historical time period (20 seconds), then the driver is determined to be driving while fatigued.

[0068] In some embodiments, the number threshold can be determined based on the size of the eye region in the face image. For example, if the size of the eye region is 30*40, then the number threshold for black pixels can be 200, and the number threshold for black pixels can be 1200. Similarly, the threshold for the proportion of black area in the eye can be 1 / 6, and the threshold for the proportion of white area in the eye can be 5 / 6. The above values ​​are exemplary, and the embodiments of this application do not impose specific limitations.

[0069] Understandable, in passing Figure 4 The method detects driver fatigue in both the left and right eyes simultaneously, focusing on key points 43, 44, 45, 46, 47, and 48 of the right eye. After confirming driver fatigue and issuing a warning, the number of times both eyes closed within the first historical time period is reset to zero, ensuring that historical data does not affect subsequent fatigue detections.

[0070] After introducing the process of detecting driver fatigue through eye strain, for example, as follows: Figure 5 As shown, the fatigue driving detection process corresponding to the driver's mouth may include:

[0071] S501: Obtain the position information of pixels on the outline of the mouth region.

[0072] For example, the local area corresponding to the mouth is called the mouth region, and the feature information corresponding to the mouth is the contour curvature of the mouth. The key data for the mouth here could be the pixel points of the driver's mouth region. See further... Figure 3 The key points in the mouth area can correspond to Figure 3 The key points are indexed as 61, 62, 63, 64, 65, 66, 67, and 68 respectively.

[0073] S502: Based on the acquired location information, determine the contour curvature of the user's mouth.

[0074] For example, the curvature (also known as roundness) of the outline of key points in the mouth area is used to determine the driver's mouth opening range. This outline can be determined by connecting the key points in the mouth area end to end; the larger the curvature, the larger the mouth opening. The formula for calculating the curvature is as follows:

[0075]

[0076] Wherein, Round represents the curvature of the mouth region, Area represents the area of ​​the mouth region, and Peri represents the perimeter of the mouth region's outline.

[0077] S503: If the contour curvature is determined to be greater than the curvature threshold, the feature information of the mouth is determined to satisfy the first fatigue state condition.

[0078] For example, when the contour curvature is greater than the curvature threshold, it is determined that the driver is yawning, that is, it is determined that the driver is driving while fatigued, and step S501 is executed to obtain the next face image.

[0079] In some embodiments, the radius threshold of the contour radius can be 0.4. The above values ​​are exemplary and are not specifically limited in this application.

[0080] After introducing the process of detecting driver fatigue by looking into the eyes and mouth, for example, as follows: Figure 6 As shown, the fatigue driving detection process corresponding to the driver's nose may include:

[0081] S601: Obtain the position information of pixels in the nose region.

[0082] For example, the local area corresponding to the nose is called the nose region, and the key data of the nose here can be the pixels of the driver's mouth region. (Continue to refer to...) Figure 3 The key points in the nose area can correspond to Figure 3 The key points are indexed as 28, 29, 30, 31, 32, 33, 34, 35, and 36 respectively.

[0083] In some embodiments, the location information includes first location information, second location information, third location information, and fourth location information. The first location information includes the location information of pixels at the bridge of the nose in the nose region, i.e., keypoints 28, 29, and 30; the second location information includes the location information of pixels at the tip of the nose in the nose region, i.e., keypoint 31; the third location information includes the location information of pixels at the left wing of the nose in the nose region, i.e., keypoint 32; and the fourth location information includes the location information of pixels at the right wing of the nose in the nose region, i.e., keypoint 36.

[0084] S602: Based on the acquired location information, determine the first line segment between the first location information and the second location information, the second line segment between the third location information and the second location information, and the third line segment between the fourth location information and the second location information.

[0085] For example, the first line segment here can be the straight line M_Line formed by key points 28, 29, 30, and 31; the second line segment can be the straight line L_Line formed by key point 32 of the left nasal wing and key point 31 of the nasal tip; and the third line segment can be the straight line R_Line formed by key point 36 of the right nasal wing and key point 31 of the nasal tip.

[0086] S603: If the ratio of the angle between the first included angle between the first line segment and the second line segment to the angle between the first line segment and the third line segment is less than the first angle ratio or greater than the second angle ratio, the feature information of the nose is determined to satisfy the first fatigue state condition.

[0087] For example, the first included angle here can be the angle between L_Line and M_Line, i.e., the left nasal angle L_Theta, and the second included angle can be the angle between R_Line and M_Line, i.e., the right nasal angle R_Theta. The angle ratio is Theta_Ratio = L_Theta / R_theta. If Theta_Ratio is less than the first angle ratio threshold, it is determined that the driver's head is deviated to the left and their gaze is not directly forward, indicating that the driver is distracted and fatigued. Similarly, if Theta_Ratio is greater than the second angle ratio threshold, it is determined that the driver's head is deviated to the right and their gaze is not directly forward, indicating fatigued driving. It can be understood that if Theta_Ratio is between the first and second angle ratio thresholds, the driver's gaze is considered to be directly forward, which is considered normal.

[0088] In some embodiments, the first angle ratio threshold and the second angle ratio threshold can be 1.5 and 0.6, respectively. The above values ​​are exemplary and are not specifically limited in the embodiments of this application.

[0089] The following is for reference. Figure 7 , Figure 7 An example is shown of an operating system configured in a terminal device for execution. Figure 2 The software module architecture of the fatigue driving detection architecture shown in the fatigue driving detection method is described.

[0090] like Figure 7 As shown, the fatigue driving detection architecture 700 includes: a data acquisition module 701, a detection module 702, and an alert module 703.

[0091] The acquisition module 701 is used to acquire facial images of the driver in the vehicle.

[0092] The detection module 702 is used to determine the key part data of the driver's face in the face image. Based on the key part data, it determines whether the feature information of each key part meets the first fatigue state condition. If the feature information of at least one key part meets the first fatigue state condition, it is determined that the driver is driving while fatigued.

[0093] The reminder module 703 is used to issue alarm information to remind drivers to pay attention to driving safety and avoid driving while fatigued.

[0094] Now for reference Figure 8 The diagram shows a block diagram of an electronic device 800 according to an embodiment of this application. The electronic device 800 is used to perform the image restoration method proposed in the embodiments of this application. The electronic device 800 may include one or more processors 801 coupled to a controller hub 803. In at least one embodiment, the controller hub 803 communicates with the processor 801 via a multi-branch bus such as a Front Side Bus (FSB), a point-to-point interface such as a QuickPath Interconnect (QPI), or a similar connection 806. The processor 801 executes instructions controlling general-type data processing operations. In one embodiment, the controller hub 803 includes, but is not limited to, a Graphics & Memory Controller Hub (GMCH) (not shown) and an Input / Output Hub (IOH) (which may be on a separate chip) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.

[0095] Electronic device 800 may also include a coprocessor 802 and a memory 804 coupled to a controller hub 803. Alternatively, one or both of the memory and GMCH may be integrated within the processor (as described in this application), with memory 804 and coprocessor 802 directly coupled to processor 801 and controller hub 803, which is on a single chip with IOH.

[0096] Memory 804 may be, for example, Dynamic Random Access Memory (DRAM), Phase Change Memory (PCM), or a combination of both. Memory 804 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. The computer-readable storage medium stores instructions, specifically, temporary and permanent copies of those instructions. The instructions may include, when executed by at least one of the processors, causing the electronic device 800 to perform, as... Figure 2 The instructions for the method are shown. When the instructions are executed on a computer, the computer performs the message display method disclosed in the embodiments of this application.

[0097] In one embodiment, the coprocessor 802 is a dedicated processor, such as, for example, a high-throughput many-integrated core (MIC) processor, a network or communication processor, a compression engine, a graphics processor, general-purpose computing on graphics processing units (GPGPU), or an embedded processor, etc. Optional properties of the coprocessor 802 are indicated by dashed lines. Figure 6 middle.

[0098] In one embodiment, electronic device 800 may further include a Network Interface Controller (NIC) 806. The network interface 806 may include a transceiver for providing a radio interface for electronic device 800 to communicate with any other suitable device, such as a front-end module, antenna, etc. In various embodiments, the network interface 806 may be integrated with other components of electronic device 800. The network interface 806 can implement the functions of the communication unit in the above embodiments.

[0099] Electronic device 800 may further include input / output (I / O) devices 805. I / O 805 may include: a user interface designed to enable a user to interact with electronic device 800; a peripheral component interface designed to enable peripheral components to also interact with electronic device 800; and / or sensors designed to determine environmental conditions and / or location information related to electronic device 800.

[0100] It is worth noting that, Figure 8 This is merely an example. That is, although... Figure 8 The electronic device 800 shown includes multiple devices such as a processor 801, a controller hub 803, and a memory 804. However, in practical applications, devices using the methods of this application may include only a portion of the devices in the electronic device 800. For example, it may include only the processor 801 and the network interface 806. Figure 6 The properties of the optional devices are shown by dashed lines.

[0101] Figure 8 The memory 804 in the present application may be a readable medium proposed in the embodiments of the present application. The memory 804 stores instructions, which are executed by the electronic device 800 to perform the fatigue driving detection method of the present application embodiments.

[0102] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0103] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0104] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0105] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A fatigue driving detection method, applied to electronic devices, characterized in that, include: Acquire facial images of the driver in the vehicle; Determine key part data of the driver's face in the face image, wherein the key part data is used to define a local region including at least a portion of the key parts of the driver's face; Based on the data of the key parts, determine whether the feature information of each key part meets the first fatigue state condition. If the feature information corresponding to at least one of the key parts satisfies the first fatigue state condition, it is determined that the driver is driving while fatigued. The key component includes the driver's nose. Determining whether the feature information of each key component meets the first fatigue state condition based on the key component data includes: Obtain the position information of pixels in the nose region, wherein the position information includes first position information, second position information, third position information, and fourth position information; Based on the obtained location information, a first line segment between the first location information and the second location information, a second line segment between the third location information and the second location information, and a third line segment between the fourth location information and the second location information are determined. If the ratio of the angle between the first included angle between the first line segment and the second line segment to the angle between the first line segment and the third line segment is less than the first angle ratio or greater than the second angle ratio, then the feature information of the nose is determined to meet the first fatigue state condition. The first location information includes the location information of the pixels at the bridge of the nose in the nose region, the second location information includes the location information of the pixels at the tip of the nose in the nose region, the third location information includes the location information of the pixels at the left wing of the nose in the nose region, and the fourth location information includes the location information of the pixels at the right wing of the nose in the nose region.

2. The method according to claim 1, characterized in that, The key components include the driver's eyes, the local area corresponding to the eyes is the eye region, and the characteristic information corresponding to the eyes is the percentage of the white area or the percentage of the black area.

3. The method according to claim 2, characterized in that, The step of determining whether the feature information of each key component satisfies the first fatigue state condition based on the key component data includes: Obtain the first pixel value of a pixel within a local area of ​​the eye; The obtained first pixel value is binarized to obtain the second pixel value; Based on the second pixel value, determine the proportion of the white area or the black area of ​​the eye; If the proportion of the white area or the black area of ​​the eye meets the proportion condition, and the number of historical face images that meet the proportion condition is greater than the number threshold within the first historical time period, then the feature information of the eye is confirmed to meet the first fatigue state condition.

4. The method according to claim 3, characterized in that, The proportion conditions include: the proportion of the white area of ​​the eye is greater than the first proportion threshold or the proportion of the black area of ​​the eye is less than the second proportion threshold.

5. The method according to claim 1, characterized in that, The key component includes the driver's mouth, the local area corresponding to the mouth is the mouth region, and the feature information corresponding to the mouth is the contour curvature of the mouth.

6. The method according to claim 5, characterized in that, The step of determining whether the feature information of each key component satisfies the first fatigue state condition based on the key component data includes: Obtain the position information of pixels on the contour of the mouth region; Based on the obtained location information, the contour curvature of the user's mouth is determined; If the contour curvature is determined to be greater than the curvature threshold, the feature information of the mouth is determined to satisfy the first fatigue state condition.

7. The method according to claim 1, characterized in that, The local area corresponding to the nose is the nose region.

8. A readable medium, characterized in that, The readable medium stores instructions that, when executed by an electronic device, enable the electronic device to perform the fatigue driving detection method as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: Memory, wherein instructions are stored, and A processor is configured to read and execute instructions in the memory to cause the electronic device to perform the fatigue driving detection method as described in any one of claims 1-7.

Images