Fatigue driving detection methods and related products

Abstract

The present application discloses a fatigue driving detection method and related products, wherein the method includes: obtaining an image to be processed, wherein the image to be processed includes a target object; performing target extraction on the image to be processed to obtain a target area image corresponding to the target object; performing feature extraction on the target area image to obtain a target feature set, wherein the target feature set includes a target expression, a target eye feature set, and a target skin parameter; determining a target fatigue level based on the target feature set; and performing a prompt operation when the target fatigue level exceeds a preset threshold. The present application embodiment realizes accurate fatigue driving detection and early warning operation, which helps to ensure driving safety.

Description

Technical Field

[0001] The present application relates to the technical field of fatigue driving detection, and specifically to a fatigue driving detection method and related products. Background Art

[0002] Fatigue driving generally refers to driving fatigue. Fatigue driving is highly likely to cause traffic accidents. It refers to the physiological and psychological imbalances that occur after prolonged driving, leading to a decline in driving skills. Drivers with poor or insufficient sleep and prolonged driving experience a lack of endogenous oxygen, which can lead to fatigue. Therefore, the problem of fatigue driving detection remains urgent. Summary of the Invention

[0003] The embodiments of the present application provide a fatigue driving detection method and related products, which can realize fatigue driving detection and improve driving safety.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting fatigue driving, the method comprising:

[0005] Acquire an image to be processed, wherein the image to be processed includes a target object;

[0006] Performing target extraction on the image to be processed to obtain a target area image corresponding to the target object;

[0007] Performing feature extraction on the target area image to obtain a target feature set, wherein the target feature set includes a target expression, a target human eye feature set, and a target skin parameter;

[0008] determining a target fatigue level based on the target feature set;

[0009] When the target fatigue level is greater than a preset threshold, a prompt operation is performed.

[0010] In a second aspect, an embodiment of the present application provides a fatigue driving detection device, the device comprising: an acquisition unit, an extraction unit, a determination unit, and a prompt unit, wherein:

[0011] The acquisition unit is configured to acquire an image to be processed, wherein the image to be processed includes a target object;

[0012] The extraction unit is used to extract the target from the image to be processed to obtain a target area image corresponding to the target object;

[0013] The extraction unit is further configured to perform feature extraction on the target area image to obtain a target feature set, wherein the target feature set includes a target expression, a target eye feature set, and target skin parameters;

[0014] The determining unit is configured to determine target fatigue based on the target feature set;

[0015] The prompt unit is used to perform a prompt operation when the target fatigue level is greater than a preset threshold.

[0016] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program comprises instructions for executing the steps of any method of the first aspect of the embodiment of the present application.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the above-mentioned computer-readable storage medium stores a computer program for electronic data exchange, wherein the above-mentioned computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0018] In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0019] The embodiments of the present application have the following beneficial effects:

[0020] It can be seen that the fatigue driving detection method and related products described in the embodiments of the present application obtain an image to be processed, which includes a target object, perform target extraction on the image to be processed, obtain a target area image corresponding to the target object, perform feature extraction on the target area image, and obtain a target feature set. The target feature set includes a target expression, a target human eye feature set, and a target skin parameter. The target fatigue level is determined based on the target feature set, and a prompt operation is performed when the target fatigue level is greater than a preset threshold. In this way, user features can be extracted and the fatigue level can be analyzed to perform corresponding prompt operations to ensure driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1AThis is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 1B This is a flowchart of a fatigue driving detection method provided in an embodiment of the present application;

[0024] Figure 2 This is a flow chart of another fatigue driving detection method provided in an embodiment of the present application;

[0025] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0026] Figure 4 This is a block diagram of the functional units of a fatigue driving detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0029] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] The electronic devices involved in the embodiments of the present application may be electronic devices with communication capabilities or electronic devices without communication capabilities. The electronic devices may include various handheld devices with wireless communication functions, vehicle-mounted devices (driving recorders, in-vehicle cameras, vehicle-mounted speakers, etc.), wearable devices (smart glasses, smart bracelets, smart watches, etc.), computing devices or other processing devices connected to wireless modems, as well as various forms of user equipment (UE), mobile stations (MS), terminal devices, etc.

[0031] See Figure 1A , Figure 1A This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a processor, memory, a signal processor, a transceiver, a display, a speaker, a microphone, a random access memory (RAM), a camera, a sensor, and a communication module. The memory, signal processor, display, speaker, microphone, RAM, camera, sensor, and communication module are connected to the processor, and the transceiver is connected to the signal processor.

[0032] The display screen may be a liquid crystal display (LCD), an organic or inorganic light-emitting diode (OLED), an active matrix organic light-emitting diode (AMOLED), or the like.

[0033] The camera may be a normal camera or an infrared camera, which is not limited here. The camera may be a front camera or a rear camera, which is not limited here.

[0034] The sensor includes at least one of the following: a light sensor, a gyroscope, an infrared proximity sensor, a fingerprint sensor, a pressure sensor, and the like. The light sensor, also known as an ambient light sensor, is used to detect the brightness of ambient light. The light sensor may include a photosensitive element and an analog-to-digital converter. The photosensitive element is used to convert the collected light signal into an electrical signal, and the analog-to-digital converter is used to convert the electrical signal into a digital signal. Optionally, the light sensor may further include a signal amplifier, which may amplify the electrical signal converted by the photosensitive element and output it to the analog-to-digital converter. The photosensitive element may include at least one of a photodiode, a phototransistor, a photoresistor, and a silicon photocell.

[0035] Among them, the processor is the control center of the electronic device, which uses various interfaces and lines to connect the various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory and calling data stored in the memory, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole.

[0036] The processor may integrate an application processor and a modem processor. The application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily processes wireless communications. It is understood that the modem processor may not be integrated into the processor. The processor may be at least one of the following: an ISP, a CPU, a GPU, an NPU, and the like, without limitation.

[0037] The memory is used to store software programs and / or modules. The processor executes the various functional applications and data processing of the electronic device by running the software programs and / or modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one software program required for a function, etc.; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0038] The communication module may be used to implement a communication function, and the communication module may be at least one of the following: an infrared module, a Bluetooth module, a mobile communication module, an NFC module, a Wi-Fi module, etc., which are not limited here.

[0039] The following is a detailed introduction to the embodiments of the present application.

[0040] See also Figure 1B , Figure 1B This embodiment of the present application provides a flow chart of a fatigue driving detection method, which is applied to Figure 1A The electronic device shown in the figure, as shown in the figure, the fatigue driving detection method includes the following operations.

[0041] 101. Acquire an image to be processed, where the image to be processed includes a target object.

[0042] The target object may be a driver. In a specific implementation, the electronic device may include a camera, which captures the target object to obtain an image to be processed. The image to be processed may be one or more images, or may be a dynamic thumbnail, or may be a video image captured over a period of time.

[0043] Optionally, the above step 101, obtaining the image to be processed, may include the following steps:

[0044] 11. Obtain target relative position information between the camera and the target position of the target object;

[0045] 12. Determine the target first shooting parameter corresponding to the target relative position information according to a preset mapping relationship between the relative position information and the first shooting parameter;

[0046] 13. Obtain target environment parameters;

[0047] 14. Determine the target second shooting parameter corresponding to the target environmental parameter according to the preset mapping relationship between the environmental parameter and the second shooting parameter;

[0048] 15. Determine final shooting parameters based on the first shooting parameter of the target and the second shooting parameter of the target;

[0049] 16. Photograph the target object according to the final photographing parameters to obtain the image to be processed.

[0050] In the embodiment of the present application, the target position can be set by the user or by the system default, and the target position can be at least one of the following: left eye, right eye, nose, mouth, mole, etc., which are not limited here. The first shooting parameter can be at least one of the following: focal length, focus, etc., which are not limited here. The environmental parameter can be at least one of the following: temperature, humidity, magnetic field interference intensity, ambient light brightness, weather, season, geographical location, background, etc., which are not limited here. The second shooting parameter can be at least one of the following: exposure time, shutter size, white balance parameter, sensitivity, shooting mode, etc., which are not limited here.

[0051] In a specific implementation, the electronic device can obtain target relative position information between the camera and the target position of the target object through a ranging sensor. The target relative position information may include relative distance and relative angle. The electronic device can pre-store a mapping relationship between the preset relative position information and the first shooting parameter. Then, the target first shooting parameter corresponding to the target relative position information can be determined according to the mapping relationship between the preset relative position information and the first shooting parameter. The electronic device can also pre-store a mapping relationship between the preset environmental parameters and the second shooting parameter. Then, the electronic device can obtain the target environmental parameters and determine the target second shooting parameters corresponding to the target environmental parameters according to the mapping relationship between the preset environmental parameters and the second shooting parameter. The final shooting parameters can be the union of the target first shooting parameters and the target second shooting parameters. Finally, the electronic device can shoot the target object according to the final shooting parameters to obtain an image to be processed. In this way, an image to be processed corresponding to the environment, distance, and angle can be obtained, which helps to improve the accuracy of subsequent fatigue driving detection.

[0052] 102. Perform target extraction on the image to be processed to obtain a target area image corresponding to the target object.

[0053] In a specific implementation, the electronic device can perform target extraction on the image to be processed, where the target may be the driver area, and the target may include the face of the target object or the behavior of the target object, to obtain a target area image.

[0054] Optionally, the above step 102, performing target extraction on the image to be processed to obtain a target area image corresponding to the target object, may include the following steps:

[0055] 21. Divide the image to be processed into multiple regions;

[0056] 22. Determine a signal-to-noise ratio for each of the multiple regions to obtain multiple signal-to-noise ratios;

[0057] 23. Determine an average signal-to-noise ratio and a target mean square error corresponding to the multiple signal-to-noise ratios;

[0058] 24. Determine a first target extraction algorithm corresponding to the average signal-to-noise ratio according to a preset mapping relationship between the signal-to-noise ratio and the target extraction algorithm;

[0059] 25. Determine the target control parameter corresponding to the target mean square error according to a preset mapping relationship between the mean square error and the control parameter of the first target extraction algorithm;

[0060] 26. Performing image enhancement processing on corresponding areas of the multiple signal-to-noise ratios that are lower than the average signal-to-noise ratio;

[0061] 27. Perform target extraction on the image to be processed after image enhancement processing according to the first target extraction algorithm and the target control parameters to obtain the target area image of the target object.

[0062] The electronic device may pre-store a mapping relationship between a preset signal-to-noise ratio and a target extraction algorithm. The target extraction algorithm may be at least one of the following: a neural network algorithm, a multi-scale transformation algorithm, an image segmentation algorithm based on information entropy, an edge-based image segmentation algorithm, and the like, without limitation. The electronic device may also store a mapping relationship between a preset mean square error and a control parameter of the first target extraction algorithm. The control parameter may be understood as a control parameter of the target extraction algorithm used to control the accuracy of its target extraction. Each target extraction algorithm may correspond to a mapping relationship between a preset mean square error and a control parameter.

[0063] In a specific implementation, the electronic device can divide the image to be processed into multiple areas, the size of each area can be the same or different, and the signal-to-noise ratio of each area in the multiple areas can be determined to obtain multiple signal-to-noise ratios. The signal-to-noise ratio reflects the degree of influence of noise to a certain extent. Then, the average signal-to-noise ratio and target mean square error corresponding to the multiple signal-to-noise ratios can be determined. According to the mapping relationship between the preset signal-to-noise ratio and the target extraction algorithm, the first target extraction algorithm corresponding to the average signal-to-noise ratio is determined. According to the mapping relationship between the preset mean square error and the control parameters of the first target extraction algorithm, the target control parameters corresponding to the target mean square error are determined. The average signal-to-noise ratio reflects the overall characteristics of the image, and the mean square error reflects the correlation and differentiation between the image neighborhoods. Then, the target extraction algorithm and corresponding control parameters related to the overall characteristics of the image and the neighborhood characteristics can be selected.

[0064] Furthermore, the electronic device can perform image enhancement processing on the corresponding areas with lower than the average signal-to-noise ratio among multiple signal-to-noise ratios. The image enhancement processing can be achieved through an image enhancement algorithm. The image enhancement algorithm can be at least one of the following: histogram equalization, grayscale stretching, wavelet transform, etc., which are not limited here. Furthermore, the electronic device can perform target extraction on the image to be processed after image enhancement processing based on the first target extraction algorithm and target control parameters to obtain the target area image of the target object. In this way, accurate extraction of the target area can be achieved.

[0065] 103. Perform feature extraction on the target area image to obtain a target feature set, where the target feature set includes a target expression, a target human eye feature set, and target skin parameters.

[0066] The electronic device may perform feature extraction on the target area image to obtain a target feature set. The target feature set may include at least one of the following: a feature point set, a feature outline, an expression, a facial feature set, skin parameters, etc., without limitation. The skin parameters may include at least one of the following: skin color, pore size, oiliness, dryness, skin laxity, etc., without limitation. The target eye feature set may include at least one of the following: number of blinks, degree of eye closure, pupil gloss, eye size, thickness of eye bags, duration of squinting or closing eyes, etc., without limitation.

[0067] 104. Determine target fatigue based on the target feature set.

[0068] The degree of fatigue determines the degree of danger of the user's driving. In a specific implementation, the electronic device can input a target feature set into a preset neural network model to obtain a target fatigue level. The preset neural network model can be at least one of the following: a convolutional neural network model, a recurrent neural network model, a cascaded neural network model, etc., without limitation herein.

[0069] In a specific implementation, the above step 104, determining the target fatigue level based on the target feature set, may include the following steps:

[0070] 41. Acquire target physiological characteristic parameters of the target object;

[0071] 42. Determine a target weight set corresponding to the target physiological characteristic parameter according to a preset mapping relationship between the physiological characteristic parameter and the weight set, wherein the weight set includes a first weight value, a second weight value, and a third weight value, wherein the first weight value is a weight value corresponding to facial expression, the second weight value is a weight value corresponding to human eye features, and the third weight value is a weight value corresponding to skin;

[0072] 43. Determine a first fatigue level corresponding to the target expression according to a preset mapping relationship between expressions and fatigue levels;

[0073] 44. Compare the target human eye feature set with the reference human eye feature set to obtain a target comparison value;

[0074] 45. Determine a second fatigue level corresponding to the target comparison value according to a preset mapping relationship between the comparison value and the fatigue level;

[0075] 46. ​​Determine a target evaluation value corresponding to the target skin parameter;

[0076] 47. Determine a third fatigue level corresponding to the target evaluation value according to a preset mapping relationship between the evaluation value and the fatigue level;

[0077] 48. Perform a weighted operation based on the first fatigue level, the second fatigue level, the third fatigue level, the first weight, the second weight, and the third weight to obtain the target fatigue level.

[0078] In the embodiment of the present application, the physiological characteristic parameters may include at least one of the following: heart rate, blood pressure, blood sugar, blood lipids, blood temperature, respiratory rate, brain wave parameters, etc., which are not limited here. The brain wave parameters may include at least one of the following: amplitude, frequency, average energy, waveform, etc., which are not limited here. In a specific implementation, the electronic device may pre-store a mapping relationship between preset physiological characteristic parameters and a weight set. The weight set may include a first weight corresponding to facial expression, a second weight corresponding to eye features, and a third weight corresponding to skin. The first weight is a weight corresponding to facial expression, the second weight is a weight corresponding to eye features, and the third weight is a weight corresponding to skin. The reference eye feature set may include at least one of the following: number of blinks, degree of eye closure, pupil gloss, eye size, thickness of eye bags, duration of squinting or closing eyes, etc., which are not limited here. The electronic device may also pre-store a mapping relationship between preset facial expression and fatigue, a mapping relationship between preset comparison values ​​and fatigue, and a mapping relationship between preset evaluation values ​​and fatigue.

[0079] In a specific implementation, the electronic device can obtain the target physiological characteristic parameters of the target object through a human body sensor. The human body sensor can be attached to the human skin or embedded in the human body. Furthermore, the electronic device can determine the target weight set corresponding to the target physiological characteristic parameters according to the preset mapping relationship between the physiological characteristic parameters and the weight set. The weight set includes a first weight value, a second weight value, and a third weight value. The first weight value is the weight corresponding to the facial expression, the second weight value is the weight corresponding to the human eye feature, and the third weight value is the weight corresponding to the skin. The sum of the first weight value, the second weight value, and the second weight value can be 1. Furthermore, the electronic device can determine the first fatigue level corresponding to the target expression according to the preset mapping relationship between the expression and the fatigue level, compare the target human eye feature set with the reference human eye feature set to obtain the target comparison value, and then determine the second fatigue level corresponding to the target comparison value according to the preset mapping relationship between the comparison value and the fatigue level, determine the target evaluation value corresponding to the target skin parameter, and determine the third fatigue level corresponding to the target evaluation value according to the mapping relationship between the preset evaluation value and the fatigue level. A weighted operation is performed based on the first fatigue level, the second fatigue level, the third fatigue level, the first weight, the second weight and the third weight to obtain the target fatigue level. In this way, the user's fatigue level can be accurately determined from three levels: expression, human eye state and skin.

[0080] 105. When the target fatigue level is greater than a preset threshold, perform a prompt operation.

[0081] The preset threshold value can be set by the user or by the system as a default. In a specific implementation, when the target fatigue level is greater than the preset threshold value, it indicates that the user is in a fatigue state, and a prompt operation can be performed to prompt the user to rest or to pay attention to safety.

[0082] Optionally, the above step 105, performing the prompt operation, may include the following steps:

[0083] 51. Determine the duration of the target fatigue level;

[0084] 52. Obtaining average driving parameters during the duration;

[0085] 53. Determine a target prompt parameter of the average driving parameter according to a preset mapping relationship between the prompt parameter and the driving parameter;

[0086] 54. Perform prompting operations according to the target prompting parameters.

[0087] The average driving parameter may be at least one of the following: road surface smoothness, road surface unevenness, driving time, driving speed, driving route, fuel consumption, number of brakes, music playback parameters, road conditions ahead (e.g., visibility), etc., which are not limited here. The music playback parameter may be at least one of the following: music type, volume, melody, rhythm, etc., which are not limited here. The electronic device may pre-store a mapping relationship between preset prompt parameters and driving parameters. The prompt parameter may be at least one of the following: prompt method, prompt frequency, recommended driving time, recommended driving speed, driving mode switching, etc. The driving mode switching may be switching from manual driving to automatic driving, which is not limited here. The prompt method may be at least one of the following: vibration, ringtone, light, display, etc., which are not limited here.

[0088] In a specific implementation, the electronic device can determine the duration of the target fatigue level and obtain the average driving parameters within the duration. According to the mapping relationship between the preset prompt parameters and the driving parameters, the target prompt parameters of the average driving parameters are determined, and prompt operations are performed based on the target prompt parameters. In this way, when the user is tired, accurate prompt operations can be achieved based on the driving speed and music playing conditions (music also plays a role in relieving fatigue to a certain extent).

[0089] Optionally, before step 105, the following steps may also be included:

[0090] A1. Obtain target driving parameters;

[0091] A2. Determine the preset threshold corresponding to the target driving parameter according to the mapping relationship between the preset driving parameter and the fatigue threshold.

[0092] In the embodiments of the present application, the driving parameter may be at least one of the following: driving duration, driving speed, driving route, etc., which are not limited herein. The electronic device may pre-store a mapping relationship between preset driving parameters and fatigue thresholds, and then, based on the mapping relationship between the preset driving parameters and fatigue thresholds, the preset threshold corresponding to the target driving parameter may be determined. In a specific implementation, different driving durations may correspond to different fatigue thresholds; or, different driving speeds may result in different user mental stress levels and thus corresponding to different fatigue thresholds; or, different driving routes may result in different fatigue thresholds, for example, different fatigue thresholds for urban roads and highways.

[0093] Optionally, before step 101, the method further includes:

[0094] B1. Obtain the target driving trajectory of the current road section;

[0095] B2. Obtaining a preset driving trajectory corresponding to the current road section;

[0096] B3. Matching the target driving trajectory with the preset driving trajectory to obtain a target matching value;

[0097] B4. When the matching value is less than a preset matching value, executing the step of acquiring the image to be processed.

[0098] Among them, the preset matching value can be pre-set or system default. The preset driving trajectory can be a driving trajectory simulated by the electronic device based on the conditions of the current road section, such as the terrain, the specific road, the speed, etc. In a specific implementation, if the user is tired, the trajectory may be in an "S" route, that is, in a fatigued state, the vehicle will suddenly deviate, and then, if the user wakes up, it will suddenly return to the right direction. Therefore, the electronic device can obtain the target driving trajectory of the current road section, and obtain the preset driving trajectory corresponding to the current road section, match the target driving trajectory with the preset driving trajectory, and obtain a target matching value. When the matching value is less than the preset matching value, it can be considered that the user is fatigued, and step 101 is executed to perform fatigue detection on the user.

[0099] It can be seen that the fatigue driving detection method described in the embodiment of the present application obtains an image to be processed, which includes a target object, performs target extraction on the image to be processed, obtains a target area image corresponding to the target object, performs feature extraction on the target area image, and obtains a target feature set. The target feature set includes a target expression, a target human eye feature set, and a target skin parameter. The target fatigue level is determined based on the target feature set, and a prompt operation is performed when the target fatigue level is greater than a preset threshold. In this way, user features can be extracted and the fatigue level can be analyzed to perform corresponding prompt operations to ensure driving safety.

[0100] With the above Figure 1B For details on the embodiment shown, please refer to Figure 2 , Figure 2 This is a flowchart of a fatigue driving detection method provided by an embodiment of the present application, which is applied to electronic devices. As shown in the figure, the fatigue driving detection method includes the following steps:

[0101] 201. Acquire an image to be processed, where the image to be processed includes a target object.

[0102] 202. Perform target extraction on the image to be processed to obtain a target area image corresponding to the target object.

[0103] 203. Perform feature extraction on the target area image to obtain a target feature set, where the target feature set includes a target expression, a target eye feature set, and target skin parameters.

[0104] 204. Determine target fatigue based on the target feature set.

[0105] 205. Obtain target driving parameters.

[0106] 206. Determine a preset threshold corresponding to the target driving parameter according to a mapping relationship between the preset driving parameter and the fatigue threshold.

[0107] 207. When the target fatigue level is greater than the preset threshold, perform a prompt operation.

[0108] The detailed description of the above steps 201 to 207 can be found in the above Figure 1B The corresponding steps of the described fatigue driving detection method are not repeated here.

[0109] It can be seen that the fatigue driving detection method described in the embodiment of the present application obtains an image to be processed, which includes a target object, performs target extraction on the image to be processed, obtains a target area image corresponding to the target object, performs feature extraction on the target area image, obtains a target feature set, and the target feature set includes a target expression, a target human eye feature set, and a target skin parameter. The target fatigue degree is determined based on the target feature set, and the target driving parameters are obtained. According to the mapping relationship between the preset driving parameters and the fatigue threshold, the preset threshold corresponding to the target driving parameters is determined. When the target fatigue degree is greater than the preset threshold, a prompt operation is performed. In this way, user characteristics can be extracted and the fatigue degree can be analyzed to perform corresponding prompt operations to ensure driving safety.

[0110] With the above Figure 1B 、 Figure 2 For details on the embodiment shown, please refer to Figure 3 , Figure 3 3 is a structural diagram of an electronic device 300 provided in an embodiment of the present application. As shown in the figure, the electronic device 300 includes a processor 310, a memory 320, a communication interface 330, and one or more programs 321. The one or more programs 321 are stored in the memory 320 and are configured to be executed by the processor 310. The one or more programs 321 include instructions for performing the following steps:

[0111] Acquire an image to be processed, wherein the image to be processed includes a target object;

[0112] Performing target extraction on the image to be processed to obtain a target area image corresponding to the target object;

[0113] Performing feature extraction on the target area image to obtain a target feature set, wherein the target feature set includes a target expression, a target human eye feature set, and a target skin parameter;

[0114] determining a target fatigue level based on the target feature set;

[0115] When the target fatigue level is greater than a preset threshold, a prompt operation is performed.

[0116] It can be seen that the electronic device described in the embodiment of the present application obtains an image to be processed, which includes a target object, performs target extraction on the image to be processed, obtains a target area image corresponding to the target object, performs feature extraction on the target area image, obtains a target feature set, and the target feature set includes a target expression, a target human eye feature set, and a target skin parameter. The target fatigue level is determined based on the target feature set, and a prompt operation is performed when the target fatigue level is greater than a preset threshold. In this way, user features can be extracted and the fatigue level can be analyzed to perform corresponding prompt operations to ensure driving safety.

[0117] In one possible example, in determining the target fatigue level based on the target feature set, the one or more programs 321 include instructions for performing the following steps:

[0118] Acquiring target physiological characteristic parameters of the target object;

[0119] Determining a target weight set corresponding to the target physiological characteristic parameter according to a preset mapping relationship between the physiological characteristic parameter and the weight set, the weight set including a first weight value, a second weight value, and a third weight value, wherein the first weight value is a weight corresponding to the facial expression, the second weight value is a weight corresponding to the human eye feature, and the third weight value is a weight corresponding to the skin;

[0120] Determining a first fatigue level corresponding to the target expression according to a preset mapping relationship between expressions and fatigue levels;

[0121] Comparing the target human eye feature set with the reference human eye feature set to obtain a target comparison value;

[0122] Determining a second fatigue level corresponding to the target comparison value according to a preset mapping relationship between the comparison value and the fatigue level;

[0123] Determining a target evaluation value corresponding to the target skin parameter;

[0124] Determining a third fatigue level corresponding to the target evaluation value according to a preset mapping relationship between the evaluation value and the fatigue level;

[0125] A weighted operation is performed according to the first fatigue degree, the second fatigue degree, the third fatigue degree, the first weight, the second weight, and the third weight to obtain the target fatigue degree.

[0126] In one possible example, in terms of acquiring the image to be processed, the one or more programs 321 include instructions for executing the following steps:

[0127] Obtaining target relative position information between the camera and the target position of the target object;

[0128] Determining the target first shooting parameter corresponding to the target relative position information according to a preset mapping relationship between the relative position information and the first shooting parameter;

[0129] Get target environment parameters;

[0130] Determining the target second shooting parameter corresponding to the target environmental parameter according to the mapping relationship between the preset environmental parameter and the second shooting parameter;

[0131] Determining final shooting parameters according to the first shooting parameter of the target and the second shooting parameter of the target;

[0132] The target object is photographed according to the final photographing parameters to obtain the image to be processed.

[0133] In one possible example, the one or more programs 321 further include instructions for performing the following steps:

[0134] Obtain target driving parameters;

[0135] According to the mapping relationship between the preset driving parameters and the fatigue threshold, the preset threshold corresponding to the target driving parameter is determined.

[0136] In one possible example, the one or more programs 321 further include instructions for performing the following steps:

[0137] Get the target driving trajectory of the current road section;

[0138] Obtaining a preset driving trajectory corresponding to the current road section;

[0139] Matching the target driving trajectory with the preset driving trajectory to obtain a target matching value;

[0140] When the matching value is less than a preset matching value, the step of acquiring the image to be processed is performed.

[0141] The above mainly introduces the solution of the embodiment of the present application from the perspective of the execution process of the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0142] The embodiment of the present application can divide the functional units of the electronic device according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0143] Figure 4 This is a block diagram of the functional units of the fatigue driving detection device 400 involved in the embodiment of the present application. The fatigue driving detection device 400 is applied to an electronic device, and the device 400 includes: an acquisition unit 401, an extraction unit 402, a determination unit 403 and a prompt unit 404, wherein,

[0144] The acquisition unit 401 is used to acquire an image to be processed, where the image to be processed includes a target object;

[0145] The extraction unit 402 is configured to extract a target from the image to be processed to obtain a target area image corresponding to the target object;

[0146] The extraction unit 402 is further configured to perform feature extraction on the target area image to obtain a target feature set, wherein the target feature set includes a target expression, a target eye feature set, and target skin parameters;

[0147] The determining unit 403 is configured to determine target fatigue based on the target feature set;

[0148] The prompt unit 404 is configured to perform a prompt operation when the target fatigue level is greater than a preset threshold.

[0149] It can be seen that the fatigue driving detection device described in the embodiment of the present application obtains an image to be processed, which includes a target object, performs target extraction on the image to be processed, obtains a target area image corresponding to the target object, performs feature extraction on the target area image, and obtains a target feature set. The target feature set includes a target expression, a target human eye feature set, and a target skin parameter. The target fatigue level is determined based on the target feature set, and a prompt operation is performed when the target fatigue level is greater than a preset threshold. In this way, user features can be extracted and the fatigue level can be analyzed to perform corresponding prompt operations to ensure driving safety.

[0150] In a possible example, in determining the target fatigue level according to the target feature set, the determining unit 403 is specifically configured to:

[0151] Acquiring target physiological characteristic parameters of the target object;

[0152] Determining a target weight set corresponding to the target physiological characteristic parameter according to a preset mapping relationship between the physiological characteristic parameter and the weight set, the weight set including a first weight value, a second weight value, and a third weight value, wherein the first weight value is a weight corresponding to the facial expression, the second weight value is a weight corresponding to the human eye feature, and the third weight value is a weight corresponding to the skin;

[0153] Determining a first fatigue level corresponding to the target expression according to a preset mapping relationship between expressions and fatigue levels;

[0154] Comparing the target human eye feature set with the reference human eye feature set to obtain a target comparison value;

[0155] Determining a second fatigue level corresponding to the target comparison value according to a preset mapping relationship between the comparison value and the fatigue level;

[0156] Determining a target evaluation value corresponding to the target skin parameter;

[0157] Determining a third fatigue level corresponding to the target evaluation value according to a preset mapping relationship between the evaluation value and the fatigue level;

[0158] A weighted operation is performed according to the first fatigue degree, the second fatigue degree, the third fatigue degree, the first weight, the second weight, and the third weight to obtain the target fatigue degree.

[0159] In a possible example, in terms of acquiring the image to be processed, the acquiring unit 401 is specifically configured to:

[0160] Obtaining target relative position information between the camera and the target position of the target object;

[0161] Determining the target first shooting parameter corresponding to the target relative position information according to a preset mapping relationship between the relative position information and the first shooting parameter;

[0162] Get target environment parameters;

[0163] Determining the target second shooting parameter corresponding to the target environmental parameter according to the mapping relationship between the preset environmental parameter and the second shooting parameter;

[0164] Determining final shooting parameters according to the first shooting parameter of the target and the second shooting parameter of the target;

[0165] The target object is photographed according to the final photographing parameters to obtain the image to be processed.

[0166] In a possible example, the determining unit 403 is further configured to perform the following operations:

[0167] Obtain target driving parameters;

[0168] According to the mapping relationship between the preset driving parameters and the fatigue threshold, the preset threshold corresponding to the target driving parameter is determined.

[0169] In a possible example, the apparatus 400 is further configured to perform the following operations:

[0170] Get the target driving trajectory of the current road section;

[0171] Obtaining a preset driving trajectory corresponding to the current road section;

[0172] Matching the target driving trajectory with the preset driving trajectory to obtain a target matching value;

[0173] When the matching value is less than a preset matching value, the step of acquiring the image to be processed is performed.

[0174] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0175] The present application also provides a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may comprise an electronic device.

[0176] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0177] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0179] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0181] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0182] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0183] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting fatigue driving, characterized in that: The method comprises: Acquire an image to be processed, wherein the image to be processed includes a target object; Performing target extraction on the image to be processed to obtain a target area image corresponding to the target object; Performing feature extraction on the target area image to obtain a target feature set, wherein the target feature set includes a target expression, a target human eye feature set, and a target skin parameter; determining a target fatigue level based on the target feature set; When the target fatigue level is greater than a preset threshold, performing a prompt operation; Wherein, determining the target fatigue level according to the target feature set includes: Acquiring target physiological characteristic parameters of the target object; Determining a target weight set corresponding to the target physiological characteristic parameter according to a preset mapping relationship between the physiological characteristic parameter and the weight set, the weight set including a first weight value, a second weight value, and a third weight value, wherein the first weight value is a weight corresponding to the facial expression, the second weight value is a weight corresponding to the human eye feature, and the third weight value is a weight corresponding to the skin; Determining a first fatigue level corresponding to the target expression according to a preset mapping relationship between expressions and fatigue levels; Comparing the target human eye feature set with the reference human eye feature set to obtain a target comparison value; Determining a second fatigue level corresponding to the target comparison value according to a preset mapping relationship between the comparison value and the fatigue level; Determining a target evaluation value corresponding to the target skin parameter; Determining a third fatigue level corresponding to the target evaluation value according to a preset mapping relationship between the evaluation value and the fatigue level; performing a weighted operation based on the first fatigue level, the second fatigue level, the third fatigue level, the first weight, the second weight, and the third weight to obtain the target fatigue level; The prompting operation includes: determining a duration of the target fatigue level; Obtaining average driving parameters within the duration; determining a target prompt parameter of the average driving parameter according to a preset mapping relationship between the prompt parameter and the driving parameter; Perform prompting operations according to the target prompting parameters.

2. The method according to claim 1, characterized in that The step of obtaining an image to be processed includes: Obtaining target relative position information between the camera and the target position of the target object; Determining the target first shooting parameter corresponding to the target relative position information according to a preset mapping relationship between the relative position information and the first shooting parameter; Get target environment parameters; Determining the target second shooting parameter corresponding to the target environmental parameter according to the mapping relationship between the preset environmental parameter and the second shooting parameter; Determining final shooting parameters according to the first shooting parameter of the target and the second shooting parameter of the target; The target object is photographed according to the final photographing parameters to obtain the image to be processed.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Obtain target driving parameters; According to the mapping relationship between the preset driving parameters and the fatigue threshold, the preset threshold corresponding to the target driving parameter is determined.

4. The method according to claim 1 or 2, characterized in that The method further comprises: Get the target driving trajectory of the current road section; Obtaining a preset driving trajectory corresponding to the current road section; Matching the target driving trajectory with the preset driving trajectory to obtain a target matching value; When the matching value is less than a preset matching value, the step of acquiring the image to be processed is performed.

5. A fatigue driving detection device, characterized in that: The device includes: an acquisition unit, an extraction unit, a determination unit and a prompt unit, wherein the acquisition unit is used to acquire an image to be processed, wherein the image to be processed includes a target object; The extraction unit is used to extract the target from the image to be processed to obtain a target area image corresponding to the target object; The extraction unit is further configured to perform feature extraction on the target area image to obtain a target feature set, wherein the target feature set includes a target expression, a target eye feature set, and target skin parameters; The determining unit is configured to determine target fatigue based on the target feature set; The prompt unit is used to perform a prompt operation when the target fatigue level is greater than a preset threshold; Wherein, in determining the target fatigue level according to the target feature set, the determining unit is specifically configured to: Acquiring target physiological characteristic parameters of the target object; Determining a target weight set corresponding to the target physiological characteristic parameter according to a preset mapping relationship between the physiological characteristic parameter and the weight set, the weight set including a first weight value, a second weight value, and a third weight value, wherein the first weight value is a weight corresponding to the facial expression, the second weight value is a weight corresponding to the human eye feature, and the third weight value is a weight corresponding to the skin; Determining a first fatigue level corresponding to the target expression according to a preset mapping relationship between expressions and fatigue levels; Comparing the target human eye feature set with the reference human eye feature set to obtain a target comparison value; Determining a second fatigue level corresponding to the target comparison value according to a preset mapping relationship between the comparison value and the fatigue level; and determining a target evaluation value corresponding to the target skin parameter; Determining a third fatigue level corresponding to the target evaluation value according to a preset mapping relationship between the evaluation value and the fatigue level; performing a weighted operation based on the first fatigue level, the second fatigue level, the third fatigue level, the first weight, the second weight, and the third weight to obtain the target fatigue level; The prompting operation includes: determining a duration of the target fatigue level; Obtaining average driving parameters within the duration; determining a target prompt parameter of the average driving parameter according to a preset mapping relationship between the prompt parameter and the driving parameter; Perform prompting operations according to the target prompting parameters.

6. The device according to claim 5, characterized in that In terms of acquiring the image to be processed, the acquiring unit is specifically configured to: Obtaining target relative position information between the camera and the target position of the target object; Determining the target first shooting parameter corresponding to the target relative position information according to a preset mapping relationship between the relative position information and the first shooting parameter; Get target environment parameters; Determining the target second shooting parameter corresponding to the target environmental parameter according to the mapping relationship between the preset environmental parameter and the second shooting parameter; Determining final shooting parameters according to the first shooting parameter of the target and the second shooting parameter of the target; The target object is photographed according to the final photographing parameters to obtain the image to be processed.

7. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 4.

Images