Fatigue detection method, device, electronic device and storage medium

Abstract

The present application relates to the field of computer technology, and in particular to a fatigue detection method, device, electronic device and storage medium for improving the accuracy of fatigue detection, wherein the method comprises: obtaining each frame of video image of a target object during driving, and determining the fatigue according to the facial information of the target object; determining the weighted fatigue corresponding to the target object according to the determined fatigue, the weighted fatigue of the target object in the current frame: the product of the cumulative fatigue value of the target object in the previous frame and the first preset weight value, and the product of the fatigue in the current frame and the second preset weight value; taking the weighted fatigue of the last frame in the target time period as the cumulative fatigue value in the target time period, and analyzing the driving state of the target object based on the cumulative fatigue value. Since the present application detects the cumulative fatigue value of the target object in the time period and performs analysis, the accuracy of fatigue detection can be improved.

Description

Background Art

[0002] Driving fatigue refers to the situation where the driver's reaction ability decreases due to lack of sleep or long-term continuous driving. This decrease is manifested in the driver's sleepiness, dozing off, driving errors, etc. Driving fatigue and traffic accidents are often closely related. Therefore, during the driver's driving process, the degree of driving fatigue can be detected to reduce the occurrence of traffic accidents.

[0003] In the related art, there are mainly two methods for detecting driving fatigue:

[0004] Method 1: A method for detecting the driver's fatigue level based on indirect signals such as the vehicle's driving status and trajectory. However, this method has low accuracy and is prone to misjudgment. Method 2: A method for detecting the driver's fatigue level based on physiological characteristics such as the driver's heartbeat and pulse. However, this method requires additional equipment and is easily affected by external factors.

[0005] Therefore, how to accurately detect the driver's fatigue state becomes an urgent problem to be solved. Summary of the invention

[0006] The embodiments of the present application provide a fatigue detection method, device, electronic device and storage medium to improve the accuracy of fatigue detection.

[0007] A fatigue detection method provided in an embodiment of the present application includes:

[0008] Acquire each frame of video image corresponding to the target object during the driving process, and determine the fatigue degree corresponding to each frame of the video image according to the facial information of the target object in each frame of the video image;

[0009] According to the determined fatigue degree, the weighted fatigue degree corresponding to the target object in each frame of the video is determined in sequence, wherein the weighted fatigue degree of the target object in the current frame is: the sum of the product of the accumulated fatigue value corresponding to the target object in the previous frame and the first preset weight value, and the product of the fatigue degree corresponding to the target object in the current frame and the second preset weight value;

[0010] The weighted fatigue degree corresponding to the last frame in the target period is used as the cumulative fatigue value of the target object in the target period, and the driving state of the target object is analyzed based on the cumulative fatigue value.

[0011] In an optional implementation, the sum of the first preset weight value and the second preset weight value is a first specified value, and the first preset weight value is determined based on a detection frame rate of video frames and the number of video frames.

[0012] In an optional implementation, if the current frame is the first frame corresponding to the target object during the driving process, the accumulated fatigue value corresponding to the target object in the previous frame is the second specified value.

[0013] In an optional implementation, the target period is the remaining driving period of the target object except the initial driving period during the driving process, and the initial driving period is a certain period of time from the start of driving during the driving process of the target object;

[0014] The analyzing the driving state of the target object based on the fatigue accumulated value includes:

[0015] Obtaining a first difference between a fatigue reference parameter corresponding to the target object in the initial driving period and a fatigue cumulative value corresponding to the target object in the remaining driving period, wherein the fatigue reference parameter pair is a fatigue cumulative value of the target object in a sober state;

[0016] If the first difference is greater than the target threshold, it is determined that the target object is in a fatigue state.

[0017] In an optional implementation, the target threshold is obtained by:

[0018] Determine a second difference between the accumulated fatigue values ​​of each sample subject in the awake state and the accumulated fatigue value of each sample subject in the fatigue state according to the accumulated fatigue value of each sample subject in the awake state and the accumulated fatigue value of each sample subject in the fatigue state in the sample database;

[0019] The product of the mean of the second difference values ​​and the preset ratio value is used as the target threshold.

[0020] In an optional embodiment, the method further includes:

[0021] Determining the number of operations performed by the target subject on the vehicle according to the driving state of the vehicle driven by the target subject;

[0022] According to the changing trend of the number of operations within a specified time period, it is determined whether the target object is in a fatigue state.

[0023] In an optional embodiment, the method further includes:

[0024] If it is determined that the target object is in a false fatigue state, feeding back fatigue driving prompt information to the target object;

[0025] If it is determined that the target object is in a true fatigue state, the vehicle speed is controlled to decrease until it is determined that the target object is in a sober state according to the fatigue accumulation value and the driving state of the vehicle, and the original vehicle speed is restored.

[0026] In an optional implementation manner, the fatigue reference parameter is determined in the following manner:

[0027] According to a preset time interval, the starting driving period is divided into a plurality of sub-periods; for each sub-period, the weighted fatigue degree of the target object corresponding to each frame of the video in each sub-period is determined in sequence; the weighted fatigue degree corresponding to the last frame in each sub-period is used as the cumulative fatigue value of the target object in each sub-period; the average of the cumulative fatigue values ​​corresponding to the target object in each sub-period is used as the fatigue reference parameter; or

[0028] Determine in sequence the weighted fatigue degree of the target object corresponding to each video frame in the initial driving period; use the weighted fatigue degree corresponding to the last frame in the initial driving period as the fatigue degree reference parameter; or

[0029] The average value of fatigue degree corresponding to each video frame of the target object in the initial driving period is used as the fatigue degree reference parameter.

[0030] In an optional implementation, the analyzing the driving state of the target object based on the fatigue accumulation value further includes:

[0031] The ratio between the fatigue accumulated value and a preset correction formula is used as the corrected fatigue accumulated value, wherein the preset correction formula is determined based on the difference between the first specified value and the power of the first preset weight value, and the exponent of the power is the number of video frames in the target time period;

[0032] The driving state of the target object is analyzed based on the corrected fatigue cumulative value.

[0033] An embodiment of the present application provides a fatigue detection device, comprising:

[0034] An acquisition unit, used for acquiring each frame of video image corresponding to the target object during driving, and determining the fatigue degree corresponding to each frame of video image according to the facial information of the target object in each frame of video image;

[0035] A determination unit, configured to sequentially determine the weighted fatigue degree of the target object corresponding to each frame of the video according to the determined fatigue degree, wherein the weighted fatigue degree of the target object in the current frame is: the sum of the product of the accumulated fatigue degree value corresponding to the target object in the previous frame and the first preset weight value, and the product of the fatigue degree corresponding to the target object in the current frame and the second preset weight value;

[0036] The analysis unit is used to use the weighted fatigue corresponding to the last frame in the target time period as the cumulative fatigue value of the target object in the target time period, and analyze the driving state of the target object based on the cumulative fatigue value.

[0037] Optionally, the sum of the first preset weight value and the second preset weight value is a first specified value, and the first preset weight value is determined based on the detection frame rate of the video frames and the number of video frames.

[0038] Optionally, if the current frame is the first frame corresponding to the target object during the driving process, the accumulated fatigue value corresponding to the target object in the previous frame is the second specified value.

[0039] Optionally, the target period is the remaining driving period except the initial driving period during the driving process of the target object, and the initial driving period is a certain period of time from the start of driving during the driving process of the target object; the analysis unit is specifically used to:

[0040] Obtaining a first difference between a fatigue reference parameter corresponding to the target object in the initial driving period and a fatigue cumulative value corresponding to the target object in the remaining driving period, wherein the fatigue reference parameter pair is a fatigue cumulative value of the target object in a sober state;

[0041] If the first difference is greater than the target threshold, it is determined that the target object is in a fatigue state.

[0042] Optionally, the target threshold is obtained by:

[0043] Determine a second difference between the accumulated fatigue values ​​of each sample subject in the awake state and the accumulated fatigue value of each sample subject in the fatigue state according to the accumulated fatigue value of each sample subject in the awake state and the accumulated fatigue value of each sample subject in the fatigue state in the sample database;

[0044] The product of the mean of the second difference values ​​and the preset ratio value is used as the target threshold.

[0045] Optionally, the analysis unit is further used for:

[0046] Determining the number of operations performed by the target subject on the vehicle according to the driving state of the vehicle driven by the target subject;

[0047] According to the changing trend of the number of operations within a specified time period, it is determined whether the target object is in a fatigue state.

[0048] Optionally, the analysis unit is further used for:

[0049] If it is determined that the target object is in a false fatigue state, feeding back fatigue driving prompt information to the target object;

[0050] If it is determined that the target object is in a true fatigue state, the vehicle speed is controlled to decrease until it is determined that the target object is in a sober state according to the fatigue accumulation value and the driving state of the vehicle, and the original vehicle speed is restored.

[0051] Optionally, the fatigue reference parameter is determined by the following method:

[0052] According to a preset time interval, the starting driving period is divided into a plurality of sub-periods; for each sub-period, the weighted fatigue degree of the target object corresponding to each frame of the video in each sub-period is determined in sequence; the weighted fatigue degree corresponding to the last frame in each sub-period is used as the cumulative fatigue value of the target object in each sub-period; the average of the cumulative fatigue values ​​corresponding to the target object in each sub-period is used as the fatigue reference parameter; or

[0053] Determine in sequence the weighted fatigue degree of the target object corresponding to each frame of the video during the initial driving period; use the weighted fatigue degree corresponding to the last frame during the initial driving period as the fatigue degree reference parameter; or

[0054] The average value of fatigue degree corresponding to each video frame of the target object in the initial driving period is used as the fatigue degree reference parameter.

[0055] Optionally, the analysis unit is further used for:

[0056] The ratio between the fatigue accumulated value and a preset correction formula is used as the corrected fatigue accumulated value, wherein the preset correction formula is determined based on the difference between the first specified value and the power of the first preset weight value, and the exponent of the power is the number of video frames in the target time period;

[0057] The driving state of the target object is analyzed based on the corrected fatigue cumulative value.

[0058] An electronic device provided in an embodiment of the present application includes a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of any one of the above fatigue detection methods.

[0059] An embodiment of the present application provides a computer-readable storage medium, which includes a program code. When the storage medium is run on an electronic device, the program code is used to enable the electronic device to execute the steps of any one of the above fatigue detection methods.

[0060] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above fatigue detection methods.

[0061] The beneficial effects of this application are as follows:

[0062] The embodiments of the present application provide a fatigue detection method, device, electronic device and storage medium. The embodiments of the present application determine the corresponding fatigue by acquiring each frame of video image corresponding to the target object during driving, and determine the weighted fatigue corresponding to each frame of the video frame according to the determined fatigue. The weighted fatigue corresponding to the last frame in the target period is used as the cumulative fatigue value of the target object in the target period, and the driving state of the target object is analyzed based on the cumulative fatigue value. Fatigue detection based on the above method focuses on the impact of the current state on fatigue, improves the accuracy of fatigue detection, and effectively improves the safety of the driver's driving.

[0063] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0065] Figure 1 A schematic diagram of an application scenario of the fatigue detection method in an embodiment of the present application;

[0066] Figure 2This is a flowchart of an implementation method of a fatigue detection method in an embodiment of the present application;

[0067] Figure 3 A schematic diagram of a fatigue calculation method in an embodiment of the present application;

[0068] Figure 4 This is a flowchart of a method for obtaining a target threshold value in an embodiment of the present application;

[0069] Figure 5 This is an overall implementation flow chart of a fatigue detection method in an embodiment of the present application;

[0070] Figure 6 This is a schematic diagram of the structure of a fatigue detection device in an embodiment of the present application;

[0071] Figure 7 A schematic diagram of the structure of an electronic device in an embodiment of the present application;

[0072] Figure 8 A schematic diagram of the structure of a computing device to which an embodiment of the present application is applied. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0074] The following is an introduction to some concepts involved in the embodiments of the present application.

[0075] 1. In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0076] 2. The term "fatigue driving" in the embodiments of the present application refers to the phenomenon that the driver's physiological and psychological functions are disturbed after long-term continuous driving, and his driving skills objectively decline. Fatigue is used to measure the degree of fatigue of the driver during driving. Since the degree of driver fatigue driving gradually increases, the present application uses weighted fatigue combined with the fatigue of the current frame and the weighted fatigue of the previous frame to comprehensively judge the fatigue degree of the driver in the current frame.

[0077] 3. In the embodiments of the present application, the term "accumulated fatigue value" refers to the accumulated value of the driver's fatigue over a period of time during the driver's driving process. Since it is usually necessary to combine the driver's performance over a period of time to determine whether the driver is in a fatigued state, statistics on the accumulated fatigue of the driver over a period of time can more accurately reflect the driver's fatigue state.

[0078] 4. In the embodiments of the present application, the term "fatigue reference parameter" refers to the cumulative fatigue value of the driver in the awake state. The cumulative fatigue value in the awake state is used as a reference and compared with the driver's cumulative fatigue value in real time to determine whether the driver is in a fatigued state. Using the driver's own cumulative fatigue value in the awake state as a reference has a higher accuracy rate.

[0079] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0080] like Figure 1 As shown, it is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes two terminal devices 110 and a server 120. The terminal device 110 in the embodiment of the present application may be installed with a client.

[0081] In the embodiment of the present application, the terminal device 110 includes but is not limited to a user terminal such as a personal computer, a mobile phone, a tablet computer, a notebook, an e-book reader, and a vehicle-mounted terminal, which has a certain computing power and runs instant messaging software and websites or social software and websites. Each terminal device 110 is connected to the server 120 via a wireless network. The server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0082] In the embodiment of the present application, a fatigue detection client may be installed on the terminal device 110, and the fatigue detection client is used to detect the fatigue of the user in scenes such as driving. The fatigue detection client may be deployed on the terminal device 110 or on the server 120. For example, after the facial image information of the driver is obtained through the on-board camera of a smart car, it may be calculated on the terminal device 110 or transmitted back to the server 120, and the server 120 calculates the result and then transmits it back to the terminal device 110.

[0083] It should be noted that the fatigue detection method provided in the embodiments of the present application can be applied to various application scenarios involving fatigue detection tasks. This article mainly introduces the fatigue detection method applied in the process of automobile driving as an example.

[0084] It should be noted that the fatigue detection method in the embodiment of the present application can be executed by the server or the terminal device alone, or can be executed by the server and the terminal device together.

[0085] It should be noted that Figure 1 The figure is only an example. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of the present application.

[0086] The following describes the model parameter updating method provided by the exemplary embodiment of the present application in combination with the application scenario described above and with reference to the accompanying drawings. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0087] See also Figure 2 FIG. 1 is a flowchart of an implementation method of a fatigue detection method provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0088] S21: The terminal obtains each frame of video image corresponding to the target object during the driving process, and determines the fatigue degree corresponding to each frame of the video image according to the facial information of the target object in each frame of the video image;

[0089] Specifically, at a certain moment, that is, a certain frame of the video, usually 1s of video has 24 frames. For example, 3 frames of video images can be obtained per second, namely video image 1, video image 2, and video image 3. According to the facial information of each frame of the video image, the fatigue degree corresponding to each frame of the video image is determined, which are fatigue degree 1, fatigue degree 2, and fatigue degree 3.

[0090] S22: The terminal determines the weighted fatigue degree of the target object corresponding to each video frame in sequence according to the determined fatigue degree;

[0091] The weighted fatigue degree of the target object in the current frame is: the product of the weighted fatigue degree corresponding to the target object in the previous frame and the first preset weight value, and the product of the fatigue degree corresponding to the target object in the current frame and the second preset weight value;

[0092] Specifically, it is more accurate to determine whether the target object is in a fatigue state by counting the fatigue degree of the target object over a period of time. Therefore, the weighted fatigue degree of the current frame is obtained by weighted summing the fatigue degree of the current frame and the weighted fatigue degree of the previous frame to judge the current driving state of the target object.

[0093] S23: The terminal uses the weighted fatigue corresponding to the last frame in the target period as the cumulative fatigue value of the target object in the target period, and analyzes the driving state of the target object based on the cumulative fatigue value.

[0094] Specifically, the target time period may be within 1 minute from the time the target object starts driving. The weighted fatigue degree corresponding to each video frame of the target object within 1 minute is determined respectively, and the weighted fatigue degree corresponding to the last frame within 1 minute is taken as the accumulated fatigue value within 1 minute.

[0095] In the above implementation, the corresponding fatigue degree is determined by acquiring each frame of video image corresponding to the target object during driving, and the weighted fatigue degree corresponding to each frame of the video frame of the target object is determined according to the determined fatigue degree. The weighted fatigue degree corresponding to the last frame in the target period is used as the cumulative fatigue value of the target object in the target period, and the driving state of the target object is analyzed based on the cumulative fatigue value. Fatigue detection based on the above method focuses on the influence of the current state on fatigue, improves the accuracy of fatigue detection, and effectively improves the driving safety of the driver.

[0096] The following is a detailed description of step S21. Figure 3 , which is a flow chart of a fatigue calculation method in an embodiment of the present application, comprising the following steps:

[0097] S31: Acquire each frame of video image corresponding to the driver during the driving process;

[0098] Specifically, the driver's facial image information collected by the on-board camera of the smart car is obtained.

[0099] S32: Detecting facial information of the driver in each frame of video image;

[0100] Specifically, a deep learning model is used to detect each frame of the driver's facial image, and to detect the closing state of the driver's eyes and mouth, as well as the angle of the driver's head down, to calculate the fatigue level corresponding to each frame of the video. Including:

[0101] Using variable at represents the driver's eyelid state at time t (a certain moment is a certain frame of the video, usually 1s of video has 24 frames), and if the driver is in the closed eye state, then a t =1, if the driver is in the state of not closing his eyes, then a t =0;

[0102] Using variable b t Indicates whether the driver is yawning at time t. The video images of the driver in the 2.5s before time t are counted. If the number of video frames with the mouth open accounts for more than 99%, it can be determined that the driver is yawning. Any time within 2.5s before this moment is also determined to be in a yawning state. If the driver is in a yawning state, then b t =1, if the driver is not yawning, then b t =0;

[0103] Using variable c t Indicates whether the driver is in a head-down state at time t. Since the driver will doze off involuntarily when he is tired, the number of times the driver nods increases. By training the deep learning model to obtain the positions of the key points of the face, and then calculating the pitch (angle) value based on the detected key points, the driver's head posture can be judged. If the driver is in a head-down state, then c t =1, if the driver is not in the head-down state, then c t =0.

[0104] Among them, by obtaining a large number of facial images of drivers and annotating them to obtain training sample data for the deep learning model, data can be obtained by simulating yawning and dozing off, and the driver's mouth and eyes as well as the closed state can be annotated. The pitch value of the driver's head down is calculated through 21 facial key points. When the pitch value is greater than the preset threshold, it is determined that the driver is in a head-down state.

[0105] S33: Calculate the fatigue degree corresponding to each video frame.

[0106] Specifically, the fatigue degree corresponding to the current frame is calculated by the following formula:

[0107] p t =α·a t +β·b t +γ·c t

[0108] Among them, p t represents the fatigue degree corresponding to time t, a t Indicates the eyelid state at time t, variable b t Indicates whether the person is yawning at time t, c tIndicates whether the head is in the lowered state at time t. The weights can be set to α=0.4, β=0.3, and γ=0.3.

[0109] Most smart cars are equipped with artificial intelligence computing chips, so the detection and calculation of the driver's facial image can be performed by the chip inside the car, and the detection results can be given in real time. The image information can also be transmitted back to the cloud using a wireless network or mobile data network, and the results will be calculated by the back-end server and then transmitted back to the car's control system for the next step of calculation.

[0110] In the above embodiment, a non-contact method is used to identify the driver's facial information, and the fatigue state is determined by multiple fatigue state judgment indicators, which improves the recognition accuracy. A deep learning model is used to detect and identify facial features, which has high accuracy and strong robustness and can adapt to the behaviors of different drivers.

[0111] The following is a detailed introduction to step S22. According to the determined fatigue degree, the weighted fatigue degree of the target object corresponding to each video frame is calculated using the following formula:

[0112] θ 0 =0

[0113] θ 1 =λ 1 θ 0 +λ 2 p 1

[0114] θ 2 =λ 1 θ 1 +λ 2 p 2

[0115] …

[0116] θ t =λ 1 θ t-1 +λ 2 p t

[0117] Among them, θ t represents the weighted fatigue degree corresponding to time t, λ 1 represents the first preset weight value, λ 2 represents the second preset weight value, θ t-1 Represents the weighted fatigue corresponding to the previous frame, p tIndicates the fatigue degree corresponding to the current frame. Since the driver's fatigue degree gradually increases during driving, and the fatigue degree closer to the current moment can better reflect the driver's current state, when determining the current fatigue state, the fatigue degree corresponding to the current frame can be calculated, and the weighted fatigue degree can be obtained by combining the weighted sum of the fatigue degree of the previous frame. The weighted fatigue degree can be adjusted by 1 and λ 2 The size of is used to adjust the influence of the current frame fatigue and the weighted fatigue of the previous frame on the current fatigue state.

[0118] In the above implementation, the influence of the fatigue degree at the current moment on the fatigue degree cumulative value is mainly considered to improve the fatigue state detection accuracy.

[0119] In an optional implementation, the sum of the first preset weight value and the second preset weight value is the first specified value, and the first preset weight value is determined based on the detection frame rate of the video frame and the number of required video frames.

[0120] Specifically, the first specified value can be set to 1, λ 1 +λ 2 =1, and λ 1 It is determined based on the video detection frame rate and the length of time required to judge the driver's fatigue driving. 1 It can be taken as 0.999.

[0121] In an optional implementation, the weighted fatigue degree corresponding to the last frame in the target period is used as the cumulative fatigue value of the target object in the target period, and the cumulative fatigue value is corrected in the following manner:

[0122] The ratio between the fatigue cumulative value and a preset correction formula is used as the corrected fatigue cumulative value, wherein the preset correction formula is determined based on the difference between the first specified value and the power of the first preset weight value, and the exponent of the power is the number of video frames in the target time period; the driving state of the target object is analyzed based on the corrected fatigue cumulative value.

[0123] Specifically, the ratio between the fatigue cumulative value and the preset correction formula is used as the corrected fatigue cumulative value. The following formula is used as the preset correction formula:

[0124]

[0125] Among them, θ t It represents the accumulated fatigue value corresponding to the period 0-t, that is, the weighted fatigue corresponding to time t.

[0126] In an optional implementation, if the current frame is the first frame corresponding to the target object during the driving process, the accumulated fatigue value corresponding to the target object in the previous frame is the second specified value.

[0127] Specifically, if the current frame is the first frame in the target object's driving process, when calculating the weighted fatigue degree corresponding to the first frame, since the first frame has no corresponding previous frame, the accumulated fatigue degree corresponding to the previous frame can be set to 0, that is, the second specified value is 0, θ 0 =0.

[0128] In an optional implementation, if the target period is the remaining driving period except the initial driving period during the driving process of the target object, and the initial driving period is a certain period of time from the start of driving during the driving process of the target object, the driving state of the target object is analyzed based on the fatigue accumulation value in the following manner:

[0129] Obtain a first difference between a fatigue reference parameter corresponding to the target object in the initial driving period and a fatigue cumulative value corresponding to the target object in the remaining driving period, wherein the fatigue reference parameter is the fatigue cumulative value of the target object in the awake state; if the first difference is greater than a target threshold, it is determined that the target object is in a fatigue state.

[0130] Specifically, the cumulative fatigue value of the driver during the initial driving period is obtained as a fatigue reference parameter for analyzing the driving state of the target object, and the cumulative fatigue value of the driver is detected during the remaining driving period. When the difference between the cumulative fatigue value during the remaining driving period and the fatigue reference parameter is greater than the target threshold, it is determined that the target object is in a fatigue state.

[0131] In the above embodiment, the cumulative value of fatigue of the driver in the awake state is used as a fatigue reference parameter and compared with the cumulative value of fatigue of the driver during driving. Since the facial reactions of different drivers in the awake and fatigued states are different, false alarms caused by using the same fatigue reference parameter are avoided.

[0132] In an optional implementation, the fatigue reference parameter is obtained in the following manner:

[0133] Method 1: Divide the starting driving period into multiple sub-periods according to a preset time interval; for each sub-period, determine the weighted fatigue degree of the target object corresponding to each video frame in each sub-period in turn; use the weighted fatigue degree corresponding to the last frame in each sub-period as the cumulative fatigue value of the target object in each sub-period; use the average of the cumulative fatigue values ​​corresponding to the target object in each sub-period as the fatigue reference parameter;

[0134] Method 2: sequentially determine the weighted fatigue degree of the target object corresponding to each frame of the video during the initial driving period; and use the weighted fatigue degree corresponding to the last frame during the initial driving period as a fatigue degree reference parameter;

[0135] Method three: taking the average fatigue degree corresponding to each video frame of the target object in the initial driving period as the fatigue degree reference parameter.

[0136] Specifically, according to the first method, assuming that the initial driving period is within 5 minutes from the start of driving, the 5 minutes are divided into 5 sub-periods, each sub-period is 1 minute, the fatigue cumulative value corresponding to sub-period 1 is 0, the fatigue cumulative value corresponding to sub-period 2 is 0.3, the fatigue cumulative value corresponding to sub-period 3 is 0.3, the fatigue cumulative value corresponding to sub-period 4 is 0.3, and the fatigue cumulative value corresponding to sub-period 5 is 0.4. The average of the 5 sub-periods is 2.6, and the fatigue reference parameter is 2.6.

[0137] According to the second method, the weighted fatigue degree of the last frame of the target object in the initial driving period is used as a fatigue degree reference parameter;

[0138] According to method three: the average of the weighted fatigue levels of the target driving period within the initial driving period is used as the fatigue reference parameter. For example, if the initial driving period is 5 minutes, and 5 weighted fatigue levels of 0, 0, 0.3, 0.3, and 0.3 are obtained respectively, then the fatigue reference parameter is 0.18.

[0139] In an optional implementation, the target threshold is obtained by:

[0140] First, the second difference between the cumulative fatigue values ​​of each sample object in the awake state and the cumulative fatigue values ​​of each sample object in the fatigue state in the sample database is determined; then, the product between the mean of the second difference and the preset ratio value is used as the target threshold.

[0141] Specifically, by marking the awake state and the fatigue state of the sample objects, the difference in the cumulative fatigue values ​​of the sample objects in the awake state and the fatigue state is calculated, and the product of the mean of the difference values ​​of each sample object and the preset ratio value is used as the target threshold. The preset ratio value can be set to 0.8.

[0142] See also Figure 4 , which is an implementation flow chart of a method for obtaining a target threshold in an embodiment of the present application, comprising the following steps:

[0143] S41: Obtaining the accumulated fatigue value of each sample subject in the awake state from the sample database;

[0144] S42: Obtaining the fatigue accumulation value of each sample object in a fatigue state from the sample database;

[0145] S43: determining the mean of the difference between the accumulated fatigue values ​​of each sample subject in the awake state and the fatigue state;

[0146] S44: taking the product of the difference value and the preset ratio value as the target threshold value.

[0147] In an optional implementation, after determining that the target object is in a fatigue state based on the fatigue accumulation value, the following method can be used to further confirm whether the target object is in a fatigue state:

[0148] First, according to the driving state of the vehicle driven by the target object, the number of operations of the vehicle by the target object is determined; according to the changing trend of the number of operations within a specified time period, it is determined whether the target object is in a fatigue state.

[0149] Specifically, the sensors in the vehicle are used to obtain the operating status of the driver and the driving status of the vehicle. The operating status includes: the number of steering wheel adjustments, the number of accelerator pedal operations, and the number of brake pedal operations. The driving status of the vehicle includes: driving direction, vehicle speed information, and acceleration information. The number of times the driver operates the vehicle can be determined based on the driving status of the vehicle. For example, the number of times the driver adjusts the steering wheel can be determined based on the driving direction of the vehicle. If the number of times the steering wheel is adjusted is less than 10 times within 1 minute, it is determined that the driver is in a real fatigue state.

[0150] In an optional implementation, if it is determined that the target object is in a false fatigue state, fatigue driving prompt information is fed back to the target object;

[0151] If it is determined that the target object is in a true fatigue state, the vehicle speed is controlled to decrease until it is determined that the target object is in a sober state based on the accumulated fatigue value and the driving state of the vehicle, and the original speed is restored.

[0152] Specifically, if it is determined that the driver is in a false fatigue state, feedback prompt information can be provided; if it is determined that the driver is in a true fatigue state, the vehicle speed is controlled to decrease until it is determined that the driver is awake based on the accumulated fatigue value and the driving state of the vehicle.

[0153] In the above embodiment, a non-contact method is used to collect video images of the driver, and the fatigue value of the driver in the awake state is calculated as a fatigue reference parameter to avoid false alarms due to individual differences in the driver's facial reactions in the awake and fatigued states. The impact of the current state on the fatigue state is focused on, thereby improving the accuracy of fatigue detection and effectively improving the driver's driving safety.

[0154] See also Figure 5 , which is an overall implementation flow chart of a fatigue detection method in an embodiment of the present application, comprising the following steps:

[0155] S51: Collect the driver's facial image and calculate the cumulative fatigue value, and use the average of the cumulative fatigue values ​​of the driver within five minutes of starting the vehicle as a fatigue reference parameter;

[0156] S52: calculating the accumulated fatigue value of the driver five minutes after starting the vehicle at regular intervals;

[0157] S53: Determine whether the difference between the fatigue cumulative value and the fatigue reference parameter is greater than a first threshold value, if so, execute step S54, if not, return to step S52;

[0158] Among them, the first threshold is the target threshold in this application.

[0159] S54: Determine whether the number of times the driver operates the vehicle within a certain period of time is less than a second threshold value, if so, execute step S55, if not, execute step S56;

[0160] The second threshold is used to determine the changing trend of the number of operations performed by the driver on the vehicle within a specified period of time.

[0161] S55: Control the vehicle to reduce speed;

[0162] S56: Send a reminder message to remind the driver of fatigue driving.

[0163] Based on the same inventive concept, the present application also provides a structural schematic diagram of a fatigue detection device. Figure 6 As shown, it is a schematic diagram of the structure of the fatigue detection device 600, which may include:

[0164] The acquisition unit 601 is used to acquire each frame of video image corresponding to the target object during the driving process, and determine the fatigue degree corresponding to each frame of the video image according to the facial information of the target object in each frame of the video image;

[0165] The determination unit 602 is used to determine the weighted fatigue degree of the target object in each frame of the video according to the determined fatigue degree, wherein the weighted fatigue degree of the target object in the current frame is: the sum of the product of the accumulated fatigue degree value corresponding to the target object in the previous frame and the first preset weight value, and the product of the fatigue degree corresponding to the target object in the current frame and the second preset weight value;

[0166] The analysis unit 603 is used to use the weighted fatigue corresponding to the last frame in the target period as the cumulative fatigue value of the target object in the target period, and analyze the driving state of the target object based on the cumulative fatigue value.

[0167] Optionally, the sum of the first preset weight value and the second preset weight value is a first specified value, and the first preset weight value is determined based on a detection frame rate of the video frames and the number of video frames.

[0168] Optionally, if the current frame is the first frame corresponding to the target object during the driving process, the accumulated fatigue value corresponding to the target object in the previous frame is the second specified value.

[0169] Optionally, the target period is the remaining driving period except the initial driving period during the driving process of the target object, and the initial driving period is a certain period of time from the start of driving during the driving process of the target object; the analysis unit 603 is specifically used to:

[0170] Obtain a first difference between a fatigue reference parameter corresponding to the target object in the initial driving period and a fatigue cumulative value corresponding to the target object in the remaining driving period, where the fatigue reference parameter pair is the fatigue cumulative value of the target object in the awake state; if the first difference is greater than a target threshold, it is determined that the target object is in a fatigue state.

[0171] Optionally, obtain the target threshold value by:

[0172] According to the accumulated fatigue values ​​of each sample object in the awake state and the accumulated fatigue values ​​of each sample object in the fatigue state in the sample database, a second difference between the accumulated fatigue values ​​of each sample object in the awake state and the accumulated fatigue values ​​of each sample object in the fatigue state is determined; and the product between the mean of the second difference and the preset ratio value is used as the target threshold.

[0173] Optionally, the analysis unit 603 is further used for:

[0174] According to the driving state of the vehicle driven by the target object, the number of operations of the vehicle by the target object is determined; according to the changing trend of the number of operations within a specified time period, it is determined whether the target object is in a fatigue state.

[0175] Optionally, the analysis unit 603 is further used for:

[0176] If it is determined that the target object is in a false fatigue state, a fatigue driving prompt message is fed back to the target object;

[0177] If it is determined that the target object is in a true fatigue state, the vehicle speed is controlled to decrease until it is determined that the target object is in a sober state based on the accumulated fatigue value and the driving state of the vehicle, and the original speed is restored.

[0178] Optionally, fatigue reference parameters are determined by:

[0179] According to a preset time interval, the initial driving period is divided into multiple sub-periods; for each sub-period, the weighted fatigue degree of the target object corresponding to each frame of the video in each sub-period is determined in turn; the weighted fatigue degree corresponding to the last frame in each sub-period is used as the cumulative fatigue value of the target object in each sub-period; the average of the cumulative fatigue values ​​corresponding to the target object in each sub-period is used as a fatigue reference parameter; or the weighted fatigue degree of the target object corresponding to each frame of the video in the initial driving period is determined in turn; the weighted fatigue degree corresponding to the last frame in the initial driving period is used as the fatigue reference parameter; or the average of the fatigue degree corresponding to each frame of the video in the initial driving period is used as the fatigue reference parameter.

[0180] Optionally, the analysis unit 603 is further configured to:

[0181] The ratio between the fatigue cumulative value and a preset correction formula is used as the corrected fatigue cumulative value, wherein the preset correction formula is determined based on the difference between the first specified value and the power of the first preset weight value, and the exponent of the power is the number of video frames in the target time period; the driving state of the target object is analyzed based on the corrected fatigue cumulative value.

[0182] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0183] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0184] In some possible implementations, the fatigue detection device according to the present application may include at least a processor and a memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the fatigue detection method according to various exemplary implementations of the present application described in this specification. For example, the processor may execute the following steps: Figure 2 Follow the steps shown in .

[0185] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device can be as follows: Figure 7As shown, it includes a memory 701 , a communication module 703 and one or more processors 702 .

[0186] The memory 701 is used to store computer programs executed by the processor 702. The memory 701 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0187] The memory 701 may be a volatile memory, such as a random-access memory (RAM); the memory 701 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 701 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 701 may be a combination of the above memories.

[0188] The processor 702 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 702 is used to implement the above fatigue detection method when calling the computer program stored in the memory 701 .

[0189] The communication module 703 is used to communicate with terminal devices and other servers.

[0190] The specific connection medium between the memory 701, the communication module 703 and the processor 702 is not limited in the embodiment of the present application. Figure 7 In the embodiment, the memory 701 and the processor 702 are connected via a bus 704. The bus 704 is connected to the processor 702 via a bus 704. Figure 7 The connections between the other components are only for illustration and are not intended to be limiting. The bus 704 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 7 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0191] The memory 701 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the fatigue detection method of the embodiment of the present application. The processor 702 is used to execute the above fatigue detection method, such as Figure 2 shown.

[0192] In another embodiment, the electronic device may also be other electronic devices, such as Figure 1 The terminal device 110 shown in FIG. 1 is a terminal device 110 shown in FIG. 1 . In this embodiment, the structure of the electronic device can be as follows: Figure 8 As shown, it includes: a communication component 810, a memory 820, a display unit 830, a camera 840, a sensor 850, an audio circuit 860, a Bluetooth module 870, a processor 880 and other components.

[0193] The communication component 810 is used to communicate with the server. In some embodiments, a wireless fidelity (WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The electronic device can help the user to send and receive information through the WiFi module.

[0194] The memory 820 can be used to store software programs and data. The processor 880 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 820. The memory 820 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 820 stores an operating system that enables the terminal device 110 to run. In the present application, the memory 820 can store an operating system and various application programs, and can also store code for executing the fatigue detection method of the embodiment of the present application.

[0195] The display unit 830 can also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 830 may include a display screen 832 disposed on the front of the terminal device 110. The display screen 832 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 830 may be used to display the resource display interface in the embodiment of the present application, etc.

[0196] The display unit 830 can also be used to receive input digital or character information and generate signal input related to user settings and function control of the terminal device 110. Specifically, the display unit 830 may include a touch screen 831 arranged on the front of the terminal device 110, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0197] The touch screen 831 may be covered on the display screen 832, or the touch screen 831 and the display screen 832 may be integrated to realize the input and output functions of the terminal device 110, and the integrated touch screen may be referred to as a touch display screen. In the present application, the display unit 830 may display the application and the corresponding operation steps.

[0198] The camera 840 can be used to capture still images, and users can post comments on the images taken by the camera 840 through the application. The camera 840 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the processor 880 to convert it into a digital image signal.

[0199] The terminal device may further include at least one sensor 850, such as an acceleration sensor 851, a distance sensor 852, a fingerprint sensor 853, and a temperature sensor 854. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0200] The audio circuit 860, the speaker 861, and the microphone 862 can provide an audio interface between the user and the terminal device 110. The audio circuit 860 can transmit the electrical signal converted from the received audio data to the speaker 861, which is converted into a sound signal for output. The terminal device 110 can also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 862 converts the collected sound signal into an electrical signal, which is received by the audio circuit 860 and converted into audio data, and then the audio data is output to the communication component 810 to be sent to, for example, another terminal device 110, or the audio data is output to the memory 820 for further processing.

[0201] The Bluetooth module 870 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 870 to exchange data.

[0202] The processor 880 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 820 and calling data stored in the memory 820. In some embodiments, the processor 880 may include one or more processing units; the processor 880 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 880. In the present application, the processor 880 can run the operating system, application programs, user interface display and touch response, as well as the fatigue detection method of the embodiment of the present application. In addition, the processor 880 is coupled to the display unit 830.

[0203] In some possible implementations, various aspects of the fatigue detection method provided in the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the fatigue detection method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the following steps: Figure 2 Follow the steps shown in .

[0204] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0205] The program product of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a computing device. However, the program product of the present application is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, device or device.

[0206] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0207] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0208] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user equipment, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0209] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0210] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0211] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0212] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0213] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A fatigue detection method, It is characterized in that The method includes: Acquire each frame of video image corresponding to the target object during the driving process, and determine the fatigue degree corresponding to each frame of the video image according to the facial information of the target object in each frame of the video image; According to the determined fatigue degree, the weighted fatigue degree corresponding to the target object in each frame of the video is determined in sequence, wherein the weighted fatigue degree of the target object in the current frame is: the sum of the product of the weighted fatigue degree corresponding to the target object in the previous frame and the first preset weight value, and the product of the fatigue degree corresponding to the target object in the current frame and the second preset weight value; The weighted fatigue degree corresponding to the last frame in the target period is used as the cumulative fatigue value of the target object in the target period, and the driving state of the target object is analyzed based on the cumulative fatigue value.

2. The method according to claim 1, It is characterized in that The sum of the first preset weight value and the second preset weight value is a first specified value, and the first preset weight value is determined based on a detection frame rate of the video frame and a required number of video frames.

3. The method according to claim 1, It is characterized in that If the current frame is the first frame corresponding to the target object in the driving process, the accumulated fatigue value corresponding to the target object in the previous frame is the second specified value.

4. The method according to claim 1, It is characterized in that The target period is the remaining driving period of the target object except the initial driving period during the driving process, and the initial driving period is a certain period of time from the start of driving during the driving process of the target object; The analyzing the driving state of the target object based on the fatigue accumulated value includes: Acquire a first difference between a fatigue reference parameter corresponding to the target object in the initial driving period and a fatigue cumulative value corresponding to the target object in the remaining driving period, wherein the fatigue reference parameter is the fatigue cumulative value of the target object in a sober state; If the first difference is greater than the target threshold, it is determined that the target object is in a fatigue state.

5. The method according to claim 4, It is characterized in that The target threshold is obtained by: Determine a second difference between the accumulated fatigue values ​​of each sample subject in the awake state and the accumulated fatigue value of each sample subject in the fatigue state according to the accumulated fatigue value of each sample subject in the awake state and the accumulated fatigue value of each sample subject in the fatigue state in the sample database; The product of the mean of the second difference values ​​and the preset ratio value is used as the target threshold.

6. The method according to claim 4, It is characterized in that The method further comprises: Determining the number of operations performed by the target subject on the vehicle according to the driving state of the vehicle driven by the target subject; According to the changing trend of the number of operations within a specified time period, it is determined whether the target object is in a fatigue state.

7. The method according to claim 6, It is characterized in that The method further comprises: If it is determined that the target object is in a false fatigue state, feeding back fatigue driving prompt information to the target object; If it is determined that the target object is in a true fatigue state, the vehicle speed is controlled to decrease until it is determined that the target object is in a sober state according to the fatigue accumulation value and the driving state of the vehicle, and the original vehicle speed is restored.

8. The method according to claim 4, It is characterized in that The fatigue reference parameter is determined by: According to a preset time interval, the starting driving period is divided into a plurality of sub-periods; for each sub-period, the weighted fatigue degree of the target object corresponding to each frame of the video in each sub-period is determined in sequence; the weighted fatigue degree corresponding to the last frame in each sub-period is used as the cumulative fatigue value of the target object in each sub-period; and the average of the cumulative fatigue values ​​corresponding to the target object in each sub-period is used as the fatigue reference parameter; or sequentially determining a weighted fatigue degree of the target object corresponding to each video frame during the initial driving period; Using the weighted fatigue degree corresponding to the last frame in the initial driving period as the fatigue degree reference parameter; or The average value of fatigue degree corresponding to each video frame of the target object in the initial driving period is used as the fatigue degree reference parameter.

9. The method according to any one of claims 1 to 8, It is characterized in that The analyzing the driving state of the target object based on the fatigue accumulated value further includes: The ratio between the fatigue accumulated value and a preset correction formula is used as the corrected fatigue accumulated value, wherein the preset correction formula is determined based on the difference between the first specified value and the power of the first preset weight value, and the exponent of the power is the number of video frames in the target time period; The driving state of the target object is analyzed based on the corrected fatigue cumulative value.

10. A fatigue detection device, It is characterized in that include: An acquisition unit, used for acquiring each frame of video image corresponding to the target object during driving, and determining the fatigue degree corresponding to each frame of video image according to the facial information of the target object in each frame of video image; A determination unit, configured to sequentially determine the weighted fatigue degree of the target object corresponding to each frame of the video according to the determined fatigue degree, wherein the weighted fatigue degree of the target object in the current frame is: the sum of the product of the accumulated fatigue degree value corresponding to the target object in the previous frame and the first preset weight value, and the product of the fatigue degree corresponding to the target object in the current frame and the second preset weight value; The analysis unit is used to use the weighted fatigue corresponding to the last frame in the target time period as the cumulative fatigue value of the target object in the target time period, and analyze the driving state of the target object based on the cumulative fatigue value.

11. An electronic device, It is characterized in that It comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of any one of the methods in claims 1 to 9.

12. A computer-readable storage medium, It is characterized in that It includes program code which, when the storage medium runs on an electronic device, is used to cause the electronic device to execute the steps of any one of the methods recited in claims 1 to 9.

Images