Fatigue detection method, apparatus, and readable storage medium

Abstract

The embodiment of the present application provides a kind of fatigue detection method, device and readable storage medium.The fatigue detection method of the embodiment of the present application: by obtaining the eye image of driver, eye image is input into target convolutional neural network, to obtain the eye state information contained in eye image, eye state information is used to indicate the opening degree of the eye of driver, target convolutional neural network is obtained by using the eye image sample collected in advance to train convolutional neural network, according to the eye state information contained in each of the multiple frames of eye image of driver, whether driver is in fatigue state is determined.Thereby it can be more easily determined whether driver is in fatigue state, and by target convolutional neural network, whether driver is in fatigue state can be realized in real time.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a fatigue detection method, apparatus, and readable storage medium. Background Technology

[0002] With the growth of the social economy and the improvement of science and technology, the number of motor vehicles is increasing daily, bringing great convenience to people's travel. Cars have become one of the most popular means of transportation, but this has also led to a more frequent occurrence of road traffic accidents. According to incomplete statistics, a considerable portion of these traffic accidents are caused by driver fatigue. Therefore, in order to reduce the accident rate and protect people's lives and property, it is urgent to solve the problem of driver fatigue.

[0003] Existing technologies detect driver fatigue using methods based on vehicle trajectory. These methods focus on deviations from the vehicle's direction caused by driver behavior, such as detecting the driver's steering wheel angle, steering wheel pressure, or lane departure distance. However, this method is susceptible to interference from factors such as individual driving habits, driving speed, road conditions, and driving skills, increasing the difficulty of determining driver fatigue. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a fatigue detection method, apparatus and readable storage medium that overcomes or at least partially solves the above problems.

[0005] A first aspect of the present invention provides a fatigue detection method, comprising:

[0006] Obtain an image of the driver's eyes;

[0007] The eye image is input into a target convolutional neural network to obtain eye state information contained in the eye image. The eye state information is used to represent the degree of eye opening of the driver. The target convolutional neural network is obtained by training the convolutional neural network with pre-collected eye image samples.

[0008] Having obtained the eye state information of each of the multiple eye images, the system determines whether the driver is fatigued based on the eye state information of each of the multiple eye images.

[0009] A second aspect of the present invention provides a fatigue detection device, comprising:

[0010] The module is used to obtain an image of the driver's eyes;

[0011] An input module is used to input the eye image into a target convolutional neural network to obtain eye state information contained in the eye image. The eye state information is used to represent the degree of eye opening of the driver. The target convolutional neural network is obtained by training the convolutional neural network using pre-collected eye image samples.

[0012] The determination module is used to determine whether the driver is in a state of fatigue based on the eye state information of each of the multiple eye images, provided that the eye state information of each of the multiple eye images has been obtained.

[0013] A third aspect of the present invention provides a computer-readable storage medium comprising:

[0014] A computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements the steps of the fatigue detection method described above.

[0015] A fourth aspect of the present invention provides a fatigue detection device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the fatigue detection method described in any of the preceding claims.

[0016] The embodiments of the present invention have the following advantages:

[0017] This invention discloses a fatigue detection method, apparatus, and readable storage medium. By obtaining eye images of the driver and inputting these images into a target convolutional neural network (CNN), the eye state information contained in the eye images is obtained. This state information indicates the degree of eye opening of the driver. The target CNN is trained using pre-collected eye image samples. Based on the eye state information contained in multiple frames of the driver's eye images, the system determines whether the driver is fatigued. This makes it easier to determine whether the driver is fatigued, and the target CNN enables real-time detection of driver fatigue.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0020] Figure 1 This is a flowchart illustrating the steps of a fatigue detection method provided in an embodiment of the present invention.

[0021] Figure 2 A flowchart illustrating the steps of another fatigue detection method provided in an embodiment of the present invention;

[0022] Figure 3 A flowchart illustrating the specific steps of a fatigue detection method provided in an embodiment of the present invention;

[0023] Figure 4 A flowchart illustrating the specific steps of another fatigue detection method provided in this embodiment of the invention;

[0024] Figure 5 This is a schematic diagram of the structure of a fatigue detection device provided in an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of another fatigue detection device provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are merely some embodiments of the present invention, not all embodiments, and are not intended to limit the present invention.

[0028] Figure 1 This is a flowchart illustrating the steps of a fatigue detection method provided in an embodiment of the present invention. The fatigue detection method provided in this embodiment is applicable to determining whether a driver is experiencing fatigue. This fatigue detection method can be executed by a fatigue detection device, which is typically implemented in software and / or hardware. (Refer to...) Figure 1 The method in this embodiment includes the following steps:

[0029] S101, Obtain the driver's eye image.

[0030] The driver's eye image can be either an RGB image or an infrared image. RGB color refers to the three primary colors: R stands for Red, G for Green, and B for Blue. RGB represents the colors of the three channels: red, green, and blue. For example, when a driver is driving at night or in low light conditions, the image of the driver's eyes is an infrared image captured by an infrared camera. In well-lit environments, the image is an RGB image captured by an RGB camera.

[0031] S102. Input the eye image into the target convolutional neural network to obtain the eye state information contained in the eye image. The state information is used to represent the degree of eye opening of the driver. The target convolutional neural network is obtained by training the convolutional neural network using pre-collected eye image samples.

[0032] An eye image is input into a target convolutional neural network. The network processes the image to obtain a data point representing the degree of eye opening (e.g., a value between 0 and 1). Here, 0 represents a closed eye, and 1 represents an eye wide open enough to see the entire iris. It should be noted that the data representing eye opening can also be within other ranges, such as a value between 1 and 10, where 1 represents a closed eye and 10 represents an eye wide open enough to see the entire iris. This invention does not limit the specific format of the data representing eye opening.

[0033] Pre-collected eye image samples serve as training data. For example, the training data could consist of 100,000 eye images generated by the UnityEye rendering tool. UnityEyes is a 3D rendering tool that can generate a wide variety of left-eye images by setting camera and line-of-sight parameters. By setting the camera parameters of UnityEyes to (0, -20, 30, 50) and the line-of-sight parameters to (30, 0, 20, 45), left-eye images taken under different lighting conditions and at different camera positions and angles can be obtained. For example, a left-eye photo taken in low light with the camera positioned to the side. After training a convolutional neural network using the training data, a target convolutional neural network is obtained. This target convolutional neural network processes the eye images to obtain the eye state information contained within the images, that is, information representing the degree to which the human eye is open. Since the training data includes eye images from different camera positions and under different lighting conditions, the target convolutional neural network (CNN) trained on this data can estimate the eye state information in eye images taken from different angles and under different lighting conditions. In other words, the target CNN obtains the eye state information contained in eye images under different scenarios. Because drivers may not always be looking straight ahead, and may drive in low-light conditions such as at night, the target CNN can also estimate the driver's eye state information in such scenarios, thus demonstrating strong practicality.

[0034] S103. Given the eye state information of each of the multiple eye images, determine whether the driver is fatigued based on the eye state information of each of the multiple eye images.

[0035] It should be noted that for the first five seconds after the car starts moving, it is not necessary to determine whether the driver is fatigued. Starting from the first eye image obtained at the sixth second, the first eye image obtained at the sixth second is input into the target convolutional neural network to obtain the eye state information contained in the first eye image. Subsequently, the second eye image is also input into the target convolutional neural network to obtain the eye state information contained in the second eye image. After that, for each eye image obtained, that eye image is input into the target convolutional neural network to obtain the eye state information contained in that eye image. Based on the eye state information contained in each of the multiple obtained eye images, it can be determined whether the driver is fatigued. For example, in the first 18,000 frames of images, the eye state information contained in each frame of the eye image is between 0.8 and 1. In the 18,001st frame of the eye image, the eye state information contained is 0.7, in the 18,002nd frame of the eye image it is 0.6, in the 18,003rd frame of the eye image it is 0.5, and in the 18,004th and 18,005th frames of the eye image it is 0.4. If the eye state information contained in the subsequent 18,006th to 18,080th frames of the eye image is between 0 and 0.3, then the state information between 0 and 0.3 can be considered as the driver being in a state of almost closed eyes. If 30 frames of eye images can be obtained per second, then if the eye state information contained in 75 frames of the eye image is between 0 and 0.3, then the time the driver is almost closed eyes is considered to be 2.5 seconds. The almost closed eyes time is too long, and it can be determined that the driver is in a state of fatigue. If the eye state information contained in the eye image of frame 18006 is 0.5, the eye state information contained in the eye image of frame 18007 is 0.6, and the eye state information contained in the eye images of frames 18008 to 18080 is between 0.7 and 1, then it can be determined that the driver is not in a state of fatigue.

[0036] It should be noted that, since the fatigue detection method provided in this embodiment only analyzes and processes the driver's eye images obtained through the target convolutional neural network to obtain the eye state information contained in the eye images, and then determines whether the driver is in a fatigue state based on the eye state information contained in each of the driver's multiple eye images, this scheme is not affected by interference factors such as personal driving habits, driving speed, road environment, and operating skills, and can more easily determine whether the driver is in a fatigue state. Furthermore, through the lightweight target convolutional neural network, it is possible to achieve real-time detection of whether the driver is in a fatigue state.

[0037] The fatigue detection method provided in this embodiment obtains eye images of the driver and inputs these images into a target convolutional neural network (CNN) to obtain eye state information contained in the eye images. This state information represents the degree of eye opening of the driver. The target CNN is trained using pre-collected eye image samples. Based on the eye state information contained in multiple frames of the driver's eye images, it determines whether the driver is fatigued. This makes it easier to determine whether the driver is fatigued, and the target CNN enables real-time detection of driver fatigue.

[0038] Based on the above embodiments, referring to Figure 2 , Figure 2 A flowchart illustrating another fatigue detection method provided in an embodiment of the present invention. The fatigue detection method provided in this embodiment may include the following steps:

[0039] S201, obtained the driver's facial image.

[0040] S202. Obtain the coordinates of the feature points of the eyes contained in the face image.

[0041] Facial feature points can be extracted from facial images using keypoint detection algorithms (such as the MegFace toolkit). This toolkit can accurately and quickly extract the coordinates of facial feature points. The coordinates of facial feature points include those of eyebrows, eyes, nose, mouth, and other facial regions. The coordinates of the eyes can be extracted from these facial feature point coordinates, including those of the left and right eyes. Alternatively, other libraries can be used to extract feature points, such as the Dlib library. The Dlib library, written in C++, provides a range of functionalities related to machine learning, numerical computation, graphical modeling algorithms, and image processing.

[0042] S203. Determine the image area that frames the eyes based on the coordinates of the feature points of the eyes.

[0043] Determining the image region that frames the eyes based on the coordinates of the eye's feature points can be achieved through the following steps:

[0044] Determine the area of ​​the smallest first rectangle that frames the left / right eye based on the coordinates of the feature points of the left / right eye, and determine the area of ​​the smallest second rectangle that frames the right / left eye based on the coordinates of the feature points of the right / left eye; select the largest area from the areas of the first and second rectangles; and determine the rectangle corresponding to the largest area as the image region that frames the eye.

[0045] For example, if the area of ​​the first rectangle corresponding to the left eye is greater than the area of ​​the second rectangle corresponding to the right eye, then the first rectangle is defined as the image area that frames the left eye.

[0046] By selecting a larger rectangular frame to define the image region that encloses the eye, when the driver makes a playful blink (e.g., when the driver intentionally blinks one eye while the other eye remains open), the image region corresponding to the open eye can be selected. This allows for a more accurate analysis of the driver's current state, i.e., a more accurate analysis of whether the driver is fatigued, during subsequent steps.

[0047] S204. Based on the image area that frames the eyes, crop the face image to obtain the driver's eye image.

[0048] If the image area that frames the eyes is the same as the image area that frames the left eye, then the face image is cropped to obtain the image area that frames the left / right eyes.

[0049] S205. Input the eye image into the target convolutional neural network to obtain the eye state information contained in the eye image.

[0050] If the image region containing the left eye is input into the target convolutional neural network, the eye state information contained in the left eye image can be obtained.

[0051] Before inputting the eye image into the target convolutional neural network, it is necessary to obtain the target convolutional neural network, which is obtained through the following steps:

[0052] Eye image samples are obtained using a rendering tool; based on the coordinates of the feature points of the left / right eyes contained in the eye image samples, the eye image samples are cropped to obtain images of the left / right eyes that frame the eye image samples; the images of the left / right eyes that frame the eye image samples are flipped to become images of the right / left eyes; the images of the left / right eyes that frame the eye image samples and the flipped images of the right / left eyes are input into a convolutional neural network for training to obtain a target convolutional neural network.

[0053] Specifically, based on the coordinates of feature points of the left and right eyes contained in the eye image sample, the distance between the upper and lower eyelids and the iris diameter can be calculated to obtain data representing the degree of eye opening. This data is then used as a label for the eye image sample to train the convolutional neural network. For example, the degree of eye opening is equal to the distance between the upper and lower eyelids divided by the iris diameter. If the image of the left eye, which is framed within the eye image sample, is input into the convolutional neural network for training, the network will output data representing the degree of eye opening. The closer this data is to the degree of eye opening obtained by dividing the distance between the upper and lower eyelids by the iris diameter, the closer the training of the convolutional neural network is to convergence.

[0054] It should be noted that if the image of the left eye, which frames the eye image sample, is obtained, then the image of the left eye is flipped to become the image of the right eye; conversely, if the image of the right eye, which frames the eye image sample, is obtained, then the image of the right eye is flipped to become the image of the left eye. The target convolutional neural network is a lightweight target convolutional neural network. Lightweight means that the target convolutional neural network has few parameters and low computational cost, ensuring rapid output of eye state information, which meets the speed requirements of practical applications.

[0055] The training of the convolutional neural network involves inputting the left / right eye images (which frame the eye image samples) and their flipped versions into the right / left eye images. This can be achieved in the following way:

[0056] If the left and right eye images of the framed eye image sample are RGB images, then the red channels of the left and right eye images of the framed eye image sample are input into the convolutional neural network for training;

[0057] If the left and right eye images of the eye image sample are infrared images, then the left and right eye images of the eye image sample are input into the convolutional neural network for training.

[0058] It should be noted that if the training data obtained through the rendering tool includes the left eye, the left eye needs to be randomly flipped to resemble the right eye, so that the training data input to the convolutional neural network includes images of both the left and right eyes; conversely, if the training data obtained through the rendering tool includes the right eye, the right eye needs to be randomly flipped to resemble the left eye, so that the training data input to the convolutional neural network includes images of both the left and right eyes. Furthermore, to enable the trained model (i.e., the target neural network) to be applied simultaneously to estimate human eye state information from images captured by both RGB and infrared cameras, during training, if the eye image is an RGB image, the red channel of the RGB image is used as the input to the convolutional neural network; if the eye image is an infrared image, it can be directly input into the convolutional neural network for training. The rendering tool can be the UnityEyes rendering tool.

[0059] S206. Determine whether the driver is fatigued based on the eye state information contained in each of the driver's multiple eye images.

[0060] Specifically, refer to Figure 3 , Figure 3 The flowchart illustrates the specific steps of a fatigue detection method provided in this embodiment of the invention. S206: Determining whether the driver is fatigued based on the eye state information contained in each of the multiple frames of the driver's eye images can be achieved through the following steps:

[0061] S301. Determine correction parameters based on the eye state information in each frame of the driver's eye image acquired within the first preset time period.

[0062] Specifically, correction parameters are determined based on the eye state information in each frame of the driver's eye images acquired within a first preset time period. The average value of the largest n preset number of state information values ​​within the first preset time period is taken. For example, if the first preset time period is 5 seconds and the n preset number of values ​​is 10, and 30 frames of eye images can be acquired per second, then 150 state information values ​​will be acquired from the 150 acquired eye images. The 10 state information values ​​with the largest values ​​are selected from these 150 state information values, and the average value of these 10 state information values ​​is taken as the correction parameter. In this embodiment of the invention, the number of values ​​for n is not limited.

[0063] S302. Based on the correction parameters, correct the eye state information contained in each of the driver's multi-frame eye images.

[0064] The correction of eye state information contained in each of the driver's multi-frame eye images can be achieved in two ways: In one possible implementation, the eye state information contained in each of the driver's multi-frame eye images can be corrected according to a correction parameter. The corrected eye state information is equal to the eye state information estimated by the target convolutional neural network divided by the correction parameter. For example, if the eye state information contained in the current frame eye image estimated by the target convolutional neural network is equal to 0.75, and the correction parameter is equal to 0.8, then the corrected eye state information contained in the current frame eye image is equal to 0.75 divided by 0.8.

[0065] In another possible implementation, the eye state information contained in each of the driver's multiple eye images can be corrected based on the correction parameters and the driver's head deflection angle corresponding to each of the multiple eye images. The corrected eye state information is equal to the eye state information estimated by the target convolutional neural network divided by the cosine of the correction parameters and the head deflection angle. It should be noted that the driver's head deflection angle corresponding to each of the multiple eye images can be obtained. One eye image corresponds to one head deflection angle, that is, for each eye image obtained, the head deflection angle corresponding to that eye image must be obtained. The head deflection angle can be obtained using a keypoint detection algorithm (such as the MegFace toolkit). For example, when the driver is looking straight ahead, the head deflection angle is 0 degrees; when the driver raises their head by 10 degrees, the head deflection angle is 350 degrees; and when the driver lowers their head by 10 degrees, the driver's head deflection angle is 10 degrees.

[0066] It should be noted that because the driver's head tilt can cause the same eye's state information to appear differently in the eye image, the eye opening calculated by inputting the eye image into the same convolutional neural network will also be different. Therefore, when the head tilt angle is taken into account to correct the eye state information contained in each of the multiple frames of eye images, the corrected eye state information can be made more accurate.

[0067] Specifically, the method for correcting the eye state information obtained by the target convolutional neural network based on the correction parameters and the head deflection angle of the driver corresponding to each of the multiple eye images is as follows: If the correction parameter is determined to be 0.8 based on the 150 eye images in the first 5 seconds, then the first eye image is obtained in the 6th second. The head deflection angle of the driver corresponding to the first eye image is 350 degrees. The eye state information contained in the first eye image estimated by the target convolutional neural network is 0.75. Then, the corrected eye state information can be obtained by dividing 0.75 by the product of the correction parameter and the cosine value of 350. That is, the eye state information contained in the corrected first eye image is 0.75 divided by the product of the correction parameter and the cosine value of 350.

[0068] By correcting the eye state information contained in the eye image estimated by the target convolutional neural network, the corrected eye state information can be made closer to the true value, thereby enabling a more accurate determination of whether the driver is fatigued.

[0069] S303. Determine whether the driver is fatigued based on the eye state information contained in each of the corrected multi-frame eye images of the driver.

[0070] Specifically, S303, based on the eye state information contained in each of the corrected multi-frame eye images of the driver, it is determined whether the driver is fatigued, which can be achieved in the following three ways;

[0071] The first approach may include the following steps:

[0072] The driver's blinking process is determined based on the changes in eye state information contained in each of the corrected multi-frame eye images.

[0073] The first statistical frame count and the total frame count during the blinking process are determined. The first statistical frame count is determined based on the number of frames in the eye image during the blinking process that are less than or equal to a preset first threshold, and the total frame count is determined based on the number of frames in the eye image during the blinking process that are less than or equal to a preset second threshold, wherein the first threshold is less than the second threshold.

[0074] If the ratio of the first statistical frame count to the total frame count is greater than or equal to a preset third threshold and the first statistical frame count is greater than or equal to a preset fourth threshold, then the driver is determined to be in a state of fatigue.

[0075] For example, the first eye image, obtained at the 6th second, is input into the target convolutional neural network to obtain the eye state information contained in the first eye image. This eye state information is then corrected to obtain the corrected eye state information. The second eye image is then also input into the target convolutional neural network to obtain the eye state information contained in the second eye image. This state information is then corrected to obtain the corrected eye state information. This process continues for each subsequent eye image, where the obtained eye image is input into the target convolutional neural network to obtain the eye state information contained in that frame, and this state information is then corrected to obtain the corrected eye state information. For example, when obtaining the first 18,000 frames of eye images (e.g., 18,000 frames of eye images obtained within 10 minutes), the eye state information contained in each corrected frame of eye image is between 0.8 and 1. In the 18,001st frame of eye image, the corrected eye state information contained in this frame is 0.7 (if the second threshold is 0.7). Therefore, the 18,001st frame is determined as the blink start frame, and the initial total frame count is 0. At this point, the total frame count is incremented by 1, meaning the total frame count equals 1. The eye state information contained in the 18,002nd frame of eye image is 0.6. The total frame count is then incremented by 1 again, resulting in a total frame count of 2. The corrected 18,001st frame... The eye state information of frame 03 is 0.5. After adding 1 to the total frame count, the total frame count is 3. The corrected eye state information of frames 18004 and 18005 is 0.4, and the total frame count is 5. If the corrected eye state information of frame 18006 is 0.3 (if the first threshold is 0.3, that is, if the corrected eye state information of the current frame is less than or equal to 0.3, the frame is considered to be an almost closed eye frame or a closed eye frame), then the first statistical frame count is incremented by 1. The initial value of the first statistical frame count is 0, that is, after incrementing by 1, the first statistical frame count is equal to 1, and the total frame count is incremented by 1, and the total frame count is equal to 6. Subsequently, for each corrected frame of eye image where the eye state information is less than or equal to 0.3, the first statistical frame count is incremented by 1 and the total frame count is also incremented by 1. If the eye state information contained in each of the corrected frames 18007 to 18080 is between 0 and 0.3, then at frame 18080, the first statistical frame count is equal to 75 and the total frame count is equal to 80.If the eye state information contained in the corrected 18081st frame eye image is 0.4, then the first statistical frame count is not incremented by 1 (because 0.4 is greater than the set first threshold of 0.3), and the total frame count is incremented by 1, resulting in a total frame count of 81. If the eye state information contained in the corrected frames 18082 to 18100 are greater than 0.4 and less than 0.7, meaning the eye opening degree is increasing, then during this period, the first statistical frame count is not incremented by 1, but the total frame count needs to be continuously incremented by 1. The frame count is 100. If the eye state information contained in the corrected frame 18101 is 0.7, the total frame count is incremented by 1 to 101, and the first statistical frame count is 75. If the eye state information contained in the corrected frame 18102 is 0.8 (because 0.8 is greater than the set second threshold), neither the total frame count nor the first statistical frame count is incremented by 1 at frame 18102. The blinking process ends at frame 18101, meaning the blinking process occurs between frames 18001 and 18101. The ratio of the first statistical frame count to the total frame count at the end of the blinking process is 75 divided by 101. If this ratio is greater than the preset third threshold and the first statistical frame count is greater than or equal to the fourth threshold, it can be determined that the driver is fatigued. For example, if the third threshold is 0.6, this ratio is greater than 0.6; if the fourth threshold is 5, and the first statistical frame count is greater than 5, it can be determined that the driver is fatigued. Once it is determined that the driver is fatigued, voice prompts can be output to remind the driver and prevent traffic accidents caused by driver fatigue.

[0076] It should be noted that by determining whether the driver is fatigued by judging whether the ratio of the first statistical frame count to the total frame count is greater than or equal to a preset third threshold and whether the first statistical frame count is greater than or equal to a preset fourth threshold, it is possible to avoid misjudging the driver's rapid blinking as fatigue. For example, if the ratio of the first statistical frame count (first statistical frame count equals 4) to the total frame count (total frame count equals 5) is greater than the third threshold (third threshold equals 0.6), but the first statistical frame count is 4 and less than the fourth threshold (fourth threshold equals 5), the driver is not judged to be fatigued because the obtained first statistical frame count is less than 5 frames.

[0077] The second approach may include the following steps:

[0078] The driver's blinking process is determined based on the changes in eye state information contained in each of the corrected multi-frame eye images.

[0079] Determine the second statistical frame number during the blinking process, wherein the second statistical frame number is determined based on the number of frames in the corrected eye image during the blinking process where the eye state information is less than or equal to a preset fifth threshold;

[0080] If the second statistical frame count is greater than or equal to the preset sixth threshold, then the driver is determined to be in a state of fatigue.

[0081] This method differs from the first method described above in that the fifth threshold in this method has the same meaning as the first threshold in the previous method. That is, if the eye state information of the corrected current frame eye image is less than or equal to the fifth threshold, the frame is considered a nearly closed eye frame or a closed eye frame. This method only needs to determine the second statistical frame count, which is the same as the first statistical frame count described above and will not be repeated here. The sixth threshold can be set to 60 frames. For example, if the second statistical frame count is 60 frames, which equals the sixth threshold, the driver can be considered to be in a fatigued state. That is, if 30 frames of eye images are obtained per second, and 60 frames are nearly closed eye frames or closed eye frames, it means the eye-closing time is relatively long, indicating that the driver is fatigued. After determining that the driver is fatigued, a voice prompt can be output to remind the driver and prevent traffic accidents caused by driver fatigue.

[0082] The third method involves determining whether the driver is fatigued based on the eye state information contained in each of the multiple eye images corrected within a second preset time period. This method differs from the previous two in that the second preset time period can be used to determine whether the driver is fatigued over a given period. For example, if the proportion of almost closed eyes and closed frames does not increase significantly during a blink, but the blinking frequency increases significantly over a period, this is also a sign of fatigue. The number of fatigue frames within the second preset time period can be used to further determine whether the driver is fatigued. Additionally, it should be noted that if the driver is so fatigued that their eyes are no longer fully open and they are constantly squinting and dozing off, this situation also requires the second preset time period to determine whether the driver is fatigued. Specifically, this method may include the following steps, as detailed below. Figure 4 , Figure 4 A flowchart illustrating the specific steps of another fatigue detection method provided in an embodiment of the present invention.

[0083] The meaning of the seventh threshold in this method is the same as that of the first threshold in the first method and the fifth threshold in the second method. That is, if the eye state information of the corrected current frame eye image is less than or equal to 0.3 (if 0.3 is set as the seventh threshold), the frame is considered to be an almost closed eye frame or a closed eye frame.

[0084] S401. When the driver's current frame eye image is obtained, determine whether the eye state information contained in the corrected first frame eye image within a second preset time period is less than or equal to a preset seventh threshold.

[0085] If it is determined that the eye state information contained in the corrected first frame eye image within the second preset time period is less than or equal to the seventh threshold, then S402 is executed. If it is determined that the eye state information contained in the corrected first frame eye image within the second preset time period is greater than the seventh threshold, then S403 is executed.

[0086] For example, if the second preset time is 1 minute, and 30 eye images can be obtained per second, a total of 1800 eye images are obtained in the first minute. After obtaining the 1801st eye image (the 1801st eye image is obtained in the first second of the second minute), the 1801st eye image is the current frame eye image of the driver. At this time, it is necessary to determine whether the eye state information contained in the corrected first eye image in the 1800 eye images obtained in the first minute is less than or equal to the preset seventh threshold. If the eye state information contained in the corrected first eye image in the 1800 eye images obtained in the first minute is equal to 0.2 (0.2 is less than the seventh threshold 0.3), then the first frame in the first minute is considered to be an almost closed eye frame or a closed eye frame. At this time, S402 needs to be executed. That is, if the number of third statistical frames counted in the first minute is equal to 4, then the current number of third statistical frames needs to be decremented by 1. That is, after S402 is executed, the current number of third statistical frames is equal to 3, so as to ensure that the number of third statistical frames within 1 minute from the current time is always counted. If the corrected first eye image in the 1800 eye images obtained in the first minute contains eye state information equal to 0.4 (0.4 is greater than the seventh threshold of 0.3), then the first frame in the first minute is considered not to be an almost closed eye frame or a closed eye frame. At this time, S403 needs to be executed, that is, keep the current third statistical frame number unchanged. After S403 is executed, the current third statistical frame number is still equal to 4.

[0087] S402, Decrement the current third statistical frame count by 1.

[0088] The third statistical frame number is determined based on the number of frames in the corrected eye image within a preset statistical time period where the eye state information is less than or equal to the seventh threshold.

[0089] After S402 is executed, S404 is executed next.

[0090] S403, Keep the current third statistical frame number unchanged.

[0091] After S403 is executed, S404 will be executed next.

[0092] S404. Determine whether the eye state information contained in the current frame eye image of the corrected driver is less than or equal to the seventh threshold.

[0093] It should be noted that if S404 is executed after S402, and it is determined that the eye state information contained in the current frame eye image of the corrected driver is less than or equal to the seventh threshold, then S405 is executed. That is, if the eye state information contained in the current frame eye image of the corrected driver is equal to 0.1 (0.1 is less than the seventh threshold 0.3), in conjunction with the description in S401, it is necessary to add 1 to the current third statistical frame number after S402 is executed (the current third statistical frame number after S402 is executed is equal to 3) to obtain the third statistical frame number. The current third statistical frame number is equal to 3 plus 1, that is, the current third statistical frame number is equal to 4. That is, the current third statistical frame number obtained after S405 is executed (that is, the final third statistical frame number) is equal to 4. If it is determined that the eye state information contained in the current frame eye image of the corrected driver is greater than the seventh threshold, then S406 is executed. That is, if the eye state information contained in the current frame eye image of the corrected driver is equal to 0.5 (0.5 is greater than the seventh threshold 0.3), in conjunction with the description in S401, it is necessary to maintain the third statistical frame number after S402 is executed, that is, maintain the current third statistical frame number after S402 is executed, that is, the current third statistical frame number is still equal to 3, that is, the current third statistical frame number obtained after S406 is executed (that is, the final third statistical frame number) is equal to 3.

[0094] It should be noted that if S404 is executed after S403, and it is determined that the eye state information contained in the current frame eye image of the corrected driver is less than or equal to the seventh threshold, then S405 is executed. That is, if the eye state information contained in the current frame eye image of the corrected driver is equal to 0.1 (0.1 is less than the seventh threshold 0.3), in conjunction with the description in S401, the third statistical frame number after S403 is executed needs to be incremented by 1. The third statistical frame number after S403 is executed is equal to 4, that is, the third statistical frame number is equal to 4 plus 1. That is, the current third statistical frame number after S405 is executed (that is, the final third statistical frame number) is equal to 5, and then S407 is executed. If it is determined that the eye state information contained in the current frame eye image of the corrected driver is greater than the seventh threshold, then S406 is executed. That is, if the eye state information contained in the current frame eye image of the corrected driver is equal to 0.5 (0.5 is greater than the seventh threshold 0.3), in conjunction with the description in S401, it is necessary to maintain the third statistical frame number after S403 is executed. Since the third statistical frame number after S403 is executed is equal to 4, the third statistical frame number after S406 is executed (that is, the final third statistical frame number) is still equal to 4, and then S407 is executed.

[0095] S405, Increment the current third statistical frame count by 1.

[0096] S406. Keep the current third statistical frame number unchanged.

[0097] S407. Determine whether the final obtained third statistical frame count is greater than the eighth threshold, or determine whether the ratio of the final obtained third statistical frame count to the total number of driver's eye images obtained within the second preset time period is greater than or equal to the preset ninth threshold.

[0098] If the final number of third statistical frames is greater than the eighth threshold, or if the ratio of the final number of third statistical frames to the total number of frames of the driver's eye images obtained within the second preset time is greater than or equal to the preset ninth threshold, then execute S408; otherwise execute S409.

[0099] For example, after S403 is executed, S404 and S405 are executed. After S405 is executed, the final number of the third statistical frames is 5. If the second preset time is 1 minute, the ratio is obtained by dividing 5 by 30. For another example, after S403 is executed, S404 and S406 are executed. After S406 is executed, the final number of the third statistical frames is 4. The ratio is obtained by dividing 4 by 30.

[0100] S408. It is determined that the driver is fatigued.

[0101] S409. Determine that the driver is not fatigued.

[0102] S410. If it is determined that the driver is fatigued, output a voice prompt to remind the driver.

[0103] Once it is determined that the driver is fatigued, voice prompts are output to remind the driver and prevent traffic accidents caused by fatigued driving.

[0104] Figure 5 This is a schematic diagram of a fatigue detection device provided in an embodiment of the present invention. Fatigue detection devices are typically implemented in software and / or hardware. (Refer to...) Figure 5 The fatigue detection device 500 includes the following modules: an acquisition module 510, an input module 520, or a determination module 530.

[0105] The acquisition module 510 is used to acquire the driver's eye image; the input module 520 is used to input the eye image into the target convolutional neural network to obtain the eye state information contained in the eye image, the eye state information being used to represent the degree of eye opening of the driver, the target convolutional neural network being trained by using pre-collected eye image samples; the determination module 530 is used to determine whether the driver is in a state of fatigue based on the eye state information of each of the multiple frames of eye images.

[0106] The fatigue detection device provided in this embodiment obtains eye images of the driver and inputs these images into a target convolutional neural network to obtain eye state information contained in the eye images. This state information indicates the degree of eye opening of the driver. The target convolutional neural network is trained using pre-collected eye image samples. Based on the eye state information contained in multiple frames of the driver's eye images, it determines whether the driver is fatigued. This solves the problem of more easily determining whether a driver is fatigued, and through a lightweight target convolutional neural network, it enables real-time detection of driver fatigue.

[0107] Optionally, the determining module 530 is specifically used to determine correction parameters based on the eye state information in each frame of the driver's eye images acquired within a first preset time period; to correct the eye state information contained in each of the driver's multiple frame eye images based on the correction parameters; and to determine whether the driver is in a state of fatigue based on the eye state information contained in each of the corrected multiple frame eye images.

[0108] Optionally, the device 500 may also include an acquisition module for acquiring the driver’s head deflection angle corresponding to each of the multiple frames of eye images.

[0109] Accordingly, the determining module 530 is also used to correct the eye state information contained in each of the driver's multi-frame eye images based on the correction parameters and the driver's head deflection angle corresponding to each of the multi-frame eye images; and to determine whether the driver is in a state of fatigue based on the corrected eye state information contained in each of the driver's multi-frame eye images.

[0110] Optionally, the determining module 530 is specifically used to determine the driver's blinking process based on the changes in eye state information contained in each of the corrected multi-frame eye images of the driver; determine the first statistical frame count and the total number of frames in the blinking process, wherein the first statistical frame count is determined based on the number of frames in the blinking process where the state information of the eye images is less than or equal to a preset first threshold, and the total number of frames is determined based on the number of frames in the blinking process where the state information of the eye images is less than or equal to a preset second threshold, the first threshold being less than the second threshold; if the ratio of the first statistical frame count to the total number of frames is greater than or equal to a preset third threshold and the first statistical frame count is greater than or equal to a preset fourth threshold, then the driver is determined to be in a fatigued state.

[0111] Optionally, the determining module 530 is specifically used to determine the driver's blinking process based on the changes in eye state information contained in each of the corrected multi-frame eye images of the driver; determine the second statistical frame number in the blinking process, wherein the second statistical frame number is determined based on the number of frames in the corrected eye image state information during the blinking process that are less than or equal to a preset fifth threshold; if the second statistical frame number is greater than or equal to a preset sixth threshold, then the driver is determined to be in a fatigued state.

[0112] Optionally, the determining module 530 is specifically used to determine whether the driver is fatigued based on the eye state information contained in each of the multiple frames of eye images corrected within a second preset time period.

[0113] Optionally, the determining module 530 is specifically used to determine whether the eye state information contained in the current frame eye image of the driver after correction within a second preset time is less than or equal to a seventh threshold; if it is determined that the eye state information contained in the current frame eye image of the driver after correction is less than or equal to the seventh threshold, then the current third statistical frame count is incremented by 1; if the current third statistical frame count is greater than an eighth threshold, or the ratio of the current third statistical frame count to the total number of frames of the driver's eye images obtained within the second preset time is greater than or equal to a preset ninth threshold, then it is determined that the driver is in a fatigued state.

[0114] Optionally, the determination module 530 is also used for

[0115] When the driver’s current frame eye image is obtained, determine whether the eye state information contained in the corrected first frame eye image within the second preset time period is less than or equal to the seventh threshold.

[0116] If it is determined that the eye state information contained in the corrected first frame eye image within the second preset time period is less than or equal to the seventh threshold, then the current third statistical frame number is decremented by 1. The third statistical frame number is determined based on the number of frames within the preset time period where the corrected eye image state information is less than or equal to the seventh threshold.

[0117] Optionally, the determining module 530 is further configured to: if the eye state information contained in the corrected first frame eye image within the second preset time period is less than or equal to the seventh threshold, then decrement the current third statistical frame number by 1; determine whether the eye state information contained in the corrected current frame eye image of the driver is less than or equal to the seventh threshold; if the eye state information contained in the corrected current frame eye image of the driver is greater than the seventh threshold, then keep the current third statistical frame number unchanged; if the current third statistical frame number is greater than the eighth threshold, or the ratio of the current third statistical frame number to the total number of frames of the driver's eye images obtained within the second preset time period is greater than or equal to the ninth threshold, then determine that the driver is in a fatigued state.

[0118] Optionally, the determining module 530 is further configured to: if the eye state information contained in the corrected first frame eye image within the second preset time period is greater than the seventh threshold, maintain the current third statistical frame number; determine whether the eye state information contained in the corrected current frame eye image of the driver is less than or equal to the seventh threshold; if the eye state information contained in the corrected current frame eye image of the driver is less than or equal to the seventh threshold, increment the current third statistical frame number by 1; if the current third statistical frame number is greater than the eighth threshold, or the ratio of the current third statistical frame number to the total number of frames of the driver's eye images obtained within the second preset time period is greater than or equal to the ninth threshold, determine that the driver is in a fatigued state.

[0119] Optionally, the device may also include an output module for outputting voice prompts to remind the driver after determining that the driver is fatigued.

[0120] Optionally, module 510 is specifically used to obtain a driver's face image; obtain the coordinates of the feature points of the eyes in the face image; determine the image region that frames the eyes based on the coordinates of the feature points of the eyes; and crop the face image based on the image region that frames the eyes to obtain the ninth threshold driver's eye image.

[0121] Optionally, the obtaining module 510 is specifically used to obtain the coordinates of the feature points of the left eye and the right eye of the face image; correspondingly, the determining module 530 is specifically used to determine the area of ​​the smallest first rectangle that frames the left eye / right eye based on the coordinates of the feature points of the left eye / right eye, and to determine the area of ​​the smallest second rectangle that frames the right eye / left eye based on the coordinates of the feature points of the right eye / left eye; select the largest area from the areas of the first rectangle and the second rectangle; and determine the rectangle corresponding to the largest area as the image region that frames the eye.

[0122] Optionally, the device may further include a training module, which is used to obtain eye image samples by a rendering tool before inputting the ninth threshold eye image into the target convolutional neural network; to crop the eye image samples according to the coordinates of the feature points of the left / right eyes contained in the eye image samples to obtain images of the left / right eyes that frame the eye image samples; to flip the images of the left / right eyes that frame the eye image samples into images of the right / left eyes; and to input the images of the left / right eyes that frame the eye image samples and the flipped images of the right / left eyes into the convolutional neural network for training to obtain the target convolutional neural network.

[0123] Optionally, the training module is specifically used to input the red channels of the left and right eye images of the framed eye image sample into the convolutional neural network for training if the images of the left and right eyes of the framed eye image sample are RGB images.

[0124] If the left and right eye images of the eye image sample are infrared images, then the left and right eye images of the eye image sample are input into the convolutional neural network for training.

[0125] In addition, embodiments of the present invention also provide a fatigue detection device, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of another fatigue detection device provided in an embodiment of the present invention. The fatigue detection device 600 includes a processor 610, a memory 620, and a computer program stored in the memory 620 and executable on the processor 610. When the computer program is executed by the processor 610, it implements the various processes of the fatigue detection method embodiment described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0126] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the fatigue detection method embodiments described above and achieves the same technical effects. To avoid repetition, further details are omitted here. The computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0127] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0129] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] In a typical configuration, a computer device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media. Computer-readable media includes both permanent and non-persistent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media do not include transient media, such as modulated data signals and carrier waves.

[0131] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable live interactive terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable live interactive terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable live interactive terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable live interactive terminal device, causing a series of operational steps to be executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0135] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0136] The fatigue detection method, apparatus, and readable storage medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fatigue detection method characterized by, The method comprises: obtaining an eye image of a driver; inputting the eye image into a target convolutional neural network to obtain eye state information contained in the eye image, the state information being used to represent the opening degree of the driver's eyes, the target convolutional neural network being obtained by training a convolutional neural network by using pre-collected eye image samples; in a case where eye state information of each of a plurality of eye images is obtained, determining a correction parameter according to the eye state information in each of the plurality of eye images of the driver obtained within a first preset time; obtaining a head deflection angle of the driver corresponding to each of the plurality of eye images; and correcting the eye state information contained in each of the plurality of eye images of the driver according to the correction parameter and the head deflection angle of the driver corresponding to each of the plurality of eye images; determining whether the driver is in a fatigue state according to the corrected eye state information contained in each of the plurality of eye images of the driver, comprising: when a current eye image of the driver is obtained, determining whether the eye state information contained in a first corrected eye image within a second preset time is less than or equal to a seventh threshold value; if it is determined that the eye state information contained in the first corrected eye image within the second preset time is less than or equal to the seventh threshold value, then a third statistical frame number is decremented by 1, wherein the third statistical frame number is determined according to the number of frames of the eye images within the second preset time whose state information is less than or equal to the seventh threshold value.

2. The method of claim 1, wherein, The method further comprises: determining a blink process of the driver according to the change of the corrected eye state information contained in each of the plurality of eye images of the driver; determining a first statistical frame number in the blink process and a total frame number in the blink process, wherein the first statistical frame number is determined according to the number of frames of the eye images in the blink process whose state information is less than or equal to a first preset threshold value, the total frame number is determined according to the number of frames of the eye images in the blink process whose state information is less than or equal to a second preset threshold value, and the first threshold value is less than the second threshold value; if the ratio of the first statistical frame number to the total frame number is greater than or equal to a third preset threshold value and the first statistical frame number is greater than or equal to a fourth preset threshold value, then the driver is determined to be in a fatigue state.

3. The method of claim 1, wherein, The method further comprises: determining a blink process of the driver according to the change of the corrected eye state information contained in each of the plurality of eye images of the driver; determining a second statistical frame number in the blink process, wherein the second statistical frame number is determined according to the number of frames of the corrected eye images in the blink process whose state information is less than or equal to a fifth preset threshold value. If the second statistical frame number is greater than or equal to a preset sixth threshold value, it is determined that the driver is in a fatigue state.

4. The method of claim 1, wherein, The method further comprises: According to the eye state information contained in the corrected multiple frames of eye images of the driver within the second preset time, it is determined whether the driver is in a fatigue state.

5. The method of claim 4, wherein, The method further comprises: According to the eye state information contained in the corrected multiple frames of eye images of the driver within the second preset time, it is determined whether the driver is in a fatigue state. The method further comprises: It is determined whether the eye state information contained in the current frame of eye image of the driver within the second preset time is less than or equal to a seventh threshold value; 6. The method of claim 5, wherein, If it is determined that the eye state information contained in the current frame of eye image of the driver is less than or equal to the seventh threshold value, the current third statistical frame number is incremented by 1; If the current third statistical frame number is greater than an eighth threshold value, or the ratio of the current third statistical frame number to the total number of frames of eye images of the driver obtained within the second preset time is greater than or equal to a preset ninth threshold value, it is determined that the driver is in a fatigue state.

7. The method of claim 6, wherein, The method further comprises: If it is determined that the eye state information contained in the current frame of eye image of the driver is greater than the seventh threshold value, the current third statistical frame number is kept unchanged.

8. The method of claim 1, wherein, The method further comprises: If it is determined that the eye state information contained in the first frame of eye image within the second preset time is greater than the seventh threshold value, the current third statistical frame number is kept unchanged. The method further comprises: The method further comprises: The image of the left / right eye of the eye image sample is obtained by cropping the eye image sample according to the coordinates of the feature points of the left / right eye contained in the eye image sample; 9. The method of claim 8, wherein, The image of the left / right eye of the eye image sample is flipped into an image of the right / left eye; The image of the left / right eye of the eye image sample and the flipped image of the right / left eye are input into a convolutional neural network for training to obtain the target convolutional neural network. The image of the left / right eye of the eye image sample and the flipped image of the right / left eye are input into a convolutional neural network for training to obtain the target convolutional neural network.

10. The method of claim 1, wherein, If the image of the left eye and the image of the right eye of the eye image sample are RGB images, the red channel of the image of the left eye and the image of the right eye of the eye image sample is input into a convolutional neural network for training; 11. A fatigue detection device, characterized by, If the image of the left eye and the image of the right eye of the eye image sample are infrared images, the image of the left eye and the image of the right eye of the eye image sample is input into a convolutional neural network for training. The target convolutional neural network is a lightweight target convolutional neural network. The method further comprises: An obtaining module is configured to obtain an eye image of a driver. inputting the eye image into a target convolutional neural network to obtain eye state information contained in the eye image, the eye state information being used to represent an opening degree of the driver's eyes, the target convolutional neural network being obtained by training a convolutional neural network by using pre-collected eye image samples; determining a correction parameter according to the eye state information in each frame of eye image of the driver obtained in a first preset time, when the eye state information of each of the plurality of frames of eye image is obtained; obtaining the head deflection angle of the driver corresponding to each of the plurality of frames of eye image; correcting the eye state information contained in each of the plurality of frames of eye image of the driver according to the correction parameter and the head deflection angle of the driver corresponding to each of the plurality of frames of eye image; and determining whether the driver is in a fatigue state according to the eye state information contained in each of the plurality of frames of eye image of the driver after correction; The determination module is further configured to, when the current frame of eye image of the driver is obtained, determine whether the eye state information contained in the first frame of eye image after correction in a second preset time is less than or equal to a seventh threshold value; and if it is determined that the eye state information contained in the first frame of eye image after correction in the second preset time is less than or equal to the seventh threshold value, decrease a third statistical frame number by 1, wherein the third statistical frame number is determined according to a number of frames of eye image after correction in the second preset time whose eye state information is less than or equal to the seventh threshold value.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the fatigue detection method according to any one of claims 1 to 10.

13. A fatigue detection device, characterized by, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the fatigue detection method according to any one of claims 1 to 10.

Images