A fatigue driving detection method applied to an unmanned driving device and the unmanned driving device

Abstract

The embodiment of the application relates to the technical field of unmanned driving, and relates to a fatigue driving detection method applied to an unmanned driving device and the unmanned driving device. The method comprises the following steps: acquiring a first face image of a driver of the unmanned driving device; after delaying for a preset time length, continuously acquiring a second face image of the driver; processing each face image based on a deep learning algorithm to obtain first pupil size information of the first face image and second pupil size information of the second face image; when the first pupil size information and the second pupil size information are both less than a preset opening threshold value, and the difference between the first pupil size information and the second pupil size information is less than a reference difference value, it is determined that the driver meets fatigue detection conditions, and fatigue stimulation information is generated; target pupil information of the driver is acquired according to the fatigue stimulation information; and when the target pupil information meets unmanned driving triggering conditions, the unmanned driving mode is switched to. The embodiment of the application improves the safety of the unmanned driving device.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a fatigue driving detection method and an autonomous driving device. Background Technology

[0002] Currently, autonomous driving devices include both manned and autonomous driving modes, which are switched manually. However, in manned mode, if the driver becomes fatigued and fails to switch driving modes in time, it could lead to a safety accident. Summary of the Invention

[0003] The present invention aims to provide a fatigue driving detection method and an unmanned driving device that improves safety.

[0004] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0005] In a first aspect, embodiments of the present invention provide a fatigue driving detection method, comprising:

[0006] When the driver of the autonomous vehicle meets the fatigue detection conditions, fatigue stimulus information is generated.

[0007] Based on the fatigue stimulus information, obtain the driver's target pupil information;

[0008] When the target pupil information meets the autonomous driving trigger conditions, switch to autonomous driving mode.

[0009] In some embodiments, generating fatigue stimulus information when the driver of the autonomous vehicle is detected to meet fatigue detection conditions includes:

[0010] Obtain the first facial image of the driver of the autonomous vehicle;

[0011] After a preset delay, continue acquiring the second facial image of the driver of the autonomous vehicle;

[0012] Based on deep learning algorithms, each face image is processed to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0013] When both the first pupil size information and the second pupil size information are less than a preset opening threshold, and the difference between the first pupil size information and the second pupil size information is less than a reference difference, it is determined that the driver meets the fatigue detection conditions, and fatigue stimulus information is generated.

[0014] In some embodiments, the step of processing each face image based on a deep learning algorithm to obtain the first pupil size information of the first face image and the second pupil size information of the second face image includes:

[0015] Image processing is performed on each face image to obtain the target image region of the face image;

[0016] Based on a deep learning algorithm, each target image region is processed to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0017] In some embodiments, the fatigue stimulation information includes blue light stimulation information, and the step of performing image processing on each face image to obtain the target image region of the face image includes:

[0018] Noisy image regions in each face image whose RGB values ​​are not within the blending color threshold range are removed, while target image regions whose RGB values ​​are within the blending color threshold range are retained.

[0019] In some embodiments, the generation of fatigue stimulus information includes:

[0020] The lights installed around the driver's seat of the unmanned vehicle are controlled to generate blue light stimulation information according to the gradual change pattern of light intensity from small to large.

[0021] In some embodiments, the target pupil information includes target pupil size information, and the step of switching to autonomous driving mode when the target pupil information meets the autonomous driving triggering conditions includes:

[0022] Divide the target pupil size information by the second pupil size information to obtain the ratio;

[0023] When the ratio is less than or equal to a preset reference ratio, it is determined that the target pupil information meets the autonomous driving triggering conditions, and the autonomous driving mode is switched.

[0024] In some embodiments, when the ratio is less than or equal to a preset reference ratio, determining that the target pupil information meets the autonomous driving trigger condition and switching to autonomous driving mode includes:

[0025] When the ratio is less than or equal to a preset reference ratio, the duration of normal driving by the driver before switching to the autonomous driving mode and the number of times the driver is fatigued are obtained, wherein the number of times the driver is fatigued is the historical number of times the driver has entered the autonomous driving mode due to fatigue.

[0026] The driver's fatigue score is calculated based on the duration, the number of fatigue episodes, and the ratio.

[0027] When the fatigue score meets the conditions for triggering autonomous driving, switch to autonomous driving mode.

[0028] In some embodiments, calculating the driver's fatigue score based on the duration, the number of fatigue occurrences, and the ratio includes:

[0029] Calculate the first result of multiplying the first weight by the ratio, the second result of multiplying the second weight by the duration, and the third result of multiplying the third weight by the number of fatigue cycles, respectively, wherein the sum of the first weight, the second weight, and the third weight is 100%.

[0030] The driver's fatigue score is obtained by summing the first result, the second result, and the third result.

[0031] In some embodiments, switching to autonomous driving mode when the fatigue score meets the autonomous driving trigger condition includes:

[0032] When the fatigue score is greater than or equal to the preset reference score, switch to autonomous driving mode.

[0033] Secondly, embodiments of the present invention provide an unmanned driving device, comprising:

[0034] At least one processor; and

[0035] A memory that is communicatively connected to the at least one processor;

[0036] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the fatigue driving detection method as described above.

[0037] Thirdly, embodiments of the present invention provide a fatigue driving detection device, comprising:

[0038] The generation module is used to generate fatigue stimulus information when the driver of the autonomous vehicle is detected to meet the fatigue detection conditions;

[0039] The acquisition module is used to acquire the driver's target pupil information based on the fatigue stimulus information;

[0040] The switching module is used to switch to autonomous driving mode when the target pupil information meets the autonomous driving triggering conditions.

[0041] In some embodiments, the generating module includes:

[0042] The first acquisition unit is used to acquire the first facial image of the driver of the autonomous driving device;

[0043] The second acquisition unit is used to continue acquiring the second face image of the driver of the autonomous driving device after a preset delay.

[0044] The processing unit is used to process each face image based on a deep learning algorithm to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0045] The generating unit is configured to determine that the driver meets the fatigue detection conditions and generate fatigue stimulus information when both the first pupil size information and the second pupil size information are less than a preset opening threshold, and the difference between the first pupil size information and the second pupil size information is less than a reference difference.

[0046] In some embodiments, the processing unit includes:

[0047] The first processing subunit is used to perform image processing on each face image to obtain the target image region of the face image;

[0048] The second processing subunit is used to process each target image region based on a deep learning algorithm to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0049] In some embodiments, the fatigue stimulation information includes blue light stimulation information, and the first processing subunit is specifically used to remove noisy image regions in each face image whose RGB values ​​are not within the range of the mixed color threshold, and retain target image regions whose RGB values ​​are within the range of the mixed color threshold.

[0050] In some embodiments, the generating unit includes:

[0051] The generating subunit is used to control the lights installed around the driver's seat of the unmanned vehicle to generate blue light stimulation information according to a gradual change pattern of light intensity from small to large.

[0052] In some embodiments, the target pupil information includes target pupil size information, and the switching module includes:

[0053] The calculation unit is used to divide the target pupil size information by the second pupil size information to obtain a ratio;

[0054] The switching unit is used to determine that the target pupil information meets the autonomous driving triggering conditions and switch to autonomous driving mode when the ratio is less than or equal to a preset reference ratio.

[0055] In some embodiments, the switching unit includes:

[0056] The acquisition subunit is used to acquire the duration of normal driving by the driver before switching to the autonomous driving mode and the number of times the driver is fatigued when the ratio is less than or equal to a preset reference ratio. The number of fatigues is the number of times the driver has entered the autonomous driving mode due to fatigue.

[0057] The calculation subunit is used to calculate the driver's fatigue score based on the duration, the number of fatigue occurrences, and the ratio.

[0058] The switching subunit is used to switch to autonomous driving mode when the fatigue score meets the autonomous driving triggering conditions.

[0059] In some embodiments, the calculation subunit is specifically used to calculate a first result of multiplying the first weight by the ratio, a second result of multiplying the second weight by the duration, and a third result of multiplying the third weight by the number of fatigue cycles, wherein the sum of the first weight, the second weight, and the third weight is 100%.

[0060] The driver's fatigue score is obtained by summing the first result, the second result, and the third result.

[0061] In some embodiments, the switching subunit is specifically used to switch to autonomous driving mode when the fatigue score is greater than or equal to a preset reference score.

[0062] Fourthly, embodiments of the present invention also provide a non-volatile computer-readable storage medium storing computer-executable instructions for enabling a computer to perform the fatigue driving detection method as described above.

[0063] The beneficial effects of the embodiments of the present invention are as follows: Unlike the prior art, the fatigue driving detection method and unmanned driving device provided by the embodiments of the present invention generate fatigue stimulus information when the driver of the unmanned driving device meets the fatigue detection conditions, obtain the driver's target pupil information based on the fatigue stimulus information, and switch to unmanned driving mode when the target pupil information meets the unmanned driving trigger conditions. Therefore, the embodiments of the present invention improve the safety of unmanned driving devices. Attached Figure Description

[0064] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0065] Figure 1This is one application scenario of the fatigue driving detection method provided in the embodiments of the present invention;

[0066] Figure 2 This is a flowchart of a fatigue driving detection method provided in an embodiment of the present invention;

[0067] Figure 3 yes Figure 2 Flowchart of one method for step 10;

[0068] Figure 4 yes Figure 3 Flowchart of one method for step 103;

[0069] Figure 5 yes Figure 2 Flowchart of one method in step 50;

[0070] Figure 6 yes Figure 5 Flowchart of one method for step 502;

[0071] Figure 7 This is a schematic diagram of the structure of a fatigue driving detection device provided in an embodiment of the present invention;

[0072] Figure 8 This is a structural schematic diagram of an unmanned driving device provided in an embodiment of the present invention;

[0073] Figure 9 This is a flowchart of a fatigue driving detection method provided in an embodiment of the present invention. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0076] It should be noted that the fatigue driving detection method provided in this embodiment of the invention is applied to unmanned driving equipment, which includes various types of transportation vehicles, such as unmanned cars, unmanned aircraft, unmanned ships, unmanned rail transit vehicles, unmanned construction tools, etc. This embodiment uses unmanned cars as an example.

[0077] Specifically, Figure 1 This is one application scenario of the fatigue driving detection method provided in the embodiments of the present invention. The application scenario includes an autonomous vehicle 100. In order to improve the customer base and user experience of the autonomous vehicle 100, the autonomous vehicle 100 is equipped with a manned driving mode and an autonomous driving mode. The manned driving mode and the autonomous driving mode can be switched arbitrarily according to the driver's operation. The driver can also trigger the switch from the manned driving mode to the autonomous driving mode according to the fatigue driving detection method provided in the embodiments of the present invention.

[0078] The driverless car 100 is similar to existing gasoline-powered, electric, and hybrid vehicles, including an engine, chassis, body, electrical equipment, and tires. In manned mode, the driver controls the vehicle's control system to convert other forms of energy into the mechanical energy required for the driverless car 100 to move, thereby controlling its start-up, speed, direction, and braking.

[0079] As an autonomous intelligent vehicle, the self-driving car 100 should be equipped with onboard sensors (not shown in the figure), such as distance sensors like lidar to detect the distance between the vehicle and surrounding objects, and speed sensors to detect the speed of vehicles obstructing the view ahead. The self-driving car 100 should also be equipped with image acquisition devices (not shown in the figure), such as cameras, to capture facial images of the driver.

[0080] Furthermore, the driverless car 100 is equipped with an electronic vehicle control system that can acquire the facial image of the driver of the driverless car 100. Based on the facial image, it can be determined that the driver meets the fatigue detection conditions, triggering the driverless car 100 to switch from the human driving mode to the driverless driving mode.

[0081] The face image can be in video format or image format, and there can be multiple sensors and cameras.

[0082] In some embodiments, the camera is also used to collect road surface images within a preset range of the driverless vehicle 100. Based on the road surface images, the vehicle's electronic control system can identify road information, obstacle information, weather information, etc., related to the driverless vehicle 100.

[0083] Specifically, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0084] It's understandable that fatigue driving has many causes. Driver fatigue mainly stems from the fatigue of the nervous and sensory organs, as well as physical fatigue caused by poor blood circulation due to maintaining a fixed posture for extended periods. When drivers sit in a fixed seat for long periods, their movements are somewhat restricted, their attention is highly focused, and they are busy judging external stimuli, leading to a highly tense mental state. This results in symptoms of driver fatigue such as blurred vision, back pain, slow reaction time, and impaired driving agility.

[0085] Since pupillary changes are a physiological indicator not controlled by conscious effort, this invention uses pupillary changes as a criterion for determining whether fatigue detection conditions are met. Autonomous vehicles travel at high speeds, and road traffic conditions are complex and constantly changing. Traffic accidents often occur within a short period, and drivers in a fatigued state are more likely to cause accidents. Therefore, this invention proposes a fatigue driving detection method. When the driver meets the fatigue detection conditions, the autonomous vehicle is switched from manned driving mode to autonomous driving mode, thereby avoiding safety accidents caused by fatigue driving.

[0086] Please see Figure 2 This is a flowchart of a fatigue driving detection method provided in an embodiment of the present invention. This method can be executed by the aforementioned autonomous vehicle. Figure 2 As shown, the fatigue driving detection method includes, but is not limited to:

[0087] Step 10: When the driver of the autonomous vehicle is detected to meet the fatigue detection conditions, fatigue stimulus information is generated.

[0088] Please see Figure 3 and Figure 9 Step 10 includes steps 101-104, as detailed below:

[0089] Step 101: Obtain the first facial image of the driver of the autonomous vehicle.

[0090] Step 102: After a preset delay, continue to acquire the second face image of the driver of the autonomous vehicle.

[0091] When the preset duration is equal to the acquisition cycle of the camera of the autonomous driving device, the first face image and the second face image are two adjacent frames. It can be understood that the embodiments of the present invention do not limit the size of the preset duration. A reasonable preset duration can be selected according to each driver's driving habits.

[0092] Specifically, a camera located in front of the driver is used to capture the driver's first and second facial images. The camera is installed between the windshield and the driver using a bracket or other means, with the camera positioned slightly above the driver's head to ensure complete and unobstructed capture of the first and second facial images.

[0093] Step 103: Based on the deep learning algorithm, process each face image to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0094] Please see Figure 4 Step 103 includes steps 1031-1032, as follows:

[0095] Step 1031: Perform image processing on each face image to obtain the target image region of the face image.

[0096] The fatigue stimulation information includes blue light stimulation information. The step of performing image processing on each face image to obtain the target image region of the face image includes: removing noisy image regions in each face image whose RGB values ​​are not within the range of the mixed color threshold, and retaining the target image regions whose RGB values ​​are within the range of the mixed color threshold.

[0097] RGB refers to the three primary colors of light: R is red, G is green, and B is blue. Physical optics experiments have shown that red, green, and blue light cannot be mixed from other colors, and mixing these three colors in different proportions can produce almost all colors found in nature. The numerical value of a color is generally represented by its RGB value, which indicates the brightness of the face image. RGB values ​​are represented by integers 0, 1, 2... up to 255, with 255 being the highest brightness. Each of R, G, and B has 256 brightness levels. In this embodiment, the RGB values ​​need to be converted into data recognizable by the electronic control system of the autonomous driving device, such as hexadecimal numbers.

[0098] It is understood that the illumination applied to each facial image includes at least one combination of sunlight, vehicle lighting, lighting from other autonomous vehicles, building lighting, billboard lighting, etc., and blue light stimulation information. Image processing is performed on each facial image to obtain the target image region, which can eliminate the influence of noisy image regions and improve the accuracy of the calculation results for the first and second pupil size information.

[0099] Step 1032: Based on a deep learning algorithm, process each of the target image regions to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0100] In this embodiment, a convolutional neural network is used to process each target image region. The convolutional neural network includes convolutional layers, pooling layers, and fully connected layers. The convolutional layers initially extract features, the pooling layers extract the main features, and the fully connected layers summarize the features from all parts to generate a classifier for prediction and recognition.

[0101] The function of the convolutional layer is to extract features from each small part of the target image region. Specifically, by continuously changing the convolutional kernel, it determines which convolutional kernels are useful for initially representing the target image region, and then obtains the output matrix after multiplying with the corresponding convolutional kernels. The input of the pooling layer is the output matrix after multiplying the original data output by the convolutional layer with the corresponding convolutional kernel. The purpose of pooling the target image region is: 1. To reduce the number of training parameters and reduce the dimension of the feature vector output by the convolutional layer; 2. To reduce overfitting, retain only the most useful image information, and reduce the propagation of noise.

[0102] As can be seen, the work of convolutional and pooling layers is to extract features and reduce the parameters introduced by the original image. However, to generate the final output, fully connected layers are applied to generate a classifier equal to the number of classes needed. The working principle of the fully connected layer is to re-slice the tensor output by the pooling layer into vectors, multiply them by a weight matrix, add a bias value, and then apply the ReLU activation function to them. Gradient descent is used to optimize the parameters, which summarizes the features from each part to generate a classifier. This classifier is used to identify the first pupil size information of the first face image and the second pupil size information of the second face image.

[0103] In some embodiments, an eye contour image can be detected based on the face image, and based on the eye contour image, first pupil size information of the first face image and second pupil size information of the second face image can be obtained respectively. Specifically, detecting the eye contour image based on the face image includes: establishing an active shape model; searching for the eye contour image in the face image based on the active shape model, wherein the eye contour image includes pupil center coordinate information, pupil diameter information, etc., that is, the first pupil size information and the second pupil size information include pupil center coordinate information, pupil diameter information, etc.

[0104] Step 104: When both the first pupil size information and the second pupil size information are less than the preset opening threshold, and the difference between the first pupil size information and the second pupil size information is less than the reference difference, it is determined that the driver meets the fatigue detection conditions, and fatigue stimulus information is generated.

[0105] The generation of fatigue stimulation information includes: controlling the lights installed around the driver's seat of the unmanned vehicle to generate blue light stimulation information according to a gradual increase in light intensity. Step 30: Obtain the target pupil information of the driver based on the fatigue stimulation information.

[0106] It is understandable that when the driver meets the fatigue detection conditions, i.e., the driver is determined to be driving while fatigued, the blue light stimulation information has a short duration of influence on the driver, and its impact on the driver is also relatively small within a short period of time. Correspondingly, the fatigue stimulation information acts on the driver, causing a change in the driver's pupil size, thus obtaining the driver's target pupil information. Based on the relative change between the target pupil information and the second pupil size information, it is determined whether the autonomous driving triggering conditions are met.

[0107] Step 50: When the target pupil information meets the autonomous driving triggering conditions, switch to autonomous driving mode.

[0108] Please see Figure 5 Step 50 includes steps 501-502, as detailed below:

[0109] Step 501: Divide the target pupil size information by the second pupil size information to obtain the ratio.

[0110] Step 502: When the ratio is less than or equal to the preset reference ratio, determine that the target pupil information meets the autonomous driving trigger condition, and switch to autonomous driving mode.

[0111] Please see Figure 6 Step 502 includes steps 5021-5023, as follows:

[0112] Step 5021: When the ratio is less than or equal to a preset reference ratio, obtain the duration of normal driving by the driver before switching to the autonomous driving mode and the number of times the driver is fatigued, wherein the number of times the driver is fatigued is the historical number of times the driver has entered the autonomous driving mode due to fatigue.

[0113] Step 5022: Calculate the driver's fatigue score based on the duration, the number of fatigue episodes, and the ratio.

[0114] The step of calculating the driver's fatigue score based on the duration, the number of fatigue occurrences, and the ratio includes: calculating a first result by multiplying a first weight by the ratio, a second result by multiplying a second weight by the duration, and a third result by multiplying a third weight by the number of fatigue occurrences, wherein the sum of the first weight, the second weight, and the third weight is 100%; and summing the first result, the second result, and the third result to obtain the driver's fatigue score.

[0115] Step 5023: When the fatigue score meets the autonomous driving trigger condition, switch to autonomous driving mode.

[0116] Specifically, switching to autonomous driving mode when the fatigue score meets the autonomous driving trigger conditions includes: switching to autonomous driving mode when the fatigue score is greater than or equal to a preset reference score.

[0117] In summary, based on the fatigue stimulus information, the driver's target pupil information is obtained. When the ratio is less than or equal to a preset reference ratio, it is determined that the target pupil information meets the autonomous driving trigger condition and switches to autonomous driving mode. Furthermore, by combining the driver's driving time and the number of times the driver has entered autonomous driving mode due to fatigue, the driver's fatigue score is calculated. Based on the fatigue score, it is determined whether the autonomous driving trigger condition is met, thus improving the reliability of switching to autonomous driving mode due to driver fatigue.

[0118] The present invention provides a fatigue driving detection method, which generates fatigue stimulus information when the driver of an unmanned driving device meets the fatigue detection conditions, obtains the driver's target pupil information based on the fatigue stimulus information, and switches to unmanned driving mode when the target pupil information meets the unmanned driving trigger conditions. Therefore, the present invention improves the safety of unmanned driving devices.

[0119] Please see Figure 7 This is a structural schematic diagram of a fatigue driving detection device provided in an embodiment of the present invention. Figure 7 As shown, the fatigue driving detection device is applied to the above-mentioned driverless vehicle. The fatigue driving detection device 700 includes a generation module 71, an acquisition module 72, and a switching module 73.

[0120] The generation module 71 is used to generate fatigue stimulus information when it detects that the driver of the unmanned vehicle meets the fatigue detection conditions.

[0121] In some embodiments, the generation module 71 includes a first acquisition unit 711, a second acquisition unit 712, a processing unit 713, and a generation unit 714.

[0122] The first acquisition unit 711 is used to acquire the first facial image of the driver of the unmanned vehicle.

[0123] The second acquisition unit 712 is used to continue acquiring the second face image of the driver of the unmanned vehicle after a preset delay.

[0124] The processing unit 713 is used to process each face image based on a deep learning algorithm to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0125] In some embodiments, the processing unit 713 includes a first processing subunit 7131 and a second processing subunit 7132.

[0126] The first processing subunit 7131 is used to perform image processing on each face image to obtain the target image region of the face image.

[0127] In some embodiments, the fatigue stimulation information includes blue light stimulation information, and the first processing subunit is specifically used to remove noisy image regions in each face image whose RGB values ​​are not within the range of the mixed color threshold, and retain target image regions whose RGB values ​​are within the range of the mixed color threshold.

[0128] The second processing subunit 7132 is used to process each of the target image regions based on a deep learning algorithm to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

[0129] The generating unit 714 is used to determine that the driver meets the fatigue detection conditions and generate fatigue stimulation information when both the first pupil size information and the second pupil size information are less than a preset opening threshold, and the difference between the first pupil size information and the second pupil size information is less than a reference difference.

[0130] In some embodiments, the generating unit 714 includes a generating subunit 7141, which controls the lamps installed around the driver's seat of the unmanned vehicle to generate blue light stimulation information in a gradual pattern of light intensity from small to large.

[0131] The acquisition module 72 is used to acquire the driver's target pupil information based on the fatigue stimulation information.

[0132] The switching module 73 is used to switch to autonomous driving mode when the target pupil information meets the autonomous driving triggering conditions.

[0133] In some embodiments, the target pupil information includes target pupil size information, and the switching module 73 includes a calculation unit 731 and a switching unit 732.

[0134] The calculation unit 731 is used to divide the target pupil size information by the second pupil size information to obtain a ratio.

[0135] The switching unit 732 is used to determine that the target pupil information meets the autonomous driving triggering conditions when the ratio is less than or equal to a preset reference ratio, and then switch to the autonomous driving mode.

[0136] In some embodiments, the switching unit 732 includes an acquisition subunit 7321, a calculation subunit 7322, and a switching subunit 7323.

[0137] The acquisition subunit 7321 is used to acquire the duration of normal driving by the driver before switching to the autonomous driving mode and the number of times the driver is fatigued when the ratio is less than or equal to a preset reference ratio. The number of times fatigued refers to the number of times the driver has entered the autonomous driving mode due to fatigue.

[0138] The calculation subunit 7322 is used to calculate the driver's fatigue score based on the duration, the number of fatigue episodes, and the ratio.

[0139] In some embodiments, the calculation subunit 7322 is specifically used to calculate a first result of multiplying the first weight by the ratio, a second result of multiplying the second weight by the duration, and a third result of multiplying the third weight by the number of fatigue occurrences, wherein the sum of the first weight, the second weight, and the third weight is 100%; the first result, the second result, and the third result are accumulated to obtain the driver's fatigue score.

[0140] The switching subunit 7323 is used to switch to the autonomous driving mode when the fatigue score meets the autonomous driving triggering conditions.

[0141] In some embodiments, the switching subunit 7323 is specifically used to switch to autonomous driving mode when the fatigue score is greater than or equal to a preset reference score.

[0142] The present invention provides a fatigue driving detection device, which generates fatigue stimulus information when the driver of the unmanned driving device meets the fatigue detection conditions by a generation module, acquires the target pupil information of the driver based on the fatigue stimulus information by an acquisition module, and switches to unmanned driving mode when the target pupil information meets the unmanned driving trigger conditions by a switching module. Therefore, the present invention improves the safety of unmanned driving devices.

[0143] Please see Figure 8 This is a structural schematic diagram of an unmanned driving device provided in an embodiment of the present invention. Figure 8As shown, the unmanned driving device 100 includes at least one processor 101 and a memory 102 communicatively connected to the at least one processor 101. Figure 8 The example described uses a processor 101. The memory 102 stores instructions that can be executed by the at least one processor 101, which, when executed, enable the at least one processor 101 to perform the fatigue driving detection method as described in the above method embodiment.

[0144] The processor 101 and the memory 102 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0145] The memory 102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the fatigue driving detection method in this embodiment of the invention, for example, as shown in the attached... Figure 7 The various modules shown. The processor 101 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 102, thereby implementing the fatigue driving detection method described in the above method embodiments.

[0146] The memory 102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the fatigue driving detection device, etc. Furthermore, the memory 102 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 102 may optionally include memory remotely located relative to the processor 101, and these remote memories can be connected via a network to devices controlling the autonomous vehicle's operation. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0147] The one or more modules are stored in the memory 102. When executed by the one or more processors 101, they perform the fatigue driving detection method in any of the above method embodiments, for example, the method described above. Figures 2 to 6 The method and steps to achieve Figure 7 The functions of each module and unit within it.

[0148] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0149] This invention also provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to perform the operations described above. Figures 2 to 6 The method and steps to achieve Figure 7 The functions of each module.

[0150] This invention also provides a computer program product, including a computing program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the fatigue driving detection method described in any of the above method embodiments, for example, to perform the above-described method. Figures 2 to 6 The method and steps to achieve Figure 7 The functions of each module.

[0151] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fatigue driving detection method for unmanned driving equipment, characterized in that, include: Obtain the first facial image of the driver of the autonomous vehicle; After a preset delay, continue acquiring the second facial image of the driver of the autonomous vehicle; Based on deep learning algorithms, each face image is processed to obtain the first pupil size information of the first face image and the second pupil size information of the second face image. When both the first pupil size information and the second pupil size information are less than a preset opening threshold, and the difference between the first pupil size information and the second pupil size information is less than a reference difference, it is determined that the driver meets the fatigue detection conditions, and fatigue stimulus information is generated. Based on the fatigue stimulus information, obtain the target pupil size information of the driver; Divide the target pupil size information by the second pupil size information to obtain the ratio; When the ratio is less than or equal to a preset reference ratio, the duration of normal driving by the driver before switching to the autonomous driving mode and the number of times the driver is fatigued are obtained, wherein the number of times the driver is fatigued is the historical number of times the driver has entered the autonomous driving mode due to fatigue. The driver's fatigue score is calculated based on the duration, the number of fatigue episodes, and the ratio. When the fatigue score is greater than or equal to the preset reference score, switch to autonomous driving mode.

2. The method according to claim 1, characterized in that, Both the first pupil size information and the second pupil size information include pupil diameter information.

3. The method according to claim 1, characterized in that, The deep learning algorithm processes each face image to obtain the first pupil size information of the first face image and the second pupil size information of the second face image, including: Image processing is performed on each face image to obtain the target image region of the face image; Based on a deep learning algorithm, each target image region is processed to obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

4. The method according to claim 3, characterized in that, The deep learning algorithm is used to process each target image region to obtain the first pupil size information of the first face image and the second pupil size information of the second face image, including: Detect the human eye contour image, and based on the human eye contour image, obtain the first pupil size information of the first face image and the second pupil size information of the second face image.

5. The method according to claim 4, characterized in that, The detection of the human eye contour image includes: An active shape model is established; based on the active shape model, the human eye contour image is searched in the face image, wherein the human eye contour image includes the center coordinate information of the pupil and the diameter information of the pupil.

6. The method according to claim 3, characterized in that, The step of performing image processing on each face image to obtain the target image region of the face image includes: Noisy image regions in each face image whose RGB values ​​are not within the blending color threshold range are removed, while target image regions whose RGB values ​​are within the blending color threshold range are retained.

7. The method according to claim 1, characterized in that, The fatigue-generating stimulus information includes: The lights installed around the driver's seat of the unmanned vehicle are controlled to generate blue light stimulation information according to the gradual change pattern of light intensity from small to large.

8. The method according to claim 1, characterized in that, The calculation of the driver's fatigue score based on the duration, the number of fatigue occurrences, and the ratio includes: Calculate the first result of multiplying the first weight by the ratio, the second result of multiplying the second weight by the duration, and the third result of multiplying the third weight by the number of fatigue cycles, wherein the sum of the first weight, the second weight, and the third weight is 100%. The driver's fatigue score is obtained by summing the first result, the second result, and the third result.

9. An unmanned driving device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the fatigue driving detection method as described in any one of claims 1 to 8.

Images