Fatigue driving detection method based on facial information

Abstract

According to the facial information-based fatigue driving detection method provided by the invention, features of different scales are effectively fused by adding an adaptive spatial feature fusion ASFF, so that an algorithm can more comprehensively understand an image, the scale invariance is enhanced, and the model precision and robustness are improved; by adding an attention mechanism module BiFormer, a network only pays attention to a model of a specific area of an image instead of the whole image to reduce the calculation complexity of image processing, so that the feature extraction capability and the detection precision of an algorithm under a complex background are improved. Test results show that compared with an original YOLOv5s model, the improved model has the advantages that the accuracy is improved by 1.7%, the recall rate is improved by 1.8%, the network precision is improved by 3.6%, the size and parameter quantity are slightly increased, the eye closing detection precision is improved by 3.6%, the yawn detection precision is improved by 3.8%, and the convergence speed of the model is increased. According to the scheme, the deep learning model with the small size is used, eye opening and closing and yawn detection is achieved, and then the fatigue state of a driver is judged.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue driving detection, and in particular to a fatigue driving detection method based on facial information. Background Art

[0002] At present, vehicle driving safety has become the top priority of urban road traffic safety. Studies have shown that traffic accidents caused by driver fatigue account for a considerable proportion of urban traffic accidents. Therefore, timely detection of driver fatigue status, including yawning, distraction, closing eyes, etc., is crucial to timely avoid accidents, reduce the risk of traffic accidents, and protect the lives of drivers and passengers.

[0003] At present, the existing driver fatigue detection methods at home and abroad can be divided into three categories: based on vehicle driving mode, biological signals and computer vision technology. With the rapid development of information technology and network technology, deep learning algorithms have been widely used in the field of target detection. The algorithm is usually divided into two categories: two-stage algorithm and single-stage detection algorithm. Among them, the two-stage algorithm first proposes candidate areas when performing target detection, and then classifies and regresses these candidate areas. The classic representative is the R-CNN series algorithm; the single-stage detection does not need to generate candidate areas, and directly completes target detection in one stage, mainly including the YOLO series algorithm, SSD algorithm and RetinaNet. The single-stage algorithm directly detects the target label and position, and is faster than the two-stage algorithm in training and detection. Since the two-stage algorithm first extracts the candidate box and then classifies it, the algorithm has more steps, so its speed is slower than the single-stage algorithm, but the accuracy is higher. Although the two-stage algorithm has a high accuracy, its deployability and real-time performance are not strong, and it is rarely used in driver behavior detection tasks. Relatively speaking, the single-stage algorithm is widely used due to its faster speed.

[0004] At present, many scholars at home and abroad have made a lot of algorithm research on improving the accuracy and speed of driver fatigue driving detection. Cao et al. have also conducted relevant research on using low-cost photoplethysmography PPG sensors and motion sensors integrated in wrist devices to build sleepiness detection systems. Tao et al. extracted facial micro-features by improving the main direction average optical flow MDMO algorithm, and applied the physiological feature detection algorithm based on remote photoplethysmography to improve the convenience and accuracy of heart rate variability signal extraction. Finally, the two time series features of physiological features and micro-expression features were fused to predict the driver's fatigue state. Haris et al. found through driving simulator experiments that lane departure is most sensitive to fatigue and can be used as an objective evaluation indicator for fatigue detection. Melvin et al. proposed a novel method based on facial slight movement recognition to identify subtle facial activities. The network uses bidirectional and 3D convolutional networks, key frame selection technology is used to find the most content from delicate facial gestures, photo histograms are used to quickly eliminate redundant frames, and median absolute deviation is used to locate outliers. Gomaa et al. used a combination of convolutional neural network CNN and long short-term memory LSTM to predict driver drowsiness. Akrout et al. proposed a fusion system based on yawn detection, sleepiness detection and 3D head pose estimation. Yi et al. used the Dlib toolkit to mark and histogram equalize the grayscale image frames, extracted facial features based on facial landmarks, and then evaluated them to obtain the eye aspect ratio EAR, mouth aspect ratio MAR, and real-time head Euler angle HEA. Vijaypriya et al. used cross-guided bilateral filtering to preprocess the image frames, and then used hybrid dual-tree complex wavelet transform and Walsh-Hadamard transform for feature extraction to extract feature vectors in image frame blocks, and then used the Flamingo search algorithm FSA integrated with the deep learning model multi-scale convolutional neural network MCNN to optimize the feature points to predict fatigue.

[0005] In summary, the fatigue driving detection method based on facial information has important application prospects in the field of fatigue detection. It can comprehensively analyze the driver's facial features and provide an effective means to improve traffic safety. However, although some studies using methods based on machine learning or convolutional neural networks have successfully achieved satisfactory results, the current fatigue detection system still has some shortcomings. The detection system has limitations in equipment deployment, environmental changes, and real-time monitoring. For example, facial feature extraction is affected by lighting conditions and partial occlusion, low-quality input data affects system performance, fatigue feature information extraction is not comprehensive, and the amount of calculation is large. Therefore, solving these problems is an important research direction for the future development of fatigue driving detection systems.

[0006] At present, fatigue detection mainly has the following problems: 1) Unlike ordinary target detection, the object of fatigue detection is the driver during vehicle driving. Since fatigue state usually occurs at the moment of driving, the fatigue detection system needs to quickly and accurately identify the driver's state to ensure driving safety, but the fatigue detection system often has a certain response delay after detecting the fatigue state, and cannot guarantee the real-time detection; 2) In the actual driving environment, due to the complexity and variability of weather and light, it is often impossible to accurately identify the driver's facial fatigue features, resulting in low detection accuracy and inability to guarantee the robustness of detection.

[0007] Traditional fatigue detection methods use driver physiological signals or vehicle status to detect the driver's fatigue state. They rely on professional detection equipment, have low detection accuracy and slow speed, cannot detect driver status changes in real time, lack practicality and scalability, and must find an efficient and accurate detection method. Summary of the invention

[0008] The purpose of the present invention is to provide a fatigue driving detection method based on facial information, which can effectively fuse features of different scales so that the algorithm can understand the image more comprehensively, enhance scale invariance, and improve model accuracy and robustness; and make the network focus only on specific areas of the image rather than the model of the entire image to reduce the computational complexity of image processing, thereby improving the feature extraction capability and detection accuracy of the algorithm under complex backgrounds.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is: a fatigue driving detection method based on facial information, comprising the following steps: Step 1: The input of the ASBI-YOLOv5s model preprocesses the input driver's facial image; Step 2: The backbone network of the ASBI-YOLOv5s model extracts features from the preprocessed driver's facial image; Step 3: The neck structure Neck of the ASBI-YOLOv5s model fuses the features extracted in step 2 to obtain feature maps of different sizes. The BiFormer attention mechanism is embedded in the neck structure Neck, and the adaptive spatial feature fusion ASFF is added to the neck structure Neck; Step 4: The head of the ASBI-YOLOv5s model classifies and locates the feature map obtained in step 3, generates bounding boxes and category predictions, and obtains output results.

[0010] Preferably, the preprocessing of the input driver's facial image in step 1 includes Mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling.

[0011] Preferably, the Mosaic data enhancement includes stitching four randomly cropped, scaled and arranged images into a new image to expand the data set; the adaptive anchor box calculation includes embedding the calculation of the anchor box size into the model training process; the adaptive image scaling includes scaling the original image to a standard size while filling the minimum number of pixels.

[0012] Preferably, the backbone network backbone of the ASBI-YOLOv5s model adds a Focus module and two different CSPNet structures.

[0013] Preferably, the neck structure Neck of the ASBI-YOLOv5s model includes a spatial pyramid pooling SPP module and an FPN+PAN structure.

[0014] Preferably, the BiFormer attention mechanism is embedded in the first convolutional layer of the neck structure Neck, after the SPPF structure.

[0015] Preferably, the BiFormer attention mechanism is based on BiformerBlock, follows the visiontransformer architecture design, and adopts a four-level pyramid structure. BiFormer uses overlapping block embedding in the first stage, and uses block merging modules in the second to fourth stages to reduce the input spatial resolution while increasing the number of channels, and uses continuous Bi-Former blocks for feature transformation.

[0016] Preferably, the adaptive spatial feature fusion ASFF performs feature filtering based on learning parameters.

[0017] The beneficial effects of the present invention are:

[0018] This scheme effectively integrates features of different scales by adding adaptive spatial feature fusion ASFF, so that the algorithm can understand the image more comprehensively, enhance scale invariance, and improve model accuracy and robustness; by adding the attention mechanism module BiFormer, the network only focuses on the model of specific areas of the image instead of the entire image to reduce the computational complexity of image processing, thereby improving the algorithm's feature extraction ability and detection accuracy under complex backgrounds. The experimental results show that compared with the original YOLOv5s model, the improved model has an accuracy rate of 1.7%, a recall rate of 1.8%, and a network accuracy of 3.6%. The volume and parameter volume have increased slightly, the eye closure detection accuracy has increased by 3.6%, the yawn detection accuracy has increased by 3.8%, and the model convergence speed has accelerated. This scheme uses a small deep learning model to detect eye opening and closing and yawning, and then judge the driver's fatigue state, which has certain engineering application value. The next step will be to further optimize the network structure, study the problems and solutions encountered after the model is actually deployed on the edge, and improve and improve the model in actual fatigue driving detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is the network structure diagram of YOLOv5s.

[0021] Figure 2 It is a network structure diagram of the ASBI-YOLOv5s model of the present invention.

[0022] Figure 3 Schematic diagram of the structure of the BiFormer attention mechanism of the present invention.

[0023] Figure 4 It is a schematic diagram of the structure of the feature pyramid network FPN of the present invention.

[0024] Figure 5 It is a structural diagram of the adaptive spatial feature fusion ASFF of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0026] The present invention discloses a method for detecting fatigue driving based on facial information. The embodiment includes the following steps: Step 1: The input of the ASBI-YOLOv5s model preprocesses the input driver's facial image; Step 2: The backbone network of the ASBI-YOLOv5s model extracts features from the preprocessed driver's facial image; Step 3: The neck structure Neck of the ASBI-YOLOv5s model fuses the features extracted in step 2 to obtain feature maps of different sizes. The BiFormer attention mechanism is embedded in the neck structure Neck, and the adaptive spatial feature fusion ASFF is added to the neck structure Neck; Step 4: The head of the ASBI-YOLOv5s model classifies and locates the feature map obtained in step 3, generates bounding boxes and category predictions, and obtains output results.

[0027] In industry, the most commonly used single-stage target detection model is the YOLOv5 series model. There are currently four versions, and the model parameters are YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x, from small to large. Among them, the YOLOv5s model has a small number of parameters and a fast detection speed, and is often used for edge device deployment. In fatigue driving detection, according to the application scenario, real-time detection of the driver's face is required, and high requirements are placed on the detection speed of the model. Taking into account the cost and energy consumption, as well as the weak computing power of the detection equipment, this embodiment uses the YOLOv5s target detection network as the basic network for improvement, and applies the improved model to fatigue driving detection. Figure 1 As shown in the figure, the network structure of YOLOv5s mainly includes four parts: Input, Backbone, Neck and Head.

[0028] Although YOLOv5s has demonstrated excellent performance in various target detection tasks, in complex and changeable driving environments such as fatigue detection, the original YOLOv5s model has problems such as low detection accuracy, slow detection speed, and poor robustness. To address these problems, this embodiment proposes to improve the network structure of the original YOLOv5s model based on the ASBI-YOLOv5s model. By adding adaptive spatial feature fusion ASFF, the scale invariance is enhanced, the model accuracy and robustness are improved, and the network convergence speed is accelerated; by adding the attention mechanism module BiFormer, the feature extraction capability and detection accuracy of the algorithm are improved. Its network structure is as follows Figure 2 shown.

[0029] The preprocessing of the input driver's facial image in step 1 includes Mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling.

[0030] The Mosaic data enhancement includes stitching four randomly cropped, scaled and arranged images into a new image to expand the data set. It can increase the diversity of the data set, thereby improving the model's detection accuracy of the target. It can also reduce the impact of overfitting and improve the generalization ability of the model.

[0031] The adaptive anchor box calculation includes embedding the calculation of the anchor box size into the model training process. In traditional object detection models, the size of the anchor box needs to be manually set before training. However, this method may cause some anchor boxes of inappropriate size to appear in the model, thereby reducing the accuracy of the model. To solve this problem, the ASBI-YOLOv5s model embeds the calculation of the anchor box size into the model training process, so that the model can automatically calculate the optimal anchor box. It can greatly reduce the burden of manually setting the anchor box size and improve training efficiency and model performance.

[0032] The adaptive image scaling includes scaling the original image to a standard size and filling the minimum number of pixels at the same time. This reduces redundant information, reduces computing and storage costs, and improves network reasoning speed and efficiency. In addition, it can also reduce the burden of data preprocessing, thereby speeding up training.

[0033] The backbone network backbone of the ASBI-YOLOv5s model adds a Focus module and two different CSPNet structures.

[0034] The Focus module is added to perform slicing operations, which, combined with the convolution kernel convolution operation, can better preserve the image detail features. The Focus module can improve the receptive field and feature extraction capabilities of the model without increasing the amount of calculation. This module divides the input tensor into multiple sub-tensors, performs different convolution operations on each sub-tensor, and finally splices the results of all sub-tensors to obtain the final output. Compared with traditional convolution operations, the Focus module can better capture features of different scales, thereby improving the detection accuracy and robustness of the model. In the Focus module, the most important thing is the Slice operation, which can reduce the image size and expand the image dimension.

[0035] Two different CSPNet structures are added, namely CSP1_X structure and CSP2_X structure. CSP1_X structure is applied to the backbone network of the model, and CSP2_X structure is applied to the Neck of the model. It can enhance the feature extraction effect of the Backbone and Neck parts.

[0036] The neck structure Neck of the ASBI-YOLOv5s model includes a spatial pyramid pooling SPP module and an FPN+PAN structure.

[0037] The spatial pyramid pooling SPP module is different from the traditional global pooling layer. It uses pooling windows of multiple sizes to stitch all pooling results of different sizes together to form an output vector of fixed length. Since the size of the input image and the size of the real object are very different in the target detection task, the traditional pooling method will lead to information loss. To avoid this problem, SPP can adaptively process inputs of different sizes at the same scale to obtain the same output size, thereby fusing the information of feature maps of different scales.

[0038] In the network structure design, the ASBI-YOLOv5s model also draws on the structure of FPN and PAN. It splices the PAN structure after the FPN backbone network, combines downsampling and upsampling to generate a feature pyramid, and passes the strong positioning feature information of the low layer to the upper layer, thereby improving the feature fusion capability.

[0039] The attention mechanism attempts to reduce the computational complexity of image processing by introducing a model that only focuses on specific areas of the image rather than the entire image. Adding a suitable attention mechanism can significantly improve the performance of the model. The network input image resolution of the ASBI-YOLOv5s model is low, and small targets provide few valid pixels. Therefore, the ASBI-YOLOv5s model has a more difficult time detecting small targets in complex backgrounds. To improve this situation, the BiFormer attention mechanism is embedded in the original network.

[0040] Embedding BiFormer in the backbone of the ASBI-YOLOv5s model generates too much computational load on the network, so we chose to embed the BiFormer attention mechanism in the first convolutional layer of the neck structure, after the SPPF structure. This embedding method greatly reduces the amount of computation generated by BiFormer without affecting performance.

[0041] like Figure 3 As shown in the figure, the BiFormer attention mechanism is based on BiformerBlock, follows the visiontransformer architecture design, and adopts a four-level pyramid structure, that is, downsampling by 32 times. BiFormer uses overlapping block embedding in the first stage, and uses block merging modules in the second to fourth stages to reduce the input spatial resolution while increasing the number of channels, and uses continuous Bi-Former blocks for feature transformation.

[0042] Among the existing object recognition algorithms, the most common and classic architecture is the Feature Pyramid Network (FPN), which is characterized by utilizing the different characteristics of high-level and low-level features, the semantic information of the former and the refined fusion of the latter. The essence of adaptive spatial feature fusion (ASFF) is to integrate high-level and low-level feature characteristics through a certain algorithm mechanism, thereby obtaining functions at different levels.

[0043] PAN is an enhanced version of the feature pyramid FPN. FPN is a top-down feature pyramid that passes down strong semantic features at high levels to enhance the entire pyramid. However, FPN only enhances semantic information but does not pass on positioning information, or because the upward transmission path is too long, the transmission effect is not good. PAN is aimed at this point. It adds a bottom-up pyramid behind FPN to supplement FPN and pass on the positioning features of the lower layers. The pyramid formed in this way combines both semantic information and positioning information.

[0044] One of the main drawbacks of feature pyramid PAN is the inconsistency between features of different scales, especially for first-order detectors. Specifically, a heuristic-guided feature selection is used when detecting objects: large objects are usually associated with higher feature maps, while small objects are usually associated with lower feature maps. Therefore, once an image contains both large and small objects, there is a conflict between features at different levels. Positive and negative samples have their own definition methods, such as iou, distance, etc. A certain object may be classified as a positive sample or a negative sample on feature maps of different levels. They all want to occupy a dominant position in the feature pyramid, and ASFF enables the network to directly learn how to spatially filter features at other levels, so that only useful information is retained for combination. For features at a certain level, features at other levels are first adjusted to the same resolution and simply integrated, and then trained to find the best fusion method. At each spatial location, features at different levels are adaptively fused together.

[0045] The adaptive spatial feature fusion ASFF performs feature filtering based on learning parameters. The characteristics and advantages of the adaptive spatial feature fusion ASFF are used to improve the quality of target recognition. The detailed structure of the feature pyramid network FPN is as follows Figure 4 As shown in the figure, the adaptive spatial feature fusion ASFF structure is as follows Figure 5 shown.

[0046] The core feature of ASFF is that it needs to filter features based on learning parameters. Therefore, when applying this structure, if the scope of the information object is limited to the attention information at the current level, the information will be layered based on the algorithm logic of this structure, which will help optimize the learning efficiency of the model. For example, ASFF-1 contains feature layers of different scales, which are multiplied by their corresponding weight parameters α, β, χ, and then the resulting information is added to obtain the following formula:

[0047] In the formula, the new feature map obtained by ASFF-1 are the weights of different layers and different feature layers The output of . Since addition is used, it is necessary to ensure that the ASFF layer obtains the same output feature dimension and number of channels from different levels. The size of the convolution kernel uses a 1×1 convolution layer with the same number of channels. The sum of the weight parameters α, β, χ is 1, and the value of [0.1] is locked by the normalization function.

[0048] The Head part of the ASBI-YOLOv5s model is the last part of the network, which is mainly responsible for classifying and locating feature maps and generating bounding boxes and category predictions. The Head part contains three detectors of different scales, each of which detects a feature map of a specific scale, and the final output is the fusion of the output results of each detector. For the network structure of the ASBI-YOLOv5s model used in this embodiment, its outputs are feature maps of sizes 80×80, 40×40, and 20×20 respectively.

[0049] The loss function of the ASBI-YOLOv5s model mainly includes three parts: classification loss, bounding box loss and confidence loss. The weighted sum of the three types of losses constitutes the overall loss function of the ASBI-YOLOv5s model. Among them, the classification loss refers to the error between the network's classification prediction of the target object and the true category; the confidence loss refers to the error between the network's confidence prediction result of whether the target object exists and the true result; both losses are calculated using the binary cross entropy function. The bounding box loss refers to the error between the network's prediction value of the target object's bounding box and the true bounding box. In the ASBI-YOLOv5s model, the CIOU_Loss loss function is used.

[0050] In the fatigue driving detection task of this embodiment, the driver's facial fatigue features are selected for detection, and the target detection classification is closed_eye closed eyes, close_mouth closed mouth, open_eye open eyes, open_mouth open mouth, Normal normal state, Nod head down. This experimental data set uses the domestic open source data set fdd-dataset.

[0051] This dataset mainly uses the on-board camera to capture different driving states of the driver, which is very consistent with the fatigue scene in actual driving. The dataset has a total of 2914 pictures, including 545 images of fatigue driving including closed eyes and yawning, with a resolution of 640×480. In order to train an excellent detection model, the pictures obtained by sample expansion are randomly divided into training set, validation set and test set in a ratio of 3:1:1. The divided dataset is shown in Table 1, and the experimental environment is shown in Table 2. The experiment is based on the deep learning open source framework TensorFlow and is programmed in Python. Table 1 Number of images in training, validation and test sets Table 2 Test environment

[0052] This experiment uses precision P (Precision), recall R (Recall), and mean average precision mAP (mean average precision) of multiple categories as evaluation indicators of model performance.

[0053] Precision: The ratio of the number of samples predicted to be positive and actually positive to the number of samples predicted to be positive. Recall: The ratio of the number of samples predicted to be positive and actually positive to the number of samples predicted to be positive. The recall rate is inversely proportional to the missed detection rate. The formula is as follows:

[0054] Where: TP is the number of samples predicted to be positive but actually positive; FP is the number of samples predicted to be negative but actually positive; FN is the number of samples predicted to be negative but actually positive; Precision is the proportion of correctly predicted positive samples; Recall is the proportion of correctly judged positive samples to the total positive samples.

[0055] PR curve: A curve drawn with Recall as the horizontal axis and Precision as the vertical axis. The PR curve reflects the relationship between precision and recall. The average precision AP (average precision) is the area under the PR curve. The larger the AP, the higher the accuracy of the model:

[0056] mAP (average precision of all categories): represents the average accuracy of all detection categories. It is generally used as a measure of the accuracy of the target detection algorithm. The accuracy of the target detection algorithm is proportional to the mAP value:

[0057] In order to verify the performance gain brought by the BiFormer and ASFF improvement strategies to the model. Three groups of experiments were designed, with mAP, accuracy, recall, number of parameters, and model size as evaluation criteria. Experiment 1: No improvement was made to the original YOLOv5s model; Experiment 2: Add the attention mechanism module BiFormer to the Neck of YOLOv5s, and this model is called BI-YOLOv5s; Experiment 3: Add adaptive spatial feature fusion ASFF to the head of YOLOv5s on the basis of Experiment 2, and this model is called ASBI-YOLOv5s. Table 3 compares the detection results of YOLOv5s, BI-YOLOv5s and ASBI-YOLOv5s before and after adding adaptive spatial feature fusion ASFF. Table 3 Test results of three models

[0058] As shown in Table 3, compared with the original YOLOv5s, the ASBI-YOLOv5s with the addition of adaptive spatial feature fusion ASFF has improved detection accuracy and recall, among which the accuracy has increased by 1.7%, the recall has increased by 1.8%, the network accuracy has increased by 3.6%, and the model volume and parameter volume have increased slightly. Compared with BI-YOLOv5s, the ASBI-YOLOv5s with the addition of adaptive spatial feature fusion ASFF has improved accuracy, recall and mAP, among which the accuracy has increased by 1.1%, the recall has increased by 0.6%, the network accuracy has increased by 1.1%, and the model volume and parameter volume have increased slightly. The comparison results show that the addition of the attention mechanism module BiFormer and the adaptive spatial feature fusion ASFF enables the network to effectively extract image features and improve the accuracy of target detection, which is more suitable for the fatigue driving detection task in this paper.

[0059] In order to further evaluate the performance of the deep learning model, this experiment subdivided the selected test set into two sub-test sets according to eye closure and yawning, and used the cross-validation method to detect the effectiveness of the method in this paper. To avoid contingency, two tests were conducted on each of the two sub-test sets, and the accuracy of the two tests was calculated respectively. The results are shown in Table 4. It can be seen from Table 4 that compared with the original YOLOv5s, the ASBI-YOLOv5s using the attention mechanism module BiFormer and the adaptive spatial feature fusion ASFF has a small improvement in the accuracy of both detections, among which the accuracy of eye closure detection has increased by 3.6%, and the accuracy of yawn detection has increased by 3.8%. Table 4 Cross-validation detection accuracy

[0060] It should be noted that the parts not described in detail in the above embodiments are all prior art.

[0061] The above description is only a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting fatigue driving based on facial information, characterized in that: The following steps are involved: Step 1: The input of the ASBI-YOLOv5s model preprocesses the input driver's facial image; Step 2: The backbone network of the ASBI-YOLOv5s model extracts features from the preprocessed driver's facial image; Step 3: The neck structure Neck of the ASBI-YOLOv5s model fuses the features extracted in step 2 to obtain feature maps of different sizes. The BiFormer attention mechanism is embedded in the neck structure Neck, and the adaptive spatial feature fusion ASFF is added to the neck structure Neck; Step 4: The head of the ASBI-YOLOv5s model classifies and locates the feature map obtained in step 3, generates bounding boxes and category predictions, and obtains output results.

2. The method for detecting fatigue driving based on facial information according to claim 1, characterized in that: The preprocessing of the input driver's facial image in step 1 includes Mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling.

3. The method for detecting fatigue driving based on facial information according to claim 2, characterized in that: The Mosaic data enhancement includes stitching four randomly cropped, scaled and arranged images into a new image to expand the data set; the adaptive anchor box calculation includes embedding the calculation of the anchor box size into the model training process; the adaptive image scaling includes scaling the original image to a standard size and filling the minimum number of pixels at the same time.

4. The method for detecting fatigue driving based on facial information according to claim 1, characterized in that: The backbone network backbone of the ASBI-YOLOv5s model adds a Focus module and two different CSPNet structures.

5. The method for detecting fatigue driving based on facial information according to claim 1, characterized in that: The neck structure Neck of the ASBI-YOLOv5s model includes a spatial pyramid pooling SPP module and an FPN+PAN structure.

6. The method for detecting fatigue driving based on facial information according to claim 1, characterized in that: The BiFormer attention mechanism is embedded in the first convolutional layer of the neck structure Neck, after the SPPF structure.

7. The method for detecting fatigue driving based on facial information according to claim 1, characterized in that: The BiFormer attention mechanism is based on BiformerBlock, follows the visiontransformer architecture design, and adopts a four-level pyramid structure. BiFormer uses overlapping block embedding in the first stage, and uses block merging modules in the second to fourth stages to reduce the input spatial resolution while increasing the number of channels, and uses continuous Bi-Former blocks for feature transformation.

8. The method for detecting fatigue driving based on facial information according to claim 1, characterized in that: The adaptive spatial feature fusion ASFF performs feature filtering based on learning parameters.

Images