Fatigue driving detection method and device, electronic equipment and storage medium

Abstract

The application relates to a fatigue driving detection method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a driver face image to be detected; inputting the driver face image to be detected into a feature extraction network for feature extraction, obtaining a first feature map of different scales output by a multi-head difference kernel convolution module and a second feature map output by a multi-scale large convolution fusion module; determining a maximum scale feature map, a minimum scale feature map and an intermediate scale feature map in the first feature map of different scales; inputting the second feature map and the intermediate scale feature map into a feature fusion network for feature fusion, obtaining a third feature map of different scales output by an adaptive convolution attention module; inputting the third feature map of different scales into a target detection network for target detection, and obtaining a fatigue driving detection result, so that the fatigue driving detection accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a fatigue driving detection method, device, electronic device, and storage medium. Background Technology

[0002] With the increasing number of cars, while automobiles bring speed and convenience to people's lives, traffic accidents also occur frequently. Among them, one of the main causes of serious traffic accidents is driver fatigue.

[0003] Currently, there are methods to detect driver fatigue based on physiological parameters. However, relying on physiological parameters requires a large number of detection instruments, which can be distracting for drivers and are also costly and complex to install. There are also deep learning-based models for detecting and classifying driver fatigue. However, these models have low detection accuracy. Summary of the Invention

[0004] Based on this, the purpose of this application is to provide a fatigue driving detection method, device, electronic device, and storage medium, which can improve the accuracy of fatigue driving detection.

[0005] According to a first aspect of the embodiments of this application, a fatigue driving detection method is provided, comprising the following steps: Obtain the facial image of the driver to be detected; The driver's face image to be detected is input into a feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by a multi-scale large convolutional fusion module. The feature extraction network includes multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolutional modules, and a multi-scale large convolutional fusion module connected in sequence. The step of inputting the driver's face image to be detected into the feature extraction network for feature extraction to obtain first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by the multi-scale large convolutional fusion module includes: inputting the driver's face image to be detected into a first set of first downsampling layers and a first set of multi-head differential kernel convolutional modules to obtain a first feature map at a first scale; wherein the first set of first downsampling layers and the first set of multi-head differential kernel convolutional modules are connected in sequence; and inputting the first... A first feature map at a second scale is input to the first downsampling layer of the second group and the multi-head differential kernel convolution module of the second group to obtain a first feature map at a second scale; wherein, the first downsampling layer of the second group and the multi-head differential kernel convolution module of the second group are connected in sequence; the first feature map at the second scale is input to the first downsampling layer of the third group and the multi-head differential kernel convolution module of the third group to obtain a first feature map at a third scale; wherein, the first downsampling layer of the third group and the multi-head differential kernel convolution module of the third group are connected in sequence; the first feature map at the third scale is input to the first downsampling layer of the fourth group and the multi-head differential kernel convolution module of the fourth group to obtain a first feature map at a fourth scale; wherein, the first downsampling layer of the fourth group and the multi-head differential kernel convolution module of the fourth group are connected in sequence; the first feature map at the fourth scale is input to the multi-scale large convolution fusion module to obtain a second feature map; Determine the maximum-scale feature map, minimum-scale feature map, and intermediate-scale feature map in the first feature map at different scales; wherein, the intermediate-scale feature map is the feature map in the first feature map at different scales excluding the maximum-scale feature map and the minimum-scale feature map; The second feature map and the intermediate-scale feature map are input into the feature fusion network for feature fusion to obtain third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of adaptive convolutional attention modules; Third feature maps of different scales are input into the target detection network for target detection to obtain fatigue driving detection results.

[0006] According to a second aspect of the embodiments of this application, a fatigue driving detection device is provided, comprising: The face image acquisition module is used to acquire the face image of the driver to be detected; The feature map acquisition module is used to input the driver's face image to be detected into a feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by a multi-scale large convolutional fusion module. The feature extraction network includes multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolutional modules, and a multi-scale large convolutional fusion module connected in sequence. The step of inputting the driver's face image to be detected into the feature extraction network for feature extraction to obtain first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by the multi-scale large convolutional fusion module includes: inputting the driver's face image to be detected into a first set of first downsampling layers and a first set of multi-head differential kernel convolutional modules to obtain a first feature map of a first scale; wherein the first set of first downsampling layers and the first set of multi-head differential kernel convolutional modules are connected in sequence. The first feature map at the first scale is input to the second group of the first downsampling layer and the second group of the multi-head differential kernel convolutional modules to obtain the first feature map at the second scale; wherein the second group of the first downsampling layer and the second group of the multi-head differential kernel convolutional modules are connected in sequence. The first feature map at the second scale is input to the third group of the first downsampling layer and the third group of the multi-head differential kernel convolutional modules to obtain the first feature map at the third scale; wherein the third group of the first downsampling layer and the third group of the multi-head differential kernel convolutional modules are connected in sequence. The first feature map at the third scale is input to the fourth group of the first downsampling layer and the fourth group of the multi-head differential kernel convolutional modules to obtain the first feature map at the fourth scale; wherein the fourth group of the first downsampling layer and the fourth group of the multi-head differential kernel convolutional modules are connected in sequence. The first feature map at the fourth scale is input to the multi-scale large convolutional fusion module to obtain the second feature map. The feature map determination module is used to determine the maximum-scale feature map, minimum-scale feature map, and intermediate-scale feature map in the first feature map at different scales; wherein, the intermediate-scale feature map is the feature map in the first feature map at different scales excluding the maximum-scale feature map and the minimum-scale feature map; The feature map fusion module is used to input the second feature map and the intermediate scale feature map into the feature fusion network for feature fusion to obtain third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of adaptive convolutional attention modules; The target detection module is used to input third feature maps of different scales into the target detection network to detect targets and obtain fatigue driving detection results.

[0007] According to a third aspect of the embodiments of this application, an electronic device is provided, including: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method of the first aspect.

[0008] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the method of the first aspect.

[0009] This application embodiment acquires a driver's face image to be detected; inputs the driver's face image to be detected into a feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by a multi-scale large convolutional fusion module; wherein, the feature extraction network includes multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolutional modules, and a multi-scale large convolutional fusion module connected in sequence; the maximum scale feature map, minimum scale feature map, and intermediate scale feature map are determined in the first feature maps of different scales; wherein, the intermediate scale feature map is the feature map in the first feature maps of different scales excluding the maximum scale feature map and the minimum scale feature map; the second feature map and the intermediate scale feature map are input into a feature fusion network for feature fusion, obtaining third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of adaptive convolutional attention modules; the third feature maps of different scales are input into a target detection network for target detection, obtaining fatigue driving detection results. This application improves the feature extraction network's learning ability for local and global features by using multiple sets of multi-head differential kernel convolutional modules for feature extraction. Furthermore, the feature extraction capability of the feature extraction network is further enhanced by a multi-scale large convolutional fusion module. Multiple sets of adaptive convolutional attention modules enable better fusion of the second feature map and intermediate-scale feature maps, thereby improving the accuracy of fatigue driving detection.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application.

[0011] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0012] Figure 1 A flowchart illustrating a fatigue driving detection method provided in one embodiment of this application; Figure 2 This is a structural block diagram of a fatigue driving detection device provided in one embodiment of this application; Figure 3 This is a schematic block diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0014] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0015] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0016] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0017] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0018] Please see Figure 1 This is a flowchart illustrating a fatigue driving detection method provided in one embodiment of this application. The fatigue driving detection method provided in this embodiment includes the following steps: S10: Obtain the face image of the driver to be detected.

[0019] Among them, the entity that performs the fatigue driving detection method can be a fatigue driving detection device, which includes, but is not limited to, computers, tablets, and mobile phones.

[0020] The driver's face image to be detected can be a driver's face image uploaded by the user to the fatigue driving detection device, a driver's face image captured by the fatigue driving detection device from the Internet, or a face image of the driver taken in real time during the driving process.

[0021] Optionally, the driver's face image to be detected can be a series of face images parsed from a driver face detection video. The driver face detection video is a video recording the driver's driving behavior during the driving process.

[0022] S20: Input the driver's face image to be detected into the feature extraction network for feature extraction, and obtain first feature maps of different scales output by multiple sets of multi-head differential kernel convolution modules and second feature maps output by multi-scale large convolution fusion modules; wherein, the feature extraction network includes multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolution modules and multi-scale large convolution fusion modules connected in sequence.

[0023] The feature extraction network is used to extract features from the driver's face image to be detected, and can be understood as the backbone network in the Yolov5 model.

[0024] In this embodiment, the first downsampling layer is used to reduce the scale of the image. The multi-head differential kernel convolution module utilizes a multi-head large-small convolution structure designed with multi-head hybrid convolution and large-small convolution kernels, enabling the feature extraction network to retain most of the effective global information. It separates local and global information from the two scales of the multi-head hybrid convolution to concatenate and extract gradient information, thereby achieving global information extraction and local feature detail enhancement, thus improving the feature extraction capability of the feature extraction network. The multi-scale large convolution fusion module utilizes a combination of depthwise separable convolution and dilated convolution to capture contextual information of different dimensions, further improving the feature extraction capability of the feature extraction network.

[0025] By inputting the driver's face image to be detected into multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolution modules, and a multi-scale large convolution fusion module connected in sequence, multiple first feature maps with successively decreasing scales and the final second feature map can be obtained.

[0026] S30: Determine the maximum scale feature map, minimum scale feature map, and intermediate scale feature map in the first feature map at different scales; wherein, the intermediate scale feature map is the feature map in the first feature map at different scales excluding the maximum scale feature map and the minimum scale feature map.

[0027] In this embodiment of the application, the scale of each first feature map is obtained, and the scale of each first feature map is compared in turn. The first feature map with the largest scale is determined to be the maximum scale feature map, the first feature map with the smallest scale is determined to be the minimum scale feature map, and the first feature maps other than the first feature map with the largest scale and the first feature map with the smallest scale are determined to be intermediate scale feature maps.

[0028] S40: Input the second feature map and the intermediate scale feature map into the feature fusion network for feature fusion to obtain third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of adaptive convolutional attention modules.

[0029] The feature fusion network is used to perform multi-scale feature fusion on the feature maps and then send the fused features to the object detection network. The feature fusion network can be understood as the Neck network in the Yolov5 model.

[0030] In the embodiments of this application, the adaptive convolutional attention module utilizes the combination design of attention mechanism and adaptive convolution module to enhance the feature fusion capability of the network and achieve faster convergence speed.

[0031] The second feature map and the intermediate scale feature map are input into the feature fusion network for feature fusion. Each set of adaptive convolutional attention modules will output a second feature map of one scale.

[0032] S50: Input the third feature maps of different scales into the target detection network to perform target detection and obtain fatigue driving detection results.

[0033] The object detection network is used to perform multi-object detection on the feature map. The object detection network can be understood as the head network in the Yolov5 model.

[0034] In this embodiment of the application, third feature maps of different scales are input into the target detection network to detect different targets, and fatigue driving detection results are determined based on the state detection results of each target.

[0035] According to the embodiments of this application, a driver's face image to be detected is acquired; the driver's face image to be detected is input into a feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by multi-scale large convolutional fusion modules; wherein, the feature extraction network includes multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolutional modules, and multi-scale large convolutional fusion modules connected in sequence; the maximum scale feature map, minimum scale feature map, and intermediate scale feature map are determined in the first feature maps of different scales; wherein, the intermediate scale feature map is the feature map in the first feature maps of different scales excluding the maximum scale feature map and the minimum scale feature map; the second feature map and the intermediate scale feature map are input into a feature fusion network for feature fusion, obtaining third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of adaptive convolutional attention modules; the third feature maps of different scales are input into a target detection network for target detection, obtaining fatigue driving detection results. This application improves the feature extraction network's learning ability for local and global features by using multiple sets of multi-head differential kernel convolutional modules for feature extraction. Furthermore, the feature extraction capability of the feature extraction network is further enhanced by a multi-scale large convolutional fusion module. Multiple sets of adaptive convolutional attention modules enable better fusion of the second feature map and intermediate-scale feature maps, thereby improving the accuracy of fatigue driving detection.

[0036] In one embodiment, step S20 includes steps S201 to S205, as follows: S201: Input the driver's face image to be detected into the first group of first downsampling layers and the first group of multi-head differential kernel convolution modules to obtain the first feature map at the first scale; wherein, the first group of first downsampling layers and the first group of multi-head differential kernel convolution modules are connected in sequence.

[0037] The number of groups corresponding to the first downsampling layer in the feature extraction network and the number of groups corresponding to the multi-head differential kernel convolution module are not limited and can be set according to actual needs.

[0038] In this embodiment, the feature extraction network includes four sets of first downsampling layers, four sets of multi-head differential kernel convolutional modules, and a multi-scale large convolutional fusion module. Specifically, the first set of first downsampling layers, the first set of multi-head differential kernel convolutional modules, the second set of first downsampling layers, the second set of multi-head differential kernel convolutional modules, the third set of first downsampling layers, the third set of multi-head differential kernel convolutional modules, the fourth set of first downsampling layers, the fourth set of multi-head differential kernel convolutional modules, and the multi-scale large convolutional fusion module are connected sequentially.

[0039] The driver's face image to be detected is input into the first downsampling layer of the first group for downsampling to obtain the downsampled image. The downsampled image is then input into the first multi-head differential kernel convolution module for convolution processing to obtain the first feature map at the first scale.

[0040] S202: Input the first feature map of the first scale into the second group of the first downsampling layer and the second group of the multi-head differential kernel convolution module to obtain the first feature map of the second scale; wherein, the second group of the first downsampling layer and the second group of the multi-head differential kernel convolution module are connected in sequence.

[0041] In this embodiment, the first feature map at the first scale is input to the second group of first downsampling layers for downsampling to obtain the downsampled feature map. The downsampled feature map is then input to the second group of multi-head differential kernel convolution modules for convolution processing to obtain the first feature map at the second scale.

[0042] S203: Input the first feature map of the second scale into the first downsampling layer of the third group and the third multi-head differential kernel convolution module to obtain the first feature map of the third scale; wherein, the first downsampling layer of the third group and the third multi-head differential kernel convolution module are connected in sequence.

[0043] In this embodiment, the first feature map at the second scale is input to the first downsampling layer of the third group for downsampling to obtain the downsampled feature map. The downsampled feature map is then input to the third multi-head differential kernel convolution module for convolution processing to obtain the first feature map at the third scale.

[0044] S204: Input the first feature map of the third scale into the first downsampling layer of the fourth group and the fourth multi-head differential kernel convolution module to obtain the first feature map of the fourth scale; wherein, the first downsampling layer of the fourth group and the fourth multi-head differential kernel convolution module are connected in sequence.

[0045] In this embodiment, the first feature map at the third scale is input to the first downsampling layer of the fourth group for downsampling to obtain the downsampled feature map. The downsampled feature map is then input to the fourth multi-head differential kernel convolution module for convolution processing to obtain the first feature map at the fourth scale.

[0046] Sorted from largest to smallest scale, they are: the driver's face image to be detected, the first feature map at the first scale, the first feature map at the second scale, the first feature map at the third scale, and the first feature map at the fourth scale.

[0047] Therefore, the first feature map at the first scale is the largest scale feature map, the first feature map at the fourth scale is the smallest scale feature map, and the first feature maps at the second and third scales are both intermediate scale feature maps.

[0048] S205: Input the first feature map of the fourth scale into the multi-scale large convolution fusion module to obtain the second feature map.

[0049] In this embodiment, the first feature map at the fourth scale is input into the multi-scale large convolutional fusion module for convolutional fusion processing to obtain the second feature map.

[0050] By setting up multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolution modules, and multi-scale large convolution fusion modules connected in sequence, feature extraction can be performed on the driver face image to be detected, and first feature maps and second feature maps of different scales can be obtained automatically and quickly.

[0051] In one embodiment, the multi-head differential kernel convolutional module includes a first convolutional module, a second convolutional module, a third convolutional module, a first convolutional layer, a second convolutional layer, a multi-head differential kernel convolutional layer, a first activation layer, and a batch normalization layer. Step S201 includes steps S2011 to S2017, as follows: S2011: Input the driver's face image to be detected into the first downsampling layer of the first group to obtain the downsampling feature map.

[0052] Each set of multi-head differential kernel convolutional modules includes a first convolutional module, a second convolutional module, a third convolutional module, a first convolutional layer, a second convolutional layer, a multi-head differential kernel convolutional layer, a first activation layer, and a batch normalization layer.

[0053] In the embodiments of this application, the first convolutional module, the second convolutional module, and the third convolutional module are all CBS modules, used for feature extraction and downsampling. The CBS module includes a Conv (convolutional) layer, a BN (batch normalization) layer, and a Silu (activation function) layer.

[0054] The driver's face image to be detected is input into the first downsampling layer of the first group to reduce the image scale and obtain the downsampled feature map.

[0055] S2012: Input the downsampled feature map into the first convolutional module to obtain the first initial feature map.

[0056] In this embodiment of the application, the downsampled feature map is input into the first convolution module for convolution processing to obtain the first initial feature map.

[0057] S2013: Input the downsampled feature map into the second convolution module to obtain the second initial feature map.

[0058] In this embodiment of the application, the downsampled feature map is input into the second convolution module for convolution processing to obtain the second initial feature map.

[0059] S2014: Input the second initial feature map into the first convolutional layer, the multi-head differential kernel convolutional layer, the second convolutional layer, the first activation layer, and the batch normalization layer to obtain the third initial feature map; wherein, the first convolutional layer, the multi-head differential kernel convolutional layer, the second convolutional layer, the first activation layer, and the batch normalization layer are connected in sequence.

[0060] Among them, the multi-head differential kernel convolutional layer incorporates multiple convolutional kernels of different sizes in a single convolution, enabling it to capture features of various spatial scales within a single convolution. Furthermore, the multi-head hybrid convolution uses both large and small convolutional kernels; large kernels increase the receptive field for feature extraction, while small kernels extract richer feature information.

[0061] In this embodiment, the second initial feature map is input into the first convolutional layer, the multi-head differential kernel convolutional layer, the second convolutional layer, the first activation layer, and the batch normalization layer for feature extraction to obtain the third initial feature map. The multi-head differential kernel convolutional layer includes a 5x5 large convolutional kernel and a 3x3 small convolutional kernel. It can split the output of the previous layer (the first convolutional layer) into two parts, perform convolution processing using the large and small convolutional kernels respectively, and then concatenate the corresponding convolution processing results of the two parts before outputting them to the next layer (the second convolutional layer).

[0062] S2015: Summing the second initial feature map with the third initial feature map yields the fourth initial feature map.

[0063] In this embodiment of the application, the second initial feature map and the third initial feature map are added element by element to obtain the fourth initial feature map.

[0064] S2016: Concatenate the fourth initial feature map with the first initial feature map to obtain the fifth initial feature map.

[0065] In this embodiment of the application, the fourth initial feature map and the first initial feature map are stacked in the channel dimension to obtain the fifth initial feature map.

[0066] S2017: Input the fifth initial feature map into the third convolution module to obtain the first feature map at the first scale.

[0067] In this embodiment of the application, the fifth initial feature map is input into the third convolution module for convolution processing to obtain the first feature map at the first scale.

[0068] It should be understood that the generation process of the first feature map at each scale corresponding to steps S202 to S204 can be referred to steps S2011 to S2017, and will not be repeated here.

[0069] By setting up a first convolutional module, a second convolutional module, a third convolutional module, a first convolutional layer, a second convolutional layer, a multi-head differential kernel convolutional layer, a first activation layer, and a batch normalization layer, the first feature map at the first scale can be obtained automatically and quickly.

[0070] In one embodiment, the multi-scale large convolutional fusion module includes a third convolutional layer, a fourth convolutional layer, a second activation layer, a third activation layer, and a multi-scale large convolutional fusion layer; the multi-scale large convolutional fusion layer includes a fifth convolutional layer, a sixth convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a third depthwise separable convolutional layer, a first dilated convolutional layer, a second dilated convolutional layer, a third dilated convolutional layer, and a max pooling layer. Step S205 includes steps S2051 to S2058, as follows: S2051: Input the first feature map of the fourth scale into the third convolutional layer and the second activation layer to obtain the first intermediate feature map; wherein the third convolutional layer and the second activation layer are connected in sequence.

[0071] The second activation layer uses the GELU activation function.

[0072] In this embodiment, the first feature map at the fourth scale is input into the third convolutional layer for convolution processing, and the convolution processing result is input into the second activation layer for activation processing to obtain the first intermediate feature map.

[0073] S2052: Input the first intermediate feature map into the fifth convolutional layer to obtain the second intermediate feature map.

[0074] In this embodiment of the application, the first intermediate feature map is input into the fifth convolutional layer for convolution processing to obtain the second intermediate feature map.

[0075] S2053: Input the second intermediate feature map into the first depthwise separable convolutional layer and the first dilated convolutional layer to obtain the third intermediate feature map; wherein the first depthwise separable convolutional layer and the first dilated convolutional layer are connected in sequence.

[0076] The first depth-separable convolutional layer and the first dilated convolutional layer both include convolutional layers, batch normalization layers, and activation layers. The convolutional layer structures of the first depth-separable convolutional layer and the first dilated convolutional layer are different.

[0077] In this embodiment, the second intermediate feature map is input into a first depthwise separable convolutional layer for depthwise separable convolution, and the result of the depthwise separable convolution is input into a first dilated convolutional layer for dilated convolution to obtain a third intermediate feature map. The kernel size of both the first depthwise separable convolutional layer and the first dilated convolutional layer is 7*7.

[0078] S2054: Input the second intermediate feature map into the second depthwise separable convolutional layer and the second dilated convolutional layer to obtain the fourth intermediate feature map; wherein the second depthwise separable convolutional layer and the second dilated convolutional layer are connected in sequence.

[0079] The second depth separable convolutional layer and the second dilated convolutional layer both include convolutional layers, batch normalization layers, and activation layers. The convolutional layer structures of the second depth separable convolutional layer and the second dilated convolutional layer are different.

[0080] In this embodiment, the second intermediate feature map is input into a second depthwise separable convolutional layer for depthwise separable convolution, and the result of the depthwise separable convolution is input into a second dilated convolutional layer for dilated convolution to obtain a fourth intermediate feature map. The kernel size of both the second depthwise separable convolutional layer and the second dilated convolutional layer is 11*11.

[0081] S2055: Input the second intermediate feature map into the third depthwise separable convolutional layer and the third dilated convolutional layer to obtain the fifth intermediate feature map; wherein the third depthwise separable convolutional layer and the third dilated convolutional layer are connected in sequence.

[0082] The third depth separable convolutional layer and the third dilated convolutional layer both include convolutional layers, batch normalization layers, and activation layers. The convolutional layer structures of the third depth separable convolutional layer and the third dilated convolutional layer are different.

[0083] In this embodiment, the second intermediate feature map is input into a third depthwise separable convolutional layer for depthwise separable convolution, and the result of the depthwise separable convolution is input into a third dilated convolutional layer for dilated convolution to obtain a fifth intermediate feature map. The kernel size of both the third depthwise separable convolutional layer and the third dilated convolutional layer is 21*21.

[0084] S2056: Summing the third, fourth, and fifth intermediate feature maps yields the sixth intermediate feature map.

[0085] In this embodiment of the application, the third intermediate feature map, the fourth intermediate feature map, and the fifth intermediate feature map are added element by element to obtain the sixth intermediate feature map.

[0086] S2057: Input the sixth intermediate feature map into the sixth convolutional layer and the max pooling layer to obtain the seventh intermediate feature map; wherein the sixth convolutional layer and the max pooling layer are connected sequentially.

[0087] In this embodiment, the sixth intermediate feature map is input into the sixth convolutional layer for convolution processing, and the convolution processing result is input into the max pooling layer for pooling processing to obtain the seventh intermediate feature map.

[0088] S2058: Input the seventh intermediate feature map into the fourth convolutional layer and the third activation layer to obtain the second feature map; wherein the fourth convolutional layer and the third activation layer are connected in sequence.

[0089] In this embodiment, the seventh intermediate feature map is input into the fourth convolutional layer for convolution processing, and the convolution processing result is input into the third activation layer for activation processing to obtain the second feature map.

[0090] By setting a third convolutional layer, a fourth convolutional layer, a second activation layer, a third activation layer, and a multi-scale large convolutional fusion layer, the second feature map can be obtained automatically and quickly.

[0091] In one embodiment, the feature fusion network further includes a fourth convolutional module, a fifth convolutional module, multiple upsampling layers, and multiple second downsampling layers. The intermediate scale feature map includes a first intermediate scale feature map and a second intermediate scale feature map; wherein, the scale of the first intermediate scale feature map is smaller than the scale of the second intermediate scale feature map. Step S40 includes steps S401 to S413, as follows: S401: Input the second feature map into the fourth convolution module to obtain the fourth feature map.

[0092] In this embodiment of the application, the second feature map is input into the fourth convolution module for convolution processing to obtain the fourth feature map.

[0093] S402: Input the fourth feature map into the first upsampling layer to obtain the fifth feature map.

[0094] In this embodiment of the application, the fourth feature map is input into the first set of upsampling layers to enlarge the image scale and obtain the fifth feature map.

[0095] S403: Summing the fifth feature map with the first intermediate scale feature map yields the sixth feature map.

[0096] The fifth feature map has the same scale as the first intermediate scale feature map.

[0097] In this embodiment of the application, the fifth feature map is added element-wise to the first intermediate scale feature map to obtain the sixth feature map.

[0098] S404: Input the sixth feature map into the first group of adaptive convolutional attention modules and the fifth convolutional module to obtain the seventh feature map; wherein the first group of adaptive convolutional attention modules and the fifth convolutional module are connected in sequence.

[0099] In this embodiment, the sixth feature map is input into the first group of adaptive convolutional attention modules for adaptive convolution processing, and the result of the adaptive convolution processing is input into the fifth convolution module for convolution processing to obtain the seventh feature map.

[0100] S405: Input the seventh feature map into the second upsampling layer to obtain the eighth feature map.

[0101] In this embodiment, the seventh feature map is input to the second set of upsampling layers for image scaling to obtain the eighth feature map.

[0102] S406: Summing the eighth feature map with the second intermediate scale feature map yields the ninth feature map.

[0103] The eighth feature map has the same image scale as the second intermediate scale feature map.

[0104] In this embodiment of the application, the eighth feature map is added element-wise to the second intermediate scale feature map to obtain the ninth feature map.

[0105] S407: Input the ninth feature map into the second group of adaptive convolutional attention modules to obtain the third feature map at the first scale.

[0106] In this embodiment, the ninth feature map is input into the second group of adaptive convolutional attention modules for adaptive convolution processing to obtain the third feature map at the first scale.

[0107] S408: Input the third feature map of the first scale into the second downsampling layer of the first group to obtain the tenth feature map.

[0108] In this embodiment of the application, the third feature map of the first scale is input into the first group of second downsampling layers for scale reduction to obtain the tenth feature map.

[0109] S409: Summate the tenth feature map with the seventh feature map to obtain the eleventh feature map.

[0110] The tenth feature map has the same scale as the seventh feature map.

[0111] In this embodiment of the application, the tenth feature map and the seventh feature map are added element by element to obtain the eleventh feature map.

[0112] S410: Input the eleventh feature map into the third group of adaptive convolutional attention modules to obtain the third feature map at the second scale.

[0113] In this embodiment, the eleventh feature map is input into the third group of adaptive convolutional attention modules for adaptive convolution processing to obtain the third feature map at the second scale.

[0114] S411: Input the third feature map of the second scale into the second downsampling layer of the second group to obtain the twelfth feature map.

[0115] In this embodiment of the application, the third feature map at the second scale is input into the second downsampling layer of the second group for scale reduction to obtain the twelfth feature map.

[0116] S412: Summing the twelfth feature map with the fourth feature map yields the thirteenth feature map.

[0117] The twelfth feature map has the same scale as the fourth feature map.

[0118] In this embodiment of the application, the twelfth feature map is added element by element to the fourth feature map to obtain the thirteenth feature map.

[0119] S413: Input the thirteenth feature map into the fourth group of adaptive convolutional attention modules to obtain the third feature map at the third scale.

[0120] In this embodiment, the thirteenth feature map is input into the fourth group of adaptive convolutional attention modules for adaptive convolution processing to obtain the third feature map at the third scale.

[0121] It should be understood that the scale of the third feature map at the first scale is larger than the scale of the third feature map at the second scale, and the scale of the third feature map at the second scale is larger than the scale of the third feature map at the third scale.

[0122] By setting up a fourth convolutional module, a fifth convolutional module, multiple upsampling layers, and multiple second downsampling layers, third feature maps at different scales can be obtained automatically and quickly.

[0123] In one embodiment, the adaptive convolutional attention module includes a sixth convolutional module, a seventh convolutional module, an eighth convolutional module, a ninth convolutional module, a tenth convolutional module, and an adaptive convolutional attention layer. Step S404 includes steps S4041 to S4047, as follows: S4041: Input the sixth feature map into the sixth convolution module to obtain the first preliminary feature map.

[0124] Each set of adaptive convolutional attention modules includes a sixth convolutional module, a seventh convolutional module, an eighth convolutional module, a ninth convolutional module, a tenth convolutional module, and an adaptive convolutional attention layer.

[0125] In the embodiments of this application, the sixth, seventh, eighth, ninth, and tenth convolution modules are all CBS modules.

[0126] The sixth feature map is input into the sixth convolution module for convolution processing to obtain the first preliminary feature map.

[0127] S4042: Input the sixth feature map into the seventh convolution module to obtain the second preliminary feature map.

[0128] In this embodiment of the application, the sixth feature map is input into the seventh convolution module for convolution processing to obtain the second preliminary feature map.

[0129] S4043: Input the second preliminary feature map into the eighth convolutional module, the ninth convolutional module, and the adaptive convolutional attention layer to obtain the third preliminary feature map; wherein the eighth convolutional module, the ninth convolutional module, and the adaptive convolutional attention layer are connected in sequence.

[0130] The convolution kernel design of the adaptive convolutional attention layer adopts ECA adaptive parameters, making the kernel size k proportional to the channel dimension C. When capturing local cross-channel interaction information, the relationship between each channel and its k neighboring channels is considered.

[0131] In this embodiment of the application, the second preliminary feature map is input into the eighth convolutional module, the ninth convolutional module, and the adaptive convolutional attention layer for convolution processing to obtain the third preliminary feature map.

[0132] S4044: Summing the third preliminary feature map with the second preliminary feature map yields the fourth preliminary feature map.

[0133] In this embodiment of the application, the third preliminary feature map is added element by element to the second preliminary feature map to obtain the fourth preliminary feature map.

[0134] S4045: Concatenate the fourth preliminary feature map with the first preliminary feature map to obtain the fifth preliminary feature map.

[0135] In this embodiment of the application, the fourth preliminary feature map and the first preliminary feature map are stacked in the channel dimension to obtain the fifth preliminary feature map.

[0136] S4046: Input the fifth preliminary feature map into the tenth convolutional module to obtain the sixth preliminary feature map.

[0137] In this embodiment of the application, the fifth preliminary feature map is input into the tenth convolution module for convolution processing to obtain the sixth preliminary feature map.

[0138] S4047: Input the sixth preliminary feature map into the fifth convolution module to obtain the seventh feature map.

[0139] In this embodiment of the application, the sixth preliminary feature map is input into the fifth convolution module for convolution processing to obtain the sixth feature map.

[0140] By setting up the sixth, seventh, eighth, ninth, and tenth convolutional modules, as well as an adaptive convolutional attention layer, the seventh feature map can be obtained automatically and quickly.

[0141] In one embodiment, step S50 includes steps S501 to S502, as follows: S501: Input the third feature maps of different scales into the target detection network to detect the state of the eyes, mouth and head, and obtain the state detection results of the eyes, mouth and head.

[0142] Among them, the eye status detection results include open and closed eyes, the mouth status detection results include closed mouth and yawning mouth, and the head status detection results include head making a phone call and head not making a phone call.

[0143] S502: Based on the results of eye condition detection, mouth condition detection, and head condition detection, fatigue driving detection results are obtained.

[0144] In this embodiment of the application, it is possible to determine whether the driver is driving while fatigued based on at least one of the detection results of eye state detection, mouth state detection, and head state detection.

[0145] Specifically, if the driver closes their eyes, yawns, or makes a phone call while looking at the phone, it indicates that the driver is in a state of fatigued driving.

[0146] By performing target detection on third feature maps at different scales, the driver's fatigue driving state can be determined automatically and quickly.

[0147] The following are embodiments of the apparatus described in this application, which can be used to execute the methods described in the embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the methods described in the embodiments of this application.

[0148] Please see Figure 2 The diagram illustrates the structure of the fatigue driving detection device provided in this embodiment. The fatigue driving detection device 6 provided in this embodiment includes: The face image acquisition module 61 is used to acquire the face image of the driver to be detected; The feature map acquisition module 62 is used to input the driver's face image to be detected into the feature extraction network for feature extraction, and obtain first feature maps of different scales output by multiple sets of multi-head differential kernel convolution modules and second feature maps output by multi-scale large convolution fusion modules; wherein, the feature extraction network includes multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolution modules and multi-scale large convolution fusion modules connected in sequence; The feature map determination module 63 is used to determine the maximum scale feature map, the minimum scale feature map, and the intermediate scale feature map in the first feature map at different scales; wherein, the intermediate scale feature map is the feature map in the first feature map at different scales excluding the maximum scale feature map and the minimum scale feature map; The feature map fusion module 64 is used to input the second feature map and the intermediate scale feature map into the feature fusion network for feature fusion to obtain third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of adaptive convolutional attention modules; The target detection module 65 is used to input third feature maps of different scales into the target detection network for target detection and obtain fatigue driving detection results.

[0149] According to the embodiments of this application, a driver's face image to be detected is acquired; the driver's face image to be detected is input into a feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by multi-scale large convolutional fusion modules; wherein, the feature extraction network includes multiple sets of first downsampling layers, multiple sets of multi-head differential kernel convolutional modules, and multi-scale large convolutional fusion modules connected in sequence; the maximum scale feature map, minimum scale feature map, and intermediate scale feature map are determined in the first feature maps of different scales; wherein, the intermediate scale feature map is the feature map in the first feature maps of different scales excluding the maximum scale feature map and the minimum scale feature map; the second feature map and the intermediate scale feature map are input into a feature fusion network for feature fusion, obtaining third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of adaptive convolutional attention modules; the third feature maps of different scales are input into a target detection network for target detection, obtaining fatigue driving detection results. This application improves the feature extraction network's learning ability for local and global features by using multiple sets of multi-head differential kernel convolutional modules for feature extraction. Furthermore, the feature extraction capability of the feature extraction network is further enhanced by a multi-scale large convolutional fusion module. Multiple sets of adaptive convolutional attention modules enable better fusion of the second feature map and intermediate-scale feature maps, thereby improving the accuracy of fatigue driving detection.

[0150] The following are embodiments of the device described in this application, which can be used to execute the methods described in the embodiments of this application. For details not disclosed in the embodiments of the device described in this application, please refer to the methods described in the embodiments of this application.

[0151] Please see Figure 3 This application also provides an electronic device 300, which may specifically be a computer, mobile phone, tablet computer, etc. In an exemplary embodiment of this application, the electronic device 300 is a computer, which may include: at least one processor 301, at least one memory 302, at least one display, at least one network interface 303, user interface 304, and at least one communication bus 305.

[0152] The user interface 304 is primarily used to provide an input interface for the user and to acquire user input data. Optionally, the user interface may also include a standard wired interface or a wireless interface.

[0153] The network interface 303 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0154] The communication bus 305 is used to enable communication between these components.

[0155] The processor 301 may include one or more processing cores. The processor connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0156] The memory 302 may include random access memory (RAM) or read-only memory. Optionally, the memory may include a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor. Figure 3 A memory, as a type of computer storage medium, may include an operating system, a network communication module, a user interface module, and operating applications.

[0157] The processor can be used to call the application program of the fatigue driving detection method stored in the memory and specifically execute the method steps of the above-described embodiments. For the specific execution process, please refer to the detailed description shown in the embodiments, which will not be repeated here.

[0158] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0159] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting driver fatigue, characterized in that, The method includes the following steps: Obtain the facial image of the driver to be detected; The driver's face image to be detected is input into a feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by a multi-scale large convolutional fusion module; wherein, the feature extraction network includes multiple sets of first downsampling layers, multiple sets of the multi-head differential kernel convolutional modules, and the multi-scale large convolutional fusion module connected in sequence; wherein, the step of inputting the driver's face image to be detected into the feature extraction network for feature extraction to obtain first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by the multi-scale large convolutional fusion module includes: inputting the driver's face image to be detected into a first set of first downsampling layers and a first set of multi-head differential kernel convolutional modules to obtain a first feature map of a first scale; wherein, the first set of first downsampling layers and the first set of multi-head differential kernel convolutional modules are connected in sequence; The first feature map at the first scale is input to the second group of the first downsampling layer and the second group of the multi-head differential kernel convolutional modules to obtain the first feature map at the second scale; wherein, the second group of the first downsampling layer and the second group of the multi-head differential kernel convolutional modules are connected in sequence; the first feature map at the second scale is input to the third group of the first downsampling layer and the third group of the multi-head differential kernel convolutional modules to obtain the first feature map at the third scale; wherein, the third group of the first downsampling layer and the third group of the multi-head differential kernel convolutional modules are connected in sequence; the first feature map at the third scale is input to the fourth group of the first downsampling layer and the fourth group of the multi-head differential kernel convolutional modules to obtain the first feature map at the fourth scale; wherein, the fourth group of the first downsampling layer and the fourth group of the multi-head differential kernel convolutional modules are connected in sequence; the first feature map at the fourth scale is input to the multi-scale large convolutional fusion module to obtain the second feature map; Determine the maximum-scale feature map, minimum-scale feature map, and intermediate-scale feature map in the first feature map at different scales; wherein, the intermediate-scale feature map is the feature map in the first feature map at different scales excluding the maximum-scale feature map and the minimum-scale feature map; The second feature map and the intermediate-scale feature map are input into a feature fusion network for feature fusion to obtain third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of the adaptive convolutional attention modules; The third feature maps at different scales are input into the target detection network for target detection to obtain fatigue driving detection results.

2. The fatigue driving detection method according to claim 1, characterized in that: The multi-head differential kernel convolution module includes a first convolution module, a second convolution module, a third convolution module, a first convolution layer, a second convolution layer, a multi-head differential kernel convolution layer, a first activation layer, and a batch normalization layer; The step of inputting the driver's face image to be detected into the first group of the first downsampling layers and the first group of the multi-head differential kernel convolution modules to obtain the first feature map at the first scale includes: The driver's face image to be detected is input into the first downsampling layer of the first group to obtain a downsampling feature map; The downsampled feature map is input into the first convolutional module to obtain the first initial feature map; The downsampled feature map is input into the second convolutional module to obtain the second initial feature map; The second initial feature map is input into the first convolutional layer, the multi-head differential kernel convolutional layer, the second convolutional layer, the first activation layer, and the batch normalization layer to obtain the third initial feature map; wherein the first convolutional layer, the multi-head differential kernel convolutional layer, the second convolutional layer, the first activation layer, and the batch normalization layer are connected in sequence. Summing the second initial feature map with the third initial feature map yields a fourth initial feature map. The fourth initial feature map is concatenated with the first initial feature map to obtain the fifth initial feature map; The fifth initial feature map is input into the third convolution module to obtain the first feature map at the first scale.

3. The fatigue driving detection method according to claim 1, characterized in that: The multi-scale large convolutional fusion module includes a third convolutional layer, a fourth convolutional layer, a second activation layer, a third activation layer, and a multi-scale large convolutional fusion layer; the multi-scale large convolutional fusion layer includes a fifth convolutional layer, a sixth convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a third depthwise separable convolutional layer, a first dilated convolutional layer, a second dilated convolutional layer, a third dilated convolutional layer, and a max pooling layer; The step of inputting the first feature map at the fourth scale into the multi-scale large convolutional fusion module to obtain the second feature map includes: The first feature map at the fourth scale is input into the third convolutional layer and the second activation layer to obtain a first intermediate feature map; wherein the third convolutional layer and the second activation layer are connected in sequence. The first intermediate feature map is input into the fifth convolutional layer to obtain the second intermediate feature map; The second intermediate feature map is input into the first depthwise separable convolutional layer and the first dilated convolutional layer to obtain the third intermediate feature map; wherein, the first depthwise separable convolutional layer and the first dilated convolutional layer are connected in sequence. The second intermediate feature map is input into the second depthwise separable convolutional layer and the second dilated convolutional layer to obtain a fourth intermediate feature map; wherein the second depthwise separable convolutional layer and the second dilated convolutional layer are connected in sequence; The second intermediate feature map is input into the third depthwise separable convolutional layer and the third dilated convolutional layer to obtain the fifth intermediate feature map; wherein the third depthwise separable convolutional layer and the third dilated convolutional layer are connected in sequence; The third intermediate feature map, the fourth intermediate feature map, and the fifth intermediate feature map are summed to obtain the sixth intermediate feature map; The sixth intermediate feature map is input into the sixth convolutional layer and the max pooling layer to obtain the seventh intermediate feature map; wherein the sixth convolutional layer and the max pooling layer are connected sequentially. The seventh intermediate feature map is input to the fourth convolutional layer and the third activation layer to obtain the second feature map; wherein the fourth convolutional layer and the third activation layer are connected in sequence.

4. The fatigue driving detection method according to claim 1, characterized in that: The feature fusion network also includes a fourth convolutional module, a fifth convolutional module, multiple upsampling layers, and multiple second downsampling layers; The intermediate-scale feature map includes a first intermediate-scale feature map and a second intermediate-scale feature map; wherein the scale of the first intermediate-scale feature map is smaller than the scale of the second intermediate-scale feature map. The step of inputting the second feature map and the intermediate-scale feature map into a feature fusion network for feature fusion to obtain third feature maps of different scales output by multiple sets of adaptive convolutional attention modules includes: The second feature map is input into the fourth convolutional module to obtain the fourth feature map; The fourth feature map is input into the first group of upsampling layers to obtain the fifth feature map; The fifth feature map is summed with the first intermediate scale feature map to obtain the sixth feature map; The sixth feature map is input into the first group of adaptive convolutional attention modules and the fifth convolutional module to obtain the seventh feature map; wherein the first group of adaptive convolutional attention modules and the fifth convolutional module are connected in sequence; The seventh feature map is input into the second set of upsampling layers to obtain the eighth feature map; The eighth feature map is summed with the second intermediate scale feature map to obtain the ninth feature map; The ninth feature map is input into the second group of adaptive convolutional attention modules to obtain the third feature map at the first scale; The third feature map at the first scale is input into the second downsampling layer of the first group to obtain the tenth feature map; Summing the tenth feature map with the seventh feature map yields the eleventh feature map; The eleventh feature map is input into the third group of adaptive convolutional attention modules to obtain the third feature map at the second scale; The third feature map at the second scale is input into the second downsampling layer of the second group to obtain the twelfth feature map; Summing the twelfth feature map with the fourth feature map yields the thirteenth feature map; The thirteenth feature map is input into the fourth group of adaptive convolutional attention modules to obtain the third feature map at the third scale.

5. The fatigue driving detection method according to claim 4, characterized in that: The adaptive convolutional attention module includes a sixth convolutional module, a seventh convolutional module, an eighth convolutional module, a ninth convolutional module, a tenth convolutional module, and an adaptive convolutional attention layer; The step of inputting the sixth feature map into the first group of adaptive convolutional attention modules and the fifth convolutional module to obtain the seventh feature map includes: The sixth feature map is input into the sixth convolutional module to obtain the first preliminary feature map; The sixth feature map is input into the seventh convolutional module to obtain the second preliminary feature map; The second preliminary feature map is input into the eighth convolutional module, the ninth convolutional module, and the adaptive convolutional attention layer to obtain the third preliminary feature map; wherein the eighth convolutional module, the ninth convolutional module, and the adaptive convolutional attention layer are connected in sequence; The third preliminary feature map is summed with the second preliminary feature map to obtain the fourth preliminary feature map; The fourth preliminary feature map is concatenated with the first preliminary feature map to obtain the fifth preliminary feature map; The fifth preliminary feature map is input into the tenth convolutional module to obtain the sixth preliminary feature map; The sixth preliminary feature map is input into the fifth convolution module to obtain the seventh feature map.

6. The fatigue driving detection method according to any one of claims 1 to 5, characterized in that: The step of inputting the third feature maps of different scales into the target detection network for target detection to obtain fatigue driving detection results includes: The third feature maps at different scales are input into the target detection network to detect the state of the eyes, mouth, and head, and the results of eye state detection, mouth state detection, and head state detection are obtained. Based on the eye state detection results, mouth state detection results, and head state detection results, fatigue driving detection results are obtained.

7. A fatigue driving detection device, characterized in that, include: The face image acquisition module is used to acquire the face image of the driver to be detected; The feature map acquisition module is used to input the driver's face image to be detected into a feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by a multi-scale large convolutional fusion module; wherein, the feature extraction network includes multiple sets of first downsampling layers, multiple sets of the multi-head differential kernel convolutional modules, and the multi-scale large convolutional fusion module connected in sequence; wherein, the step of inputting the driver's face image to be detected into the feature extraction network for feature extraction, obtaining first feature maps of different scales output by multiple sets of multi-head differential kernel convolutional modules and second feature maps output by the multi-scale large convolutional fusion module includes: inputting the driver's face image to be detected into a first set of first downsampling layers and a first set of multi-head differential kernel convolutional modules to obtain a first feature map of a first scale; wherein, the first set of first downsampling layers and the first set of multi-head differential kernel convolutional modules are connected in sequence. The first feature map at the first scale is input to the second group of the first downsampling layer and the second group of the multi-head differential kernel convolutional modules to obtain the first feature map at the second scale; wherein, the second group of the first downsampling layer and the second group of the multi-head differential kernel convolutional modules are connected in sequence; the first feature map at the second scale is input to the third group of the first downsampling layer and the third group of the multi-head differential kernel convolutional modules to obtain the first feature map at the third scale; wherein, the third group of the first downsampling layer and the third group of the multi-head differential kernel convolutional modules are connected in sequence; the first feature map at the third scale is input to the fourth group of the first downsampling layer and the fourth group of the multi-head differential kernel convolutional modules to obtain the first feature map at the fourth scale; wherein, the fourth group of the first downsampling layer and the fourth group of the multi-head differential kernel convolutional modules are connected in sequence; the first feature map at the fourth scale is input to the multi-scale large convolutional fusion module to obtain the second feature map; The feature map determination module is used to determine the maximum-scale feature map, the minimum-scale feature map, and the intermediate-scale feature map in the first feature map at different scales; wherein, the intermediate-scale feature map is the feature map in the first feature map at different scales excluding the maximum-scale feature map and the minimum-scale feature map; The feature map fusion module is used to input the second feature map and the intermediate scale feature map into the feature fusion network for feature fusion to obtain third feature maps of different scales output by multiple sets of adaptive convolutional attention modules; wherein, the feature fusion network includes multiple sets of the adaptive convolutional attention modules; The target detection module is used to input the third feature map at different scales into the target detection network for target detection and to obtain fatigue driving detection results.

8. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images