Application method based on SC-YOLOv11 model in driver health condition detection

Abstract

The invention discloses an application method in driver health condition detection based on an SC-YOLOv11 model, and the method comprises the steps: making a driver facial expression data set DAFER, carrying out the expansion and preprocessing of the data set, and dividing the data set into a training set, a verification set and a test set; building a YOLOv11 basic model, initializing model parameters, and training the basic model by adopting a training set; the YOLOv11 basic model is improved, and an improved SC-YOLOv11 model is obtained; the improved SC-YOLOv11 model is verified, and model indexes of the improved SC-YOLOv11 model and a YOLOv11 basic model are compared; and judging the health condition of the driver according to the real-time detection result of the improved SC-YOLOv11 model on the facial expression of the driver.

Description

TECHNICAL FIELD

[0001] The application relates to an application method of an SC-YOLOv11 model in driver health condition detection. BACKGROUND

[0002] During driving, sudden cardiovascular diseases (such as myocardial infarction or stroke) will affect the control of the vehicle and cause traffic accidents, threatening the safety of the driver and the public. Related research shows that when the driver's body is abnormal, the face will often be accompanied by a painful expression. Therefore, it is of great significance to monitor the driver's facial expression to determine whether the health condition is abnormal. SUMMARY

[0003] The application is to solve the problems existing in the prior art and provide an application method of an SC-YOLOv11 model in driver health condition detection.

[0004] The technical scheme adopted by the application is as follows:

[0005] S1: A driver facial expression dataset DAF is made, the dataset is expanded and preprocessed, and then the dataset is divided into a training set, a validation set and a test set;

[0006] S2: A YOLOv11 basic model is built, the parameters of the basic model are initialized, and the training set is used to train the basic model;

[0007] S3: The YOLOv11 basic model is improved to obtain an improved SC-YOLOv11 model.

[0008] S4: The improved SC-YOLOv11 model is verified.

[0009] S5: The improved SC-YOLOv11 model is used to determine the health condition of the driver according to the real-time detection result of the driver's facial expression.

[0010] Further, the improvement points of the improved SC-YOLOv11 model are as follows:

[0011] The first improvement point is to improve the bottleneck module in the C3K2 module in the YOLOv11 base model into a hybrid detail enhancement module. The improvement point of the hybrid detail enhancement module is to break through the traditional series design in the structure level and adopt a multi-branch parallel architecture to construct a module containing a conventional convolution (3x3 Conv), a detail enhancement convolution (DEConv), a depth separable convolution (3x3 DWConv), a wavelet convolution (WTConv), and a self-designed dual-flow channel attention mechanism (PSE). The dual-flow channel attention mechanism captures global average features and highlights local extreme features through parallel paths of average pooling and maximum pooling, respectively. The dual-flow processing realizes panoramic-focusing dual-modal representation of the feature map in the channel dimension. After nonlinear transformation and activation of the dual-path full connection layer, two complementary channel attention weight vectors are generated, which are finally fused and applied to the feature map in a weighted manner to realize adaptive focusing on key feature channels and significantly enhance the sensitivity and representation ability of the network to target features. The processing process of the hybrid detail enhancement module is as follows: first, an initial feature output is obtained through a 3x3 conventional convolution. Then, the output is split into two independent branches: the main branch introduces a 3x3 depth separable convolution for lightweight feature extraction, and combines the dual-flow attention mechanism to adaptively focus on key feature regions and enhance the sensitivity of the network to target features; the auxiliary branch uses a 3x3 detail enhancement convolution to specifically enhance the expression ability of fine structures and texture information in the image. The feature maps output by the two branches are fused and added, and then a 3x3 wavelet convolution is used to further capture the multi-scale high-frequency details of the image. In order to protect the integrity of the original input information, the initial input and the final processing result are fused through a residual connection to effectively avoid the gradient vanishing problem and realize efficient transmission and enhancement of feature information.

[0012] Improvement point 2: the PSABlock in the C2PSA module in the YOLOv11 base model is improved to a PAGBlock; the improvement point of the PAGBlock is that, in the structural design, a mode combining multi-scale pooling (AvgPool1x1, AvgPool2x2, AvgPool4x4) and feature interaction enhancement mechanism is adopted. First, the multi-scale pooling is used to widen the feature receptive field, and after splicing, the 1x1 convolution is preliminarily fused; then the learnable rotation position coding self-attention mechanism (DLRAttention) and the feature enhancement unit (FEU) are introduced, and the contribution ratio of the rotation attention branch and the feature enhancement branch is adaptively adjusted through the learnable gating weight coefficient (g1, g2). The FEU internally realizes fine feature interaction and weight regulation through a deep separable convolution (DWConv), a normal convolution (Conv2d) and an activation function; finally, GELU is used for activation, so that the feature can more accurately capture complex information in multi-scale extraction, cross-branch modulation and nonlinear activation, and improve the feature expression ability. The learnable rotation position coding self-attention mechanism: the query (Q), key (K) and value (V) vectors are generated through a convolution layer, and a learnable rotation position coding is designed to enhance the spatial position perception ability. The core innovation lies in adapting the rotation position embedding (RoPE) in the sequence data to the two-dimensional image feature, and enhancing the sensitivity of the attention mechanism to the spatial position through frequency modulation.

[0013] Improvement point 3: the lightweight shared convolution detection head introduces a lightweight shared convolution structure, reduces the calculation redundancy through a shared feature extraction layer, and uses Group Normalization instead of Batch Normalization to improve the stability of small batch training. In addition, the new regression head optimizes the boundary box prediction, introduces a scale adaptive factor (Scale), and improves the accuracy of target detection.

[0014] Further, the data set DAFER includes a plurality of images, and the images are classified into five categories according to facial expressions, corresponding to anger, fear, happiness, neutrality and pain. Each category of expression is further classified according to driving environment and non-driving environment, wherein the driving environment refers to a user in a driving position, and the non-driving environment refers to a user in a non-driving position.

[0015] The present application has the following beneficial effects:

[0016] The improved SC-YOLOv11 model is used for analyzing the facial expression of a driver to evaluate the health condition of the driver. In view of the problem of lack of expression data in the existing driving scene, the self-made data set DAFER is used, and the data set is processed. Secondly, the bottleneck module structure in the C3K2 module is optimized, and the mixed detail enhancement module is introduced to strengthen the information interaction ability between channels and the expression ability of detail features, while reducing the calculation complexity. Then, the PSA Block in the C2PSA module is improved to PAG Block, through multi-scale feature fusion, a learnable rotation position encoding attention mechanism and a feature enhancement unit, the information flow of the feature level is optimized through an adaptive gating mechanism, the sensitivity and robustness of the model to the subtle expression changes of the driver's face are significantly improved, not only the model's ability to capture information of different scales is enhanced, but also the feature expression is effectively optimized through the adaptive gating mechanism, further improving the performance of the model in complex dynamic scenes. Finally, a lightweight shared convolution detection head is introduced, and shared convolution and Scale layer are used to reduce the parameter amount while improving the detection performance. Based on the above improvements, the model can capture and analyze the facial expression features of the driver in real time and accurately, and then accurately reflect the health condition of the driver. BRIEF DESCRIPTION OF DRAWINGS

[0017] Fig. 1 is a flowchart of the present application.

[0018] Fig. 2 is an algorithm structure diagram of the improved SC-YOLOv11 model.

[0019] Fig. 3 is a basic structure diagram of the improved C3K2.

[0020] Fig. 4 is a basic structure diagram of PAG Block.

[0021] Fig. 5 is a basic structure diagram of the learnable rotation position encoding self-attention mechanism DLRAttention.

[0022] Fig. 6 is a basic structure diagram of the lightweight shared convolution detection head.

[0023] Fig. 7 is a PR curve diagram of the prediction result.

[0024] Fig. 8 is a detection result of the model on the DAFER data set test set. DETAILED DESCRIPTION

[0025] The application will be further described below in conjunction with the drawings.

[0026] As Figure 1 The application discloses an application method of an SC-YOLOv11 model in driver health condition detection, comprising the following steps:

[0027] S1: A driver facial expression dataset DAFER is made, and after the dataset is expanded and preprocessed, the dataset is divided into a training set, a validation set and a test set;

[0028] S2: A YOLOv11 base model is built, and the model parameters are initialized, and the training set is used to train the base model;

[0029] S3: The YOLOv11 base model is improved to obtain an improved SC-YOLOv11 model, and the improvement points include:

[0030] The Bottleneck module in the C3K2 module in the YOLOv11 base model is improved to a hybrid detail enhancement module;

[0031] The PSA Block in the C2PSA module in the YOLOv11 base model is improved to a PAG Block;

[0032] The detection head in the YOLOv11 base model is changed to a lightweight shared convolution detection head;

[0033] S4: The improved SC-YOLOv11 model is verified, and the model indicators are compared with the YOLOv11 base model;

[0034] S5: According to the real-time detection result of the driver's facial expression by the improved SC-YOLOv11 model, the health status of the driver is judged.

[0035] The application evaluates the health status of the driver by analyzing the facial expression of the driver through the SC-YOLOv11 model. In view of the problem of lack of expression data in the existing driving scene, the application uses a self-made data set DAFER, and processes the data set. Secondly, the Bottleneck module structure in the C3K2 module is optimized, and a mixed detail enhancement module is introduced to enhance the information interaction ability between channels and the expression ability of detail features, while reducing the computational complexity. Then, the PSABlock in the C2PSA module is improved to PAGBlock, through multi-scale feature fusion, a learnable rotation position coding attention mechanism and a feature enhancement unit, the information flow of the feature level is optimized through an adaptive gating mechanism, which significantly improves the sensitivity and robustness of the model to the subtle changes of the driver's facial expression, not only enhances the model's ability to capture information of different scales, but also effectively optimizes the feature expression through the adaptive gating mechanism, further improving the performance of the model in complex dynamic scenes. Finally, by introducing a lightweight shared convolution detection head, using shared convolution and Scale layer to reduce the parameter amount while improving the detection performance. Based on the above improvements, the model can capture and analyze the facial expression features of the driver in real time and accurately, and then accurately reflect the health status of the driver.

[0036] The application uses a self-made data set DAFER, which is composed of part of the public driving environment data set KMU-FED, YawDD, State Farm and the non-driving environment data set KDEF, Pain_CVModel, PEMF and the driving environment pictures on the network. Among them, the driving environment refers to the user in the driving position, and the non-driving environment refers to the user in the non-driving position.

[0037] 1-3 images of three expressions of anger, happiness and fear in the KMU-FED data set are selected to avoid the influence of continuous images on the results; images of half face and front face in the KDEF data set. 1-3 pain images of each person in the Pain_CVModel data set; 1-3 pain images of each person in the PEMF data set; neutral expression images in YawDD and State Farm. The self-made process of DAFER is as follows:

[0038] Firstly, the expression pictures of the driver in different driving environments and non-driving environments are obtained, and in view of the problem of single feature caused by the lack of driver expression pictures in the driving environment and the continuous images, a variety of data enhancement methods such as random cropping, affine transformation and Gaussian are used to expand the data set.

[0039] After the data set is expanded, the LabelImg labeling tool is used to label the driver's facial expression in the picture, and the self-made data set DAFER is obtained;

[0040] The picture of the data set is subjected to a fixed-size scaling preprocessing operation before the data input model, and the picture size is scaled to 640x640.

[0041] Figure 2 The algorithm structure diagram of the improved SC-YOLOv11 model. Figure 3 The improved C3K2 module.

[0042] Optimization of the Bottleneck module in the C3K2 module:

[0043] On the basis of the architecture of traditional convolution, the Bottleneck module is effectively optimized by introducing detail enhancement convolution, depth separable convolution, and double-flow channel attention mechanism, and upgrading the last-stage traditional convolution to wavelet convolution. This optimization strategy not only reduces the computational complexity, but also strengthens the cross-channel feature interaction and multi-scale feature expression capability, thereby improving the performance of the model.

[0044] Specifically,

[0045] 1) Let the input feature map x = [x1, x2, …, x n ] ∈ R H×W×C After the first layer of convolution, local features are extracted through 3x3 depth separable convolution to obtain DWConv(x). Then, to enhance the feature interaction ability between channels, a double-flow channel attention mechanism is designed. This mechanism uses two independent paths for pooling operation. The first path uses global average pooling to compress the feature map of each channel into a scalar. The second path uses global maximum pooling to perform maximum pooling on each channel to obtain the global maximum value. Then, the two pooling results are respectively passed through a fully connected layer, each path including a fully connected layer with a scaling factor to reduce the feature dimension, followed by a ReLU activation function and a Sigmoid activation function to generate the attention coefficients of each channel. The outputs of the two paths are added to form the final channel attention weight. Finally, the input feature map x is multiplied by the weighted attention coefficients to obtain the enhanced feature map F1, which is expressed as:

[0046] ,

[0047] Where: F sq represents the use of channel global average pooling operation; F max represents the use of channel global maximum pooling; i and j represent the values of spatial dimensions H and W, respectively; W1 and W2 represent weight parameters; F ex represents the excitation operation; σ is the Sigmoid activation function; F scale represents the channel-by-channel multiplication operation; S Cis the weight of channel C; Conv(x) is the output after the first layer of convolution.

[0048] 2) The input feature map Conv(x) is simultaneously input into the detail enhancement convolution operation to enhance its spatial resolution and detail features. The output results of the above two branches are added to realize multi-path feature fusion. Then, the fused features are processed using wavelet convolution to better capture feature information at different scales. Wavelet convolution can decompose and reconstruct features in different frequency domains, thereby enhancing the model's multi-scale expression ability of features. Finally, the original input information is retained through residual connection to avoid the gradient vanishing problem, and the formula is:

[0049] ,

[0050] ,

[0051] ,

[0052] where DEConv(x) is the output after the detail enhancement convolution, and WTConv(x) is the output after the wavelet convolution operation.

[0053] As shown in Figure 4 , the basic structure diagram of PAGBlock is shown. Three parallel adaptive average pooling operations are introduced to extract different granularity context features (1x1, 2x2 and 4x4 respectively) to enhance the ability to capture multi-scale information. At the same time, the newly added learnable gating parameters dynamically adjust the weights of each calculation branch, realizing adaptive weighting of different features, thereby further improving the network's expression ability of complex representations. In addition, PAGBlock combines residual connection, which can effectively alleviate the gradient vanishing problem, promote the transmission of information flow, and further improve the training stability and overall performance of the model. The specific process is:

[0054] 1) Multi-scale feature extraction:

[0055] The input feature map x ∈ R H×W×C is extracted through three parallel adaptive average pooling layers. In the feature reconstruction stage, an optimized bilinear interpolation algorithm is used to uniformly upsample each scale feature to the original spatial resolution. Through channel dimension splicing and 1x1 convolution, multi-scale feature deep fusion is realized, which significantly enhances the model's ability to jointly model local features and global semantics:

[0056] ,

[0057] ,

[0058] where u k(⋅) denotes bilinear interpolation up-sampling value to original resolution, k ∈ {1, 2, 4} is the division granularity of the pooling kernel (such as AvgPool2 downsamples the input to ), Conv(x) is a 1 1 convolution operation, and concat(x) is channel concatenation.

[0059] 2) Feature Enhancement Unit (FEU):

[0060] The fused features are first passed through a double-layer depth separable convolution and a nonlinear activation to construct a feature transformation path, i.e., first performing a depth separable convolution on the input features, then performing batch normalization and GELU activation, then performing a depth separable convolution and GELU activation to obtain intermediate features s, then performing convolution compression on s and generating channel weight coefficients through Sigmoid, and finally obtaining enhanced features x by convolving the weight and the original features and performing batch normalization. enhanced This process realizes adaptive learning of channel dependency through lightweight depth separable convolution and nonlinear transformation, effectively enhancing the expression ability of key feature channels:

[0061] ,

[0062] where σ is the Sigmoid activation function.

[0063] 3) Learnable Rotated Position Encoding Attention Mechanism Processing:

[0064] First, a convolution layer is used to generate query vector Q, key vector K, and value vector V from input x enhanced

[0065] ,

[0066] where * _conv denotes convolution operation.

[0067] Dynamic frequency calculation process:

[0068] By introducing dynamic learnable rotated position encoding, the embedding of each position (through sine / cosine function) can adapt to different spatial positions, and at the same time, the position encoding is learnable, thereby enhancing the flexibility of the model. For the frequency vector of the rotated position encoding, the calculation method is:

[0069] ,

[0070] where head_dim is the position embedding dimension, and i = 0, 2, 4,..., head_dim-2.

[0071] Rotated transformation embedding position information: ​

[0072] In the case of two-dimensional images, the core of the rotation position coding is to apply a rotation transform of frequency modulation to generate a rotation embedding of each position. Let freqs be the frequency vector, then for the input query (Q) and key (K):

[0073] ,

[0074] where x even and x odd correspond to even and odd dimensional components respectively.

[0075] Attention calculation and feature output:

[0076] The feature map fused with position information is projected by 1x1 convolution to realize spatial perception enhancement of the input image:

[0077] ,

[0078] where, , , is a normalization factor to stabilize gradient propagation.

[0079] 4) Dynamic gated multi-path fusion:

[0080] To improve the dynamic expression ability of the feature, the feature map x enhanced output by the feature enhancement unit and the feature map output output by the learnable rotation position coding attention mechanism are weighted and summed through a gating mechanism. Specifically, first, the two features are adaptively adjusted using gating coefficients g1 and g2 to measure the importance of different features. Then, the weighted features are connected with the input features x through a residual connection to ensure effective information transmission and avoid feature loss. The final fusion formula is as follows:

[0081] ,

[0082] ,

[0083] where [ω1, ω2] are learnable parameters defined in the model; is a Sigmoid activation function; and are gating coefficients.

[0084] Figure 5 is a basic structure diagram of a lightweight shared convolution detection head.

[0085] Firstly, the module expands the channel of the feature map through 1x1 shared convolution, which promotes the information interaction between different channels and enhances the diversity of features. Then, the module uses two 3x3 shared convolution kernels in succession to aggregate information, which captures the complex feature relationships of adjacent regions through deep convolution layers, reduces redundant information, and improves the probability of adjacent feature learning. Then, the features extracted by the shared convolution are input into the classification and regression heads, which respectively predict the bounding box regression value and class distribution of the target through convolution layers. Finally, the module scales the features through the Scale layer to enhance the preservation ability of multi-scale features, ensuring that target information of different scales can be fully learned and utilized.

[0086] The specific process is as follows:

[0087] 1) Input feature map:

[0088] The input feature map x i has a shape of CxHxW.

[0089] 2) Feature extraction and processing:

[0090] Use 1x1 shared convolution, and then use two 3x3 shared convolution kernels to capture local region information and better aggregate adjacent region features:

[0091] ,

[0092] where, , are different convolution kernels, * represents convolution operation, b is bias term, and are and the mean and standard deviation on each channel, is a very small constant to prevent the denominator from being 0, and are learnable scaling and bias parameters, and are the output feature maps after GN normalization.

[0093] 3) Target detection output:

[0094] The features processed by the shared convolution module will be input into the regression head and the classification head, which are used to predict the bounding box regression, respectively. The output of the regression head is the regression value of the target box.

[0095] For bounding box regression:

[0096] ,

[0097] where, x i is the input feature, Conv1×1 represents a 1x1 convolution operation.

[0098] 4) Scale layer:

[0099] In order to enhance the preservation ability of multi-scale features, a Scale layer is used to scale the features:

[0100] ,

[0101] x i is the input feature, and scale_factor is a learnable parameter used to adjust the features of different scales.

[0102] Figure 7 is the PR curve of the SC-YOLOv11 model on the self-made dataset DAFER, and Table 1 is the detection result graph. The accuracy of the SC-YOLOv11 model in driver expression detection reaches 97%, which is 1.4% higher than that of the YOLOv11 benchmark model. At the same time, the frame rate of the algorithm reaches 220.98 frame / s, although it has decreased, but the detection speed can still meet the standard of real-time detection. Figure 8 is the detection result of the model on the test set of the self-made dataset DAFER. As can be seen from the figure, the SC-YOLOv11 model performs well in the target detection task. It can accurately identify facial expressions in the face under different scenes, and the confidence value intuitively reflects the reliability of recognition; in complex postures (driving scene), the detection box positioning can effectively distinguish the face area, reduce misjudgment and missed detection, and show the significant advantages of the detection performance of the optimized model. Existing literature shows that facial expressions are closely related to pain and can be used as an effective indicator for pain assessment. When the SC-YOLOv11 model detects the driver's expression as angry, happy, fearful, and neutral, the driver's health status is in a normal state; when the SC-YOLOv11 model detects the driver's expression as pain, the driver's health status is in an abnormal state. Further verify the innovation and effectiveness of evaluating the health status of the driver through expression detection.

[0103] Table 1

[0104] The above only describes the preferred embodiments of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements can be made, and these improvements should also be considered within the scope of protection of the present application.

Claims

1. A method for applying the SC-YOLOv11 model to driver health status detection, characterized by: include S1: Create the driver facial expression dataset DAFER, expand and preprocess the dataset, and then divide the dataset into training set, validation set, and test set; S2: Build a YOLOv11 basic model, initialize the basic model parameters, and train the basic model using a training set; S3: Improve the YOLOv11 base model to obtain an improved SC-YOLOv11 model, wherein the improvements include: Improvement 1: The Bottleneck module in the C3K2 module of the YOLOv11 base model is improved into a hybrid detail enhancement module. This module adopts a multi-branch parallel architecture and processes the global average features and local extreme value features through an embedded two-stream channel attention mechanism (PSE) to generate two complementary channel attention weight vectors. These are finally fused with the weight vector after the detail enhancement convolution and applied to the feature map in a weighted manner to achieve adaptive focusing on key feature channels. Improvement 2: The PSABlock in the C2PSA module of the YOLOv11 base model is improved to PAGBlock. PAGBlock combines multi-scale pooling with a feature interaction enhancement mechanism. Multi-scale pooling broadens the feature receptive field and initially fuses the features to form a rotational attention branch and a feature enhancement branch. The rotational attention branch incorporates a learnable rotational position encoding self-attention mechanism (DLRAttention), and the feature enhancement branch includes a feature enhancement unit (FEU). Learnable gating weight coefficients are then used to adaptively adjust the contribution ratio of the two branches. Finally, the GELU activation function is used to enhance feature expression. Improvement 3: The detection head in the YOLOv11 base model is changed to a lightweight shared convolutional detection head; S4: Verify the improved SC-YOLOv11 model; S5: The improved SC-YOLOv11 model is used to detect the driver’s facial expressions in real time to determine the driver’s health status.

2. The method for applying the SC-YOLOv11 model to driver health status detection according to claim 1, characterized in that: The dataset DAFER includes several images, which are divided into five categories according to facial expressions: anger, fear, happiness, neutrality and pain. Each type of expression is further classified according to driving environment and non-driving environment, where the driving environment is when the user is in the driving seat, and the non-driving environment is when the user is not in the driving seat.

3. The method for applying the SC-YOLOv11 model to driver health status detection according to claim 1, characterized in that: The hybrid detail enhancement module adopts a multi-branch feature fusion architecture, specifically including: Use 3×3 ordinary convolution to obtain the initial feature output; The initial feature output is split into a main branch and an auxiliary branch: the main branch uses 3×3 depthwise separable convolution for lightweight feature extraction and combines it with a two-stream channel attention mechanism (PSE) to adaptively focus on key feature areas; the auxiliary branch uses 3×3 detail enhancement convolution to enhance the representation of image subtle structure and texture information; The feature maps output by the main branch and the auxiliary branch are fused and added, and processed by 3×3 wavelet convolution; The initial input is fused with the final processed result through residual connections.

4. The method for applying the SC-YOLOv11 model to driver health status detection according to claim 1, characterized in that: In improvement point 2, three multi-scale pooling operations, AvgPool1×1, AvgPool2×2, and AvgPool4×4, are used to broaden the feature receptive field, and after splicing, they are initially fused with 1×1 convolution.

5. The method for applying the SC-YOLOv11 model to driver health status detection according to claim 1, characterized in that: The PAGBlock includes a rotational attention branch and a feature enhancement branch. The rotational attention branch introduces a learnable rotational position encoding self-attention mechanism (DLRAttention), generates a query vector Q, a key vector K and a value vector V through a convolutional layer, and designs a learnable rotational position encoding to enhance spatial position perception capabilities; the feature enhancement branch contains a feature enhancement unit (FEU), which internally uses depthwise separable convolution (DWConv) and ordinary convolution (Conv2d) with activation functions to achieve fine feature interaction and weight control.

6. The method for applying the SC-YOLOv11 model to driver health status detection according to claim 1, characterized in that: The lightweight shared convolution detection head is designed based on a lightweight shared convolution structure, specifically including: Use shared feature extraction layers; Use Group Normalization instead of Batch Normalization; A scale-adaptive factor is introduced into the regression head.

Images