Driver Behavior Recognition Method Based on Geometric Spatial Feature Fusion Deep Attention Network

Abstract

This invention proposes a driver behavior recognition method based on a deep attention network with geometric spatial fusion features. The method includes: constructing standard driver behavior image datasets SAA13 and Special; building a driver backbone spatial feature generation model based on YOLO-pose; researching a dual-stream fusion feature that combines the contextual geometric features of the image with the driver's backbone spatial features as input; researching and designing a multi-source collaborative attention MCA-DBR module specifically for complex driving scenarios; embedding the MCA-DBR module into shallow layers of the network to enhance the network's ability to extract local details of the driver's features; inputting the enhanced feature map into the classification network's construction module for deep semantic feature extraction; and using the softmax activation function to decode the deep features extracted by DAN-GSFF and output the final recognition result. DAN-GSFF achieves 90.11% accuracy and 96.2 FPS on the SAA13 dataset, providing a feasible method to improve the network's ability to recognize driver behavior.

Description

Technical Field

[0001] This invention belongs to the research fields of intelligent transportation and intelligent driving, and specifically relates to a driver behavior recognition method based on a deep attention network with geometric spatial fusion features. Background Technology

[0002] With rapid societal development and the widespread application of intelligent driving technology, motor vehicles have become the preferred mode of transportation for travel. However, at the same time, the number of traffic accidents is also increasing. A large number of traffic accidents are attributed to distracted driving, highlighting the crucial role of standardized driving in vehicle safety. Today, the successful deployment of Advanced Driver Assistance Systems (ADAS) makes real-time monitoring of driver status possible; therefore, research on driver behavior recognition systems can help reduce the number and severity of traffic accidents caused by distracted driving, ensuring driving safety.

[0003] In driver behavior recognition tasks, existing networks are typically trained and validated on a single dataset, thus over-relying on and fitting to these specific driving environments, thereby limiting their generalization ability in complex driving scenarios. Furthermore, existing networks are often limited to classification based solely on the driver's contextual geometric features, failing to adequately consider the importance of backbone spatial features. This limitation makes it difficult for the network to accurately capture subtle yet crucial visual differences between different driver behaviors. Therefore, this invention proposes a deep attention driver behavior network, DAN-GSFF, which combines global image information and driver pose information as input, and comprehensively considers both the driver's contextual geometric features and backbone spatial features for classification, by constructing a comprehensive dataset containing diverse scenes and drivers for training. Summary of the Invention

[0004] Purpose of the invention: To overcome the shortcomings of existing technologies, this invention provides a driver behavior recognition method based on a deep attention network with geometric spatial fusion features. This method simultaneously considers the driver's contextual geometric features and backbone spatial features for classification. By embedding a designed multidimensional collaborative attention module in the shallow layer of the network, it guides the network to selectively focus on the local details of the driver in the image, thereby achieving accurate recognition of driver behavior in complex driving scenarios.

[0005] Technical Solution: To achieve the above objectives, this invention provides a driver behavior recognition method based on a deep attention network with geometric spatial fusion features, comprising the following steps:

[0006] S1: Construct a standard image dataset of driver behavior, including the SAA13 comprehensive dataset and the special extreme low light sub-dataset;

[0007] S2: Construct a driver backbone spatial feature generation model based on YOLO-pose;

[0008] S3: Research on dual-stream fusion features that combine the contextual geometric features of the image with the backbone spatial features of the driver as input;

[0009] S4: Research and design a multi-source collaborative attention MCA-DBR module specifically for complex driving scenarios;

[0010] S5: The study embeds the MCA-DBR (multidimensional collaborative attention for driver behavior recognition) module into the shallow layers of the network to enhance the network's ability to extract local detailed features of the driver.

[0011] S6: Input the enhanced feature map into the classification network building module to extract deep semantic features, and gradually extract higher-level semantic features.

[0012] S7: Use the softmax activation function to perform feature decoding on the deep features extracted by DAN-GSFF, accurately map the features to the corresponding driver behavior categories, and output the final recognition result.

[0013] Furthermore, the specific method for constructing the driver behavior standard image dataset in step S1 is as follows:

[0014] S1-1: This dataset integrates the unique scene characteristics of five datasets—State Farm, AUCV2, ASU, 100-Driver, and 3MDAD—with driver data, naming the integrated dataset SAA13 based on the first character of each character. The integrated dataset covers scenarios including different lighting conditions (e.g., normal daylight or low light at night); different camera monitoring perspectives (e.g., to the driver's right or left); and cluttered, dynamic background environments (e.g., static or moving backgrounds). With 272 drivers, the diversity of scenes and drivers makes SAA13 more representative of real-world scenarios. SAA13 covers ten different driving behaviors: safe driving (C0), texting with left hand (C1), making a phone call with left hand (C2), texting with right hand (C3), making a phone call with right hand (C4), operating the center console (C5), drinking water or eating (C6), leaning back (C7), fixing hair or applying makeup (C8), and talking to passengers (C9). The latter nine are distracted driving behaviors that require focused detection.

[0015] S1-2: To effectively reduce the duplication and similarity between adjacent images in the dataset, an interval sampling strategy is adopted to perform secondary filtering of the image data. The sampling interval is determined based on the stability or dynamism of the background environment, i.e., whether the data collection vehicle is stationary or in motion. Specifically, when the vehicle is stationary, the global background of the image remains almost unchanged, so a larger sampling interval is used; while when the vehicle is in motion, the background is constantly changing, so the sampling interval is correspondingly shorter compared to the stationary state. In addition, to increase the proportion of special scene images in the dataset, this application also adopts a smaller sampling interval. After interval sampling, a total of 59,872 images were collected, involving 272 different drivers. According to the driver segmentation strategy, the dataset is divided into training set, validation set, and test set, i.e., the data of the training set, test set, and validation set should be collected from different driver groups;

[0016] S1-3: These images, totaling 4961, were collected separately under extreme low-light conditions. These images show a significant difference in driver profile sharpness compared to images taken under normal lighting conditions, thus constructing a special extreme low-light sub-dataset.

[0017] Furthermore, the specific method for constructing the driver skeleton spatial feature generation model based on YOLO-pose in step S2 is as follows: YOLO-pose is selected as the pose estimator to accurately detect the key point positions of the driver in the image and obtain the key point coordinate information. Given that in driving scenarios, the driver's lower body movements usually have no substantial effect on the recognition task, while hand movements often convey crucial information and play a vital role, this application focuses only on detecting six key points of the driver's upper body. These key points specifically include: {1: left wrist, 2: left elbow, 3: left shoulder, 4: right shoulder, 5: right elbow, 6: right wrist}. After obtaining these key points, they are connected in a specific order to generate and visualize the complete upper body skeleton pose information, i.e., 1-2-3 form the left arm, 4-5-6 form the right arm, while retaining 3 and 4 to represent the connection of the shoulders. This process not only captures the driver's body posture but also preserves the relative positional relationships between key points, providing crucial evidence for subsequent behavior analysis.

[0018] Furthermore, step S3 investigates a dual-stream fusion feature that combines the contextual geometric features of the image with the skeletal spatial features of the driver as input. Specifically, the dual-stream features input to the subsequent network fuse the contextual geometric features of the image and the skeletal spatial features of the driver. The contextual geometric features of the image originate from global information of a color image adjusted to 224×224 dimensions. This information comprehensively covers image details, constructing a rich scene and background context for the network. The skeletal spatial features of the driver are based on a skeletal framework generated by pose estimation technology. This framework not only outlines the driver's key points and their spatial relationships but also prompts the network to focus on local regions directly related to the driver's behavior, deeply capturing the spatial correlation of body posture. This is crucial for accurately identifying the driver's action patterns and intentions. The fusion of these two features not only greatly enhances the accuracy and stability of the network in driver behavior analysis but also significantly increases the importance of local regions around the driver's key points, thereby enhancing the network's ability to identify and extract these key features.

[0019] Furthermore, in step S4, a multi-source collaborative attention module (MCA-DBR) specifically designed for complex driving scenarios is researched and designed. The specific method is as follows: MCA-DBR is a lightweight multi-dimensional collaborative attention module that simultaneously models the width, height, and channel dimensions, and has a parallel three-branch structure. Its workflow is as follows: The input feature map with a scale of C×H×W is permuted with the height and width dimensions in the channel dimension, and then the feature map is input into the left and middle branches respectively. Then, the same operation is performed along with the right branch: First, the input feature F1 is compressed using global average pooling and global standard deviation pooling, and these two compressed features are fused by broadcast element-wise summation to aggregate cross-dimensional features. This fused feature is represented as X×1×1, F2. Second, the fused feature F2 is permuted into 1×1×X, F3, and processed by a 2D convolution operation with a kernel size of 1×K. Then, the convolutional feature F4 is permuted back to X×1×1, F5. Finally, F5 is processed by the sigmoid function and broadcast element-wise multiplication with the input feature F1. After this, the previously permuted feature maps are permuted back to their original dimensions. Finally, the three feature maps are broadcast element-wise averaged and summed to output the refined features.

[0020] Furthermore, in step S5, the MCA-DBR module is embedded in a shallow layer of the network to enhance the network's ability to extract local details of the driver. Specifically, the attention mechanism, as a highly flexible and integrable module, has a wide range of embedding options within the network architecture to adapt to different functional and performance requirements. Generally, shallow layers of a neural network are primarily responsible for capturing local details of an image, while deeper layers tend to abstract semantically rich information. Local details of the driver, such as the contours of the hand, arm, and head, as well as global information about small objects like water cups and mobile phones, are the most critical features of various driver behaviors. Therefore, embedding the designed MCA-DBR module in a shallow layer of the network can significantly enhance the network's ability to extract key local details, which are crucial for fine-grained driver behavior recognition tasks. Simultaneously, it can effectively suppress interference from irrelevant features such as the background and cockpit environment.

[0021] Furthermore, in step S6, the enhanced feature map is input into the classification network construction module for deep semantic feature extraction, gradually extracting higher-level semantic features. Specifically, the feature map enhanced by MCA-DBR is then imported into subsequent construction modules to perform a more in-depth and detailed feature extraction task. During this process, the large-kernel depthwise separable convolution technique not only significantly reduces the computational complexity of the model but also, by increasing the convolution kernel size, enables the network to more efficiently capture global contextual information in the image. In addition, the inverse residual structure achieves deep feature mining by first expanding the number of feature channels, then applying depthwise convolution for feature extraction, and finally compressing the number of channels using linear layers. This design not only improves the network's ability to capture complex features but also significantly enhances its generalization performance, ensuring excellent performance in various complex scenarios. In this stage, with the continuous action of multiple stacked modules, the network gradually extracts higher-level semantic features. These features encompass the driver's global profile information, such as the contour features of the head, torso, and limbs, laying a solid and accurate foundation for subsequent specific category decoding, thus enabling DAN-GSFF to more accurately identify information such as the driver's behavior or state.

[0022] Furthermore, in step S7, the softmax activation function is used to perform feature decoding on the deep features extracted by DAN-GSFF, accurately mapping the features to the corresponding driver behavior categories and outputting the final recognition result. The specific process is as follows: The deep features extracted by DAN-GSFF are first processed by a global average pooling layer. This layer effectively reduces the feature dimensionality while preserving global information, avoiding the risk of overfitting, and providing a more compact feature representation for subsequent processing. Next, a layer normalization layer is applied to the pooled features. This layer standardizes the activation values ​​of each sample, ensuring the stability and consistency of the data across different layers, which helps improve the training efficiency and performance of the model. Finally, the linear layer, as the core of the classifier, is responsible for mapping the normalized features to a predefined driver behavior category space. This layer, through the weight matrix and bias vector, achieves a linear transformation from the feature space to the category space, providing the foundation for the final classification decision. After the Linear layer, the softmax activation function is used to calculate the probability distribution of each category, which realizes accurate decoding of deep features, accurately maps the features to the corresponding driver behavior category, and finally outputs the recognition result, thus completing the accurate judgment of driver behavior.

[0023] Beneficial effects: Compared with the prior art, the present invention realizes automatic recognition of driver behavior based on geometric space fusion feature deep attention network, achieves accurate recognition of driver behavior in complex driving scenarios, and improves the generalization ability and robustness of the network in complex driving scenarios. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the DAN-GSFF network structure.

[0025] Figure 2 Examples of images from the SAA13 dataset.

[0026] Figure 3 This is a performance comparison chart of DAN-GSFF with other literature models.

[0027] Figure 4 This is a box plot comparison chart of DAM-GSFF and five other models.

[0028] Figure 5 The image shows the prediction confusion matrix results of DAM-GSFF and five other models on the SAA13 dataset. Detailed Implementation

[0029] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 As shown, this invention provides a driver behavior recognition method based on a deep attention network with geometric spatial fusion features, comprising the following steps:

[0031] S1: Construct a standard image dataset of driver behavior, specifically as follows:

[0032] S1-1: This dataset integrates the unique scene characteristics of five datasets—State Farm, AUCV2, ASU, 100-Driver, and 3MDAD—with driver data, naming the integrated dataset SAA13 based on the first character of each character. The integrated dataset covers scenarios including different lighting conditions (e.g., normal daylight or low light at night); different camera monitoring perspectives (e.g., to the driver's right or left); and cluttered, dynamic background environments (e.g., static or moving backgrounds). With 272 drivers, the diversity of scenes and drivers makes SAA13 more representative of real-world scenarios. SAA13 covers ten different driving behaviors: safe driving (C0), texting with left hand (C1), making a phone call with left hand (C2), texting with right hand (C3), making a phone call with right hand (C4), operating the center console (C5), drinking water or eating (C6), leaning back (C7), fixing hair or applying makeup (C8), and talking to passengers (C9). The latter nine are distracted driving behaviors that require focused detection.

[0033] S1-2: To effectively reduce the duplication and similarity between adjacent images in the dataset, an interval sampling strategy is adopted to perform secondary filtering of the image data. The sampling interval is determined based on the stability or dynamism of the background environment, i.e., whether the data collection vehicle is stationary or in motion. Specifically, when the vehicle is stationary, the global background of the image remains almost unchanged, so a larger sampling interval is used; while when the vehicle is in motion, the background is constantly changing, so the sampling interval is correspondingly shorter compared to the stationary state. In addition, to increase the proportion of images with special scenes in the dataset, this application also uses a smaller sampling interval. After interval sampling, a total of 59,872 images were collected, involving 272 different drivers. According to the driver segmentation strategy, the dataset is divided into training set, validation set, and test set, i.e., the data in the training set, test set, and validation set should be collected from different driver groups. The specific segmentation information is shown in Figure 1, and the example diagram is as follows. Figure 2 As shown;

[0034] Table 1. Detailed information about the SAA13 dataset.

[0035]

[0036] S1-3: These images, totaling 4961, were collected separately under extreme low-light conditions. These images show a significant difference in driver profile sharpness compared to images taken under normal lighting conditions, thus constructing a special extreme low-light sub-dataset.

[0037] S2: The specific method for constructing a driver skeleton spatial feature generation model based on YOLO-pose is as follows: YOLO-pose is selected as the pose estimator to accurately detect the key point positions of the driver in the image and obtain the key point coordinate information. Given that in driving scenarios, the driver's lower body movements usually have no substantial effect on the recognition task, while hand movements often convey crucial information and play a vital role, this application focuses only on detecting six key points of the driver's upper body. These key points specifically include: {1: left wrist, 2: left elbow, 3: left shoulder, 4: right shoulder, 5: right elbow, 6: right wrist}. After obtaining these key points, they are connected in a specific order to generate and visualize the complete upper body skeleton pose information, i.e., 1-2-3 form the left arm, 4-5-6 form the right arm, while retaining 3 and 4 to represent the connection of the shoulders. This process not only captures the driver's body posture but also preserves the relative positional relationships between key points, providing crucial evidence for subsequent behavior analysis.

[0038] S3: This study investigates a dual-stream fusion feature model that combines the contextual geometric features of an image with the skeletal spatial features of the driver as input. Specifically, the dual-stream features input to the subsequent network fuse the contextual geometric features of the image and the skeletal spatial features of the driver. The contextual geometric features of the image originate from global information of a color image adjusted to 224×224 dimensions. This information comprehensively covers image details, constructing a rich scene and background context for the network. The skeletal spatial features of the driver are based on a skeletal framework generated by pose estimation technology. This framework not only delineates the driver's key points and their spatial relationships but also prompts the network to focus on local regions directly related to the driver's behavior, deeply capturing the spatial correlation of body posture. This is crucial for accurately identifying the driver's action patterns and intentions. The fusion of these two features not only greatly enhances the accuracy and stability of the network in driver behavior analysis but also significantly increases the importance of local regions around the driver's key points, thereby enhancing the network's ability to identify and extract these key features.

[0039] S4: Research and design a multi-source collaborative attention module (MCA-DBR) specifically for complex driving scenarios. The specific method is as follows: MCA-DBR is a lightweight multi-dimensional collaborative attention module that simultaneously models the width, height, and channel dimensions, and has a parallel three-branch structure. Its workflow is as follows: The input feature map with a scale of C×H×W is permuted with the height and width dimensions in the channel dimension, and then the feature map is input into the left and middle branches respectively. Then, the same operation is performed together with the right branch: First, the input feature F1 is compressed using global average pooling and global standard deviation pooling, and then fused with these two compressed features through broadcast element-wise summation to aggregate cross-dimensional features. This fused feature is represented as X×1×1, F2; second, the fused feature F2 is permuted into 1×1×X, F3, and processed through a two-dimensional convolution operation with a kernel size of 1×K; then, the convolutional feature F4 is permuted back to X×1×1, F5; finally, F5 is processed by the sigmoid function and subjected to broadcast element-wise multiplication with the input feature F1. After this, the previously replaced feature maps are replaced back to their original dimensions. Finally, the three feature maps are broadcast element-wise averaged and summed to output the refined features.

[0040] S5: This study embeds the MCA-DBR module into a shallow layer of the network to enhance its ability to extract local details of the driver's behavior. Specifically, the attention mechanism, as a highly flexible and integrable module, has a wide range of embedding options within the network architecture to adapt to different functional and performance requirements. Generally, shallow layers of a neural network are primarily responsible for capturing local details of an image, while deeper layers tend to abstract semantically rich information. Local details of the driver, such as the contours of the hand, arm, and head, as well as global information about small objects like water cups and mobile phones, are the most critical features of various driver behaviors. Therefore, embedding the designed MCA-DBR module into a shallow layer significantly enhances the network's ability to extract key local details, which are crucial for fine-grained driver behavior recognition tasks. Simultaneously, it effectively suppresses interference from irrelevant features such as the background and cockpit environment.

[0041] S6: The enhanced feature maps are input into the classification network's construction module for deep semantic feature extraction, gradually extracting higher-level semantic features. Specifically, the feature maps enhanced by MCA-DBR are then imported into subsequent construction modules to perform more in-depth and detailed feature extraction tasks. During this process, the large-kernel deep separable convolution technique not only significantly reduces the model's computational complexity but also, by increasing the convolution kernel size, enables the network to more efficiently capture global contextual information in the image. Furthermore, the inverse residual structure achieves deep feature mining by first expanding the number of feature channels, then applying depthwise convolution for feature extraction, and finally compressing the number of channels using linear layers. This design not only improves the network's ability to capture complex features but also significantly enhances its generalization performance, ensuring excellent performance in various complex scenarios. In this stage, with the continuous action of multiple stacked modules, the network gradually extracts higher-level semantic features. These features encompass the driver's global profile information, such as the contour features of the head, torso, and limbs, laying a solid and accurate foundation for subsequent specific category decoding, thus enabling DAN-GSFF to more accurately identify information such as the driver's behavior or state.

[0042] S7: The softmax activation function is used to perform feature decoding on the deep features extracted by DAN-GSFF, accurately mapping the features to the corresponding driver behavior categories and outputting the final recognition result. The specific process is as follows: The deep features extracted by DAN-GSFF are first processed by a global average pooling layer. This layer effectively reduces the feature dimensionality while preserving global information, avoiding the risk of overfitting, and providing a more compact feature representation for subsequent processing. Next, a layer normalization layer is applied to the pooled features. This layer normalizes the activation values ​​of each sample, ensuring the stability and consistency of the data across different layers, which helps improve the training efficiency and performance of the model. Finally, the linear layer, as the core of the classifier, is responsible for mapping the normalized features to a predefined driver behavior category space. This layer, through the weight matrix and bias vector, achieves a linear transformation from the feature space to the category space, providing the foundation for the final classification decision. After the Linear layer, the softmax activation function is used to calculate the probability distribution of each category, which realizes accurate decoding of deep features, accurately maps the features to the corresponding driver behavior category, and finally outputs the recognition result, thus completing the accurate judgment of driver behavior.

[0043] To verify the driver behavior recognition performance of each component of DAN-GSFF, ablation experiments were conducted. The experimental results are shown in Table 2, where Atten. represents embedding MCA-DBR and Pose represents fusing the driver's skeleton spatial features. On the SAA13 dataset, the base network ConvNeXt achieved an accuracy of 82.99% and an F1 score of 83.52%. This result demonstrates that ConvNeXt, facing complex and varied driving scenarios, can sensitively capture detailed driver information with its powerful feature extraction capabilities, thereby achieving accurate driver behavior recognition. After embedding the designed multidimensional collaborative attention module MCA-DBR into the base network, the network's accuracy improved by 2.22%, precision by 2.9%, recall by 2.1%, and F1 score by 2.49%. These improvements indicate that embedding visual attention effectively enhances the network's ability to extract key features, such as driver contours, poses, and hand-held objects, while weakening the attention to irrelevant regions, such as complex global background features. Furthermore, by integrating the driver's backbone spatial features with the image's contextual geometric features into the baseline model, the network's precision, accuracy, recall, and F1-score improved by 5.36%, 5.62%, 5.35%, and 5.48%, respectively. This improvement demonstrates the effectiveness of backbone spatial features in driver behavior recognition tasks, proving that backbone spatial features can guide the network to focus more on extracting local region information closely related to driver behavior, thereby effectively improving the network's performance in recognizing driver behavior. Finally, by integrating the driver's backbone spatial features with the image's contextual geometric features as input, and feeding them into the classification network enhanced with MCA-DBR, the network achieved optimal performance in recognizing driver behavior, for example, with an accuracy of 90.11%, an accuracy of 90.75%, a recall of 90.23%, and an F1-score of 90.49%.

[0044] Table 2 Overall Ablation Experiment Results

[0045]

[0046] To verify the effectiveness of the method proposed in this invention, comparative experiments were conducted with methods from other literature. The experimental results are shown in Table 3 and... Figure 3As shown in the figure, the proposed DAN-GSFF is a deep attention network that integrates image contextual geometric features and driver backbone spatial features as input. It first uses a pose estimator to detect the location of driver keypoints and generate the driver's backbone spatial features, which are then combined with the image's contextual geometric features as input. Furthermore, this application adds a multi-dimensional collaborative attention module suitable for complex driving scenarios to a shallow layer of the classification network to enhance the network's feature extraction capability for driver geometric information. Ultimately, DAN-GSFF achieved state-of-the-art recognition performance on SAA13, with accuracy, precision, recall, and F1-score all exceeding 90%, reaching 90.11%, 90.75%, 90.23%, and 90.49%, respectively, while achieving 96.2 FPS. Compared to other driver behavior recognition networks, it achieves the best performance in balancing recognition accuracy and inference speed. In addition, it also achieved a recognition accuracy of 86.85% on a special subset of datasets under extremely low light conditions, second only to BiRSWinT. However, its parameter count and FPS are far superior to BiRSWinT. Therefore, DAN-GSFF has more comprehensive generalization ability and robustness in complex driving scenarios.

[0047] Table 3 shows the experimental results compared with other literature methods.

[0048]

[0049]

[0050] In addition to the results shown in Table 3, various visualization methods were used to compare the recognition performance of the proposed DAN-GSFF with five other high-performing networks: ResNet50, ResNeXt50, HCF, OLCMNet, and BiRSWinT. First, ten cross-validation experiments were conducted. For each experiment, 1000 images were randomly selected from the test set. The recognition accuracy of these ten cross-validation results was plotted as a box plot, as shown below. Figure 4 As shown in the figure, the DAN-GSFF model of this application has the lowest lower quartile, ensuring its recognition performance in complex driving scenarios. Furthermore, the relatively small box variation of DAN-GSFF indicates its strong robustness in complex driving scenarios. Additionally, to better illustrate the recognition accuracy results of these six models, Figure 5The confusion matrix analysis provides more details, showing that only the DAN-GSFF and BiRSWinT networks achieved over 80% accuracy in recognizing conversations with passengers (C9), with most misclassifications being incorrectly identified as safe driving (C0). This is because the difference between these two types of driving posture behaviors in certain scenarios lies only in slight head postures, with other features remaining largely similar. Furthermore, for the DAN-GSFF network proposed in this application, four types of driver behaviors achieved over 90% accuracy, with the body leaning back (C7) reaching an impressive 97.35%, indicating that the posture features of this type of driver behavior are significantly different from other driving behaviors. However, the DAN-GSFF network's accuracy in recognizing the right hand holding a phone (C2) was only 78.54%, with most misclassifications being for drinking water or eating (C6) and fixing hair or applying makeup (C8). This result reflects that when faced with a driver raising their right hand, the DAN-GSFF network struggles to identify whether the object in the right hand is a phone, a water cup, or nothing at all, indicating a weak ability to extract features from specific objects.

[0051] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A driver behavior recognition method based on a deep attention network with geometric spatial fusion features, characterized in that: Includes the following steps: S1: Construct a standard image dataset of driver behavior, including the SAA13 comprehensive dataset and a special extreme low light sub-dataset; among them, the unique scene characteristics of the five datasets State Farm, AUCV2, ASU, 100-Driver and 3MDAD are integrated with the driver, and this integrated dataset is named SAA13 according to the first character. S2: Construct a driver backbone spatial feature generation model based on YOLO-pose to generate the driver's backbone spatial features; S3: A dual-stream fusion feature that combines the contextual geometric features of the image with the backbone spatial features of the driver as input, wherein the contextual geometric features of the image are derived from global information of the color image; S4: A multi-source collaborative attention MCA-DBR module specifically designed for complex driving scenarios; MCA-DBR is a lightweight multidimensional collaborative attention module that models width, height, and channel dimensions simultaneously, and features a parallel three-branch structure. Its workflow is as follows: The input feature map with a scale of C×H×W is permuted with the height and width dimensions along the channel dimension, and then the feature map is input into the left and middle branches respectively. Afterwards, it undergoes the same operation as the right branch: First, the input feature map... F 1 Compression is achieved through global average pooling and global standard deviation pooling, and these two compressed features are fused by broadcast element-wise summation to aggregate cross-dimensional features. This fused feature is represented as follows: X×1×1 , F 2 Secondly, the characteristics after fusion F 2 Replaced with 1×1×X , F 3 and through a kernel size of 1× K The features are processed using two-dimensional convolution operations; then, the convolved features... F 4 Replaced X×1×1 , F 5 ;at last, F 5 Processed using the sigmoid function, along with the input features F 1 Perform broadcast element-wise multiplication; after that, the previously permuted feature maps are permuted back to their original dimensions, and finally the three feature maps are broadcast element-wise averaged and summed to output the refined features. S5: The dual-stream fusion features processed by the Stem layer are input into the MCA-DBR module embedded in the shallow layer of the network to enhance the network's ability to extract local details of the driver and obtain an enhanced feature map. S6: Input the enhanced feature map into the classification network building module to extract deep semantic features, and gradually extract higher-level semantic features to obtain deep features; S7: Use the softmax activation function to perform feature decoding on the extracted deep features, accurately map the features to the corresponding driver behavior categories, and output the final recognition result.

2. The driver behavior recognition method based on a deep attention network with geometric spatial fusion features according to claim 1, characterized in that: The specific method for step S1 is as follows: S1-1: The integrated dataset covers scenarios including different lighting conditions, different camera monitoring angles, and cluttered dynamic background environments; SAA13 covers ten different driving behaviors, namely safe driving C0, texting with left hand C1, making a phone call with left hand C2, texting with right hand C3, making a phone call with right hand C4, operating the center console C5, drinking water or eating C6, leaning back C7, fixing hair or applying makeup C8, and talking to passengers C9. The latter nine are distracted driving behaviors that need to be detected in detail. S1-2: To effectively reduce the duplication and similarity between adjacent images in the dataset, an interval sampling strategy is adopted to perform secondary filtering of the image data. The sampling interval is determined based on the stability or dynamism of the background environment, i.e., whether the data collection vehicle is stationary or in motion. Specifically, when the vehicle is stationary, the global background of the image remains almost unchanged, so a larger sampling interval is used; while when the vehicle is in motion, the background is constantly changing, so the sampling interval is correspondingly shorter compared to the stationary state. To increase the proportion of special scene images in the dataset, a smaller sampling interval is used. After interval sampling, the dataset is divided into training set, validation set, and test set according to the driver segmentation strategy, i.e., the data in the training set, test set, and validation set should be collected from different driver groups. S1-3: Collect images separately under extreme low light conditions. These images are completely different from those taken under normal lighting conditions in terms of driver profile sharpness, thus constructing a special extreme low light sub-dataset.

3. The driver behavior recognition method based on a deep attention network with geometric spatial fusion features according to claim 1, characterized in that: The specific method of step S2 is as follows: YOLO-pose is used as a pose estimator to accurately detect the key point position of the driver in the image and obtain the key point coordinate information; given that in driving scenarios, the driver's lower body movements usually do not have a substantial effect on the recognition task, while hand movements often convey key information and play a crucial role. Therefore, the system focuses on detecting only six key points of the driver's upper body, specifically: {1: left wrist, 2: left elbow, 3: left shoulder, 4: right shoulder, 5: right elbow, 6: right wrist}. After obtaining these key points, they are connected in a specific order to generate and visualize the complete upper body skeletal posture information, i.e., 1-2-3 form the left arm, 4-5-6 form the right arm, while retaining 3 and 4 to represent the connection of the shoulders.

4. The driver behavior recognition method based on a deep attention network with geometric spatial fusion features according to claim 1, characterized in that: The specific method of step S3 is as follows: the dual-stream features input into the subsequent network fuse the contextual geometric features of the image and the skeletal spatial features of the driver; wherein, the contextual geometric features of the image are derived from the global information of the color image adjusted to a size of 224×224, which fully covers the details of the image and constructs a rich scene and background context for the network; while the skeletal spatial features of the driver are based on the skeletal framework generated by the pose estimation technology. This framework not only outlines the key points of the driver and their spatial positional relationships, but also prompts the network to focus on the local areas directly related to the driver's behavior and deeply capture the spatial correlation of body posture.

5. The driver behavior recognition method based on a deep attention network with geometric spatial fusion features according to claim 1, characterized in that: The specific method of step S5 is as follows: As a highly flexible and integrable module, the attention mechanism has a wide range of choices in the network architecture for its embedding position to adapt to different functional and performance requirements; the shallow layers of the neural network are mainly responsible for capturing local details of the image, while the deep layers tend to abstract semantically rich information; the local details of the driver are the most critical feature information of various driver behaviors; therefore, the designed MCA-DBR module is embedded in the shallow layer of the network.

6. The driver behavior recognition method based on a deep attention network with geometric spatial fusion features according to claim 1, characterized in that: The specific content of step S6 is as follows: The feature map after MCA-DBR enhancement is then imported into the subsequent construction module, which aims to perform a more in-depth and detailed feature extraction task. In this process, the large kernel depth separable convolution technology not only greatly reduces the computational complexity of the model, but also enables the network to capture global contextual information in the image more efficiently by increasing the size of the convolution kernel. Furthermore, the inverse residual structure achieves in-depth mining of image features by first expanding the number of feature channels, then applying depthwise convolution for feature extraction, and finally compressing the number of channels with linear layers.

7. The driver behavior recognition method based on a deep attention network with geometric spatial fusion features according to claim 1, characterized in that: The specific process of step S7 is as follows: The extracted deep features are first processed by a global average pooling layer. This layer effectively reduces the feature dimensionality while preserving global information, avoiding the risk of overfitting, and providing a more compact feature representation for subsequent processing. Next, a layer normalization layer is applied to the pooled features. This layer standardizes the activation values ​​of each sample, ensuring the stability and consistency of the data across different layers, which helps improve the training efficiency and performance of the model. Finally, the Linear layer, as the core of the classifier, is responsible for mapping the normalized features to a predefined driver behavior category space. This layer achieves a linear transformation from the feature space to the category space through the weight matrix and bias vector, providing a foundation for the final classification decision. After the Linear layer, the softmax activation function is used to calculate the probability distribution of each category, achieving accurate decoding of the deep features and accurately mapping the features to the corresponding driver behavior category, ultimately outputting the recognition result and completing the accurate judgment of driver behavior.

Images