Driver behavior recognition method based on geometric space fusion feature deep attention network

Abstract

The invention provides a driver behavior recognition method based on a geometric space fusion feature deep attention network. The method comprises the following steps: constructing a driver behavior standard image data set SAA13 and specific; a driver backbone spatial feature generation model based on YOLO-pose is constructed; the context geometric features of the image and the backbone space features of the driver are used as input double-flow fusion features; a multi-source collaborative attention MCA-DBR module specially used in a complex driving scene is researched and designed; the MCA-DBR module is embedded into the shallow layer position of the network to enhance the extraction capability of the network on the local detail features of the driver; inputting the enhanced feature map into a building module of a classification network for deep semantic feature extraction; and decoding the depth features extracted by the DAN-GSFF by using a softmax activation function, and outputting a final recognition result. According to the DAN-GSFF, the precision of 90.11% and the FPS of 96.2 are realized on an SAA13 comprehensive data set, and a feasible method is provided for improving the recognition capability of a network on driver behaviors.

Description

Technical Field

[0001] The present invention belongs to the research field of intelligent transportation and intelligent driving, and specifically relates to a driver behavior recognition method based on a deep attention network with geometric space fusion features. Background Art

[0002] With the rapid development of society and the widespread application of intelligent driving technology, motor vehicles have become the preferred means of transportation for people when traveling. However, at the same time, the number of traffic accidents is also increasing. A large number of traffic accidents are attributed to distracted driving by driver behavior, emphasizing the key role of standardized driving in vehicle safety. Nowadays, the successful deployment of advanced driver assistance systems (ADAS) makes it possible to monitor the driver's status in real time. Therefore, the research on driver behavior recognition (DBR) systems can help reduce the number and severity of traffic accidents caused by distracted driving and ensure driving safety.

[0003] In the driver behavior recognition task, the existing networks are usually trained and verified on a single dataset, so they are overly dependent on and fit to these specific driving environments, thus limiting their generalization ability in complex driving scenarios. In addition, the existing networks are often limited to classification based only on the driver's contextual geometric features, but fail to fully appreciate the importance of backbone spatial features. This limitation makes it difficult for the network to accurately capture the subtle and critical visual differences between different driver behaviors. Therefore, the present invention constructs a comprehensive dataset containing diverse scenes and drivers for training, and proposes a deep attention driver behavior network DAN-GSFF that combines the global information of the image and the driver's posture information as input, while comprehensively considering the driver's contextual geometric features and backbone spatial features for classification. Summary of the invention

[0004] Purpose of the invention: In order to overcome the shortcomings of the prior art, a driver behavior recognition method based on a deep attention network with geometric space fusion features is provided, which comprehensively considers the driver's contextual geometric features and backbone spatial features for classification, and guides the network to selectively focus on the local detail features of the driver in the image by embedding a designed multi-dimensional collaborative attention module in the shallow position of the network, thereby achieving accurate recognition of the driver's behavior in complex driving scenarios.

[0005] Technical solution: To achieve the above purpose, the present invention provides a driver behavior recognition method based on a deep attention network with geometric space fusion features, comprising the following steps:

[0006] S1: Construct a standard image dataset of driver behavior, including the SAA13 comprehensive dataset and the special extreme weak photon dataset;

[0007] S2: Construct a YOLO-pose-based driver backbone spatial feature generation model;

[0008] S3: Research on dual-stream fusion features that combine the contextual geometric features of the image and the backbone spatial features of the driver as input;

[0009] S4: Research and design a multi-source collaborative attention MCA-DBR module specifically for complex driving scenarios;

[0010] S5: The study embeds the MCA-DBR (multidimensional collaborative attention for driver behavior recognition) module into the shallow layer of the network to enhance the network's ability to extract local detailed features of the driver;

[0011] S6: Input the enhanced feature map into the classification network building module to extract deep semantic features, and gradually extract higher-level semantic features;

[0012] S7: Use the softmax activation function to perform feature decoding operations on the deep features extracted by DAN-GSFF, accurately map the features to the corresponding driver behavior categories, and output the final recognition results.

[0013] Furthermore, the specific method of constructing the driver behavior standard image dataset in step S1 is as follows:

[0014] S1-1: The unique scene characteristics of the five datasets State Farm, AUCV2, ASU, 100-Driver and 3MDAD are integrated with the drivers, and this integrated dataset is named SAA13 according to the first letter. The scenes covered by the integrated dataset include different lighting conditions, such as normal lighting during the day or weak light at night; different camera monitoring angles, such as: located on the right or left side of the driver; cluttered and dynamic background environments, such as: static backgrounds or dynamic driving backgrounds. And the number of drivers has reached 272. The diversity of the number of scenes and the number of drivers makes SAA13 more reflective of real scenes. SAA13 covers ten different driving behaviors, namely safe driving C0, sending messages with the left hand C1, making phone calls with the left hand C2, sending messages with the right hand C3, making phone calls with the right hand C4, operating the center console C5, drinking water or eating C6, stretching the body back C7, tidying up hair or makeup C8, talking to passengers C9, and the last nine are distracted driving behaviors that need to be detected;

[0015] S1-2: In order to effectively reduce the repetition and similarity between adjacent images in the data set, an interval sampling strategy is used to perform secondary screening of the image data. The sampling interval is determined based on the stability or dynamics of the background environment, that is, whether the data collection vehicle is stationary or moving. Specifically, when the vehicle is stationary, the global background of the image remains almost unchanged, so a larger sampling interval is used; when the vehicle is moving, the background continues to change, so the sampling interval is correspondingly shortened compared to the stationary state. In addition, in order to increase the proportion of special scene images in the data set, this application also uses a smaller sampling interval. After interval sampling, a total of 59,872 images were collected, involving 272 different drivers. And according to the driver division strategy, the data set is divided into training set, validation set and test set, that is, the data of the training set, test set and validation set should be collected from different driver groups;

[0016] S1-3: We separately collected 4961 images collected under extreme weak light conditions. These images are completely different from those taken under normal lighting conditions in terms of the clarity of the driver's outline, thus constructing a special extreme weak photon dataset.

[0017] Further, the specific method of constructing the driver backbone spatial feature generation model based on YOLO-pose in step S2 is: select YOLO-pose as a posture estimator to accurately detect the key point position of the driver in the image and obtain the key point coordinate information. In view of the fact that in the driving scene, the driver's lower body movements usually have no substantial effect on the recognition task, while the hand movements can often convey key information and play a vital role. Therefore, this application only focuses on detecting six key points of the driver's upper body, which specifically include: {1: left wrist, 2: left elbow, 3: left shoulder, 4: right shoulder, 5: right elbow, 6: right wrist}. After obtaining these key points, the six key points are connected in a specific order to generate and visualize the complete upper body skeleton posture information, that is, 1-2-3 forms the left arm, 4-5-6 forms the right arm, and 3 and 4 are retained to represent the connection of the shoulders. This process not only captures the driver's body posture, but also retains the relative position relationship between the key points, providing a key basis for subsequent behavior analysis.

[0018] Furthermore, in step S3, the dual-stream fusion feature combining the contextual geometric features of the image and the backbone spatial features of the driver as input is studied, and the specific method is as follows: the dual-stream features input into the subsequent network fuse the contextual geometric features of the image and the backbone spatial features of the driver. Among them, the contextual geometric features of the image are derived from the global information of the color image adjusted to a size of 224×224. This information fully covers the image details and constructs a rich scene and background context for the network; while the backbone spatial features of the driver are based on the skeleton framework generated by the posture estimation technology. The framework not only outlines the key points of the driver and their spatial position relationship, but also prompts the network to focus on the local areas directly related to the driver's behavior, and deeply captures the spatial correlation of the body posture, which is crucial for accurately identifying the driver's action mode and intention. The fusion of these two features not only greatly enhances the accuracy and stability of the network in driver behavior analysis, but also significantly improves the importance of the local areas around the key points of the driver, thereby enhancing the network's ability to identify and extract these key features.

[0019] Furthermore, in step S4, a multi-source collaborative attention MCA-DBR module specifically used in complex driving scenarios is studied and designed, and the specific method is as follows: MCA-DBR is a lightweight multi-dimensional collaborative attention module that models width, height and channel dimensions at the same time and has a parallel three-branch structure. The workflow is as follows: the feature map with an input scale of C×H×W is replaced with the height and width dimensions in the channel dimension, and then the feature map is input into the left branch and the middle branch respectively. After that, the same operation is performed with the right branch: first, the input feature F1 is compressed by global average pooling and global standard deviation pooling, and the two compressed features are fused by broadcast element-level summation to aggregate cross-dimensional features. This fused feature is represented as X×1×1, F2; secondly, the fused feature F2 is permuted to 1×1×X, F3, and processed by a two-dimensional convolution operation with a kernel size of 1×K; then, the convolved feature F4 is permuted back to X×1×1, F5; finally, F5 is processed by the sigmoid function and broadcast element-level multiplication operation is performed with the input feature F1. After that, the previously permuted feature map is permuted back to the original dimension, and finally the three feature maps are broadcast element-level averaged and summed, and the refined features are output.

[0020] Furthermore, in step S5, the MCA-DBR module is embedded in the shallow position of the network to enhance the network's ability to extract local detail features of the driver. The specific method is as follows: the attention mechanism is a highly flexible and integrable module, and its embedding position has a wide range of selection space in the network architecture to adapt to different functional and performance requirements. Generally speaking, the shallow layer of the neural network is mainly responsible for capturing the local details of the image, while the deep layer is more inclined to abstract semantically rich information. The local detail information of the driver, such as the local detail contour information of the palm, arm and head, and the global information of local small objects such as water cups and mobile phones, are the most critical feature information of various driver behaviors. Therefore, embedding the designed MCA-DBR module in the shallow position of the network can significantly enhance the network's ability to extract key local detail features, which are crucial for fine-grained driver behavior recognition tasks, and can also effectively suppress the interference of irrelevant features such as background and cockpit environment on the recognition process.

[0021] Furthermore, in step S6, the enhanced feature map is input into the building module of the classification network for deep semantic feature extraction, and higher-level semantic features are gradually extracted. The specific contents are as follows: the feature map enhanced by MCA-DBR is then imported into the subsequent building module to perform more in-depth and detailed feature extraction tasks. In this process, the large-core deep separable convolution technology not only greatly reduces the computational complexity of the model, but also enables the network to capture the global context information in the image more efficiently by increasing the size of the convolution kernel. In addition, the inverted residual structure first expands the number of feature channels, then applies deep convolution for feature extraction, and finally compresses the number of channels with the help of linear layers to achieve deep mining of image features. This design not only improves the network's ability to capture complex features, but also significantly enhances the generalization performance of the network, ensuring that the network can show excellent performance in various complex scenarios. At this stage, with the continuous action of multiple stacked modules, the network gradually extracts higher-level semantic features. These features cover the driver's global profile information, such as the profile features of the head, torso and limbs, laying a solid and accurate foundation for subsequent specific category decoding, enabling DAN-GSFF to more accurately identify information such as the driver's behavior or status.

[0022] Furthermore, in step S7, the softmax activation function is used to perform feature decoding operation on the deep features extracted by DAN-GSFF, accurately map the features to the corresponding driver behavior categories, and output the final recognition results. The specific process is as follows: the deep features extracted by DAN-GSFF are first processed by the global average pooling layer, which effectively reduces the feature dimension by performing an average pooling operation on the feature map in a global range, while retaining global information, avoiding the risk of overfitting, and providing a more compact feature representation for subsequent processing. Next, the layer normalization layer is applied to the pooled features. This layer ensures the stability and consistency of the data between different layers by standardizing the activation value of each sample, which helps to improve the training efficiency and performance of the model. Finally, the Linear layer, as the core part of the classifier, is responsible for mapping the normalized features to the predefined driver behavior category space. This layer realizes the linear transformation from the feature space to the category space through the weight matrix and the bias vector, providing a basis for the final classification decision. After the Linear layer, the softmax activation function is used to calculate the probability distribution of each category, achieving accurate decoding of deep features, accurately mapping the features to the corresponding driver behavior categories, and finally outputting the recognition results to complete the accurate judgment of the driver's behavior.

[0023] Beneficial effects: Compared with the prior art, the present invention realizes automatic recognition of driver behavior based on the deep attention network with geometric space fusion features, realizes accurate recognition of driver behavior in complex driving scenarios, and improves the generalization ability and robustness of the network in complex driving scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the DAN-GSFF network structure.

[0025] Figure 2 This is an image example from the SAA13 dataset.

[0026] Figure 3 The performance comparison chart of DAN-GSFF and other literature models.

[0027] Figure 4 Box plot comparison of DAM-GSFF and the other five models.

[0028] Figure 5 The following is a graph showing the prediction confusion matrix results of DAM-GSFF and the other five models on the SAA13 dataset. DETAILED DESCRIPTION

[0029] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments.

[0030] like Figure 1 As shown, the present invention provides a driver behavior recognition method based on a geometric space fusion feature deep attention network, comprising the following steps:

[0031] S1: Construct a standard image dataset of driver behavior, which is as follows:

[0032] S1-1: The unique scene characteristics of the five datasets State Farm, AUCV2, ASU, 100-Driver and 3MDAD are integrated with the drivers, and this integrated dataset is named SAA13 according to the first letter. The scenes covered by the integrated dataset include different lighting conditions, such as normal lighting during the day or weak light at night; different camera monitoring angles, such as: located on the right or left side of the driver; cluttered and dynamic background environments, such as: static backgrounds or dynamic driving backgrounds. And the number of drivers has reached 272. The diversity of the number of scenes and the number of drivers makes SAA13 more reflective of real scenes. SAA13 covers ten different driving behaviors, namely safe driving C0, sending messages with the left hand C1, making phone calls with the left hand C2, sending messages with the right hand C3, making phone calls with the right hand C4, operating the center console C5, drinking water or eating C6, stretching the body back C7, tidying up hair or makeup C8, talking to passengers C9, and the last nine are distracted driving behaviors that need to be detected;

[0033] S1-2: In order to effectively reduce the repetition and similarity between adjacent images in the data set, an interval sampling strategy is used to perform secondary screening of the image data. The sampling interval is determined based on the stability or dynamics of the background environment, that is, whether the data collection vehicle is in a stationary or driving state. Specifically, when the vehicle is stationary, the global background of the image remains almost unchanged, so a larger sampling interval is used; when the vehicle is driving, the background continues to change, so the sampling interval is correspondingly shortened compared to the stationary state. In addition, in order to increase the proportion of special scene images in the data set, this application also uses a smaller sampling interval. After interval sampling, a total of 59,872 images were collected, involving 272 different drivers. And according to the driver division strategy, the data set is divided into training set, validation set and test set, that is, the data of the training set, test set and validation set should be collected from different driver groups. The specific division information is shown in Figure 1, and the pattern example is as follows Figure 2 As shown;

[0034] Table 1 Detailed information of SAA13 dataset

[0035]

[0036] S1-3: We separately collected 4961 images collected under extreme weak light conditions. These images are completely different from those taken under normal lighting conditions in terms of the clarity of the driver's outline, thus constructing a special extreme weak photon dataset.

[0037] S2: The specific method of constructing a driver backbone spatial feature generation model based on YOLO-pose is: select YOLO-pose as a posture estimator to accurately detect the key point position of the driver in the image and obtain the key point coordinate information. In view of the fact that in the driving scene, the driver's lower body movements usually have no substantial effect on the recognition task, while the hand movements can often convey key information and play a vital role. Therefore, this application only focuses on detecting the six key points of the driver's upper body, which specifically include: {1: left wrist, 2: left elbow, 3: left shoulder, 4: right shoulder, 5: right elbow, 6: right wrist}. After obtaining these key points, the six key points are connected in a specific order to generate and visualize the complete upper body skeleton posture information, that is, 1-2-3 forms the left arm, 4-5-6 forms the right arm, and 3 and 4 are retained to represent the connection of the shoulders. This process not only captures the driver's body posture, but also retains the relative position relationship between the key points, providing a key basis for subsequent behavior analysis.

[0038] S3: The dual-stream fusion feature that combines the contextual geometric features of the image and the backbone spatial features of the driver as input is studied. The specific method is as follows: the dual-stream features input into the subsequent network fuse the contextual geometric features of the image and the backbone spatial features of the driver. Among them, the contextual geometric features of the image are derived from the global information of the color image adjusted to 224×224 size. This information fully covers the image details and constructs a rich scene and background context for the network; while the backbone spatial features of the driver are based on the skeleton framework generated by the posture estimation technology. This framework not only outlines the key points of the driver and their spatial position relationship, but also prompts the network to focus on the local areas directly related to the driver's behavior and deeply capture the spatial correlation of body posture, which is crucial for accurately identifying the driver's action pattern and intention. The fusion of these two features not only greatly enhances the accuracy and stability of the network in driver behavior analysis, but also significantly improves the importance of the local areas around the key points of the driver, thereby enhancing the network's ability to identify and extract these key features.

[0039] S4: Study and design a multi-source collaborative attention MCA-DBR module specifically for complex driving scenarios. The specific method is as follows: MCA-DBR is a lightweight multi-dimensional collaborative attention module that models width, height and channel dimensions at the same time and has a parallel three-branch structure. Its workflow is as follows: the feature map of input scale C×H×W is permuted with the height and width dimensions in the channel dimension, and then the feature map is input to the left branch and the middle branch respectively. After that, the same operation is performed with the right branch: first, the input feature F1 is compressed by global average pooling and global standard deviation pooling, and the two compressed features are fused by broadcast element-wise summation to aggregate cross-dimensional features. This fused feature is represented as X×1×1, F2; secondly, the fused feature F2 is permuted to 1×1×X, F3 and processed by a 2D convolution operation with a kernel size of 1×K; then, the convolved feature F4 is permuted back to X×1×1, F5; finally, F5 is processed by the sigmoid function and broadcast element-wise multiplication operation is performed with the input feature F1. After that, the previously permuted feature maps are permuted back to their original dimensions, and finally the three feature maps are broadcast element-wise averaged and summed, and the refined features are output.

[0040] S5: The MCA-DBR module is embedded in the shallow position of the network to enhance the network's ability to extract local details of the driver. The specific method is as follows: As a highly flexible and integrable module, the attention mechanism has a wide range of choices in the network architecture to adapt to different functional and performance requirements. Generally speaking, the shallow layer of the neural network is mainly responsible for capturing local details of the image, while the deep layer is more inclined to abstract semantically rich information. The local details of the driver, such as the local details of the palm, arm and head, and the global information of small local objects such as water cups and mobile phones, are the most critical feature information of various driver behaviors. Therefore, embedding the designed MCA-DBR module in the shallow position of the network can significantly enhance the network's ability to extract key local details, which are crucial for fine-grained driver behavior recognition tasks, and can also effectively suppress the interference of irrelevant features such as background and cockpit environment on the recognition process.

[0041] S6: The enhanced feature map is input into the building module of the classification network for deep semantic feature extraction, and higher-level semantic features are gradually extracted. The specific content is as follows: The feature map enhanced by MCA-DBR is then imported into the subsequent building module to perform more in-depth and detailed feature extraction tasks. In this process, the large-core deep separable convolution technology not only greatly reduces the computational complexity of the model, but also enables the network to capture the global context information in the image more efficiently by increasing the size of the convolution kernel. In addition, the inverted residual structure first expands the number of feature channels, then applies deep convolution for feature extraction, and finally compresses the number of channels with the help of linear layers to achieve deep mining of image features. This design not only improves the network's ability to capture complex features, but also significantly enhances the generalization performance of the network, ensuring that the network can show excellent performance in various complex scenarios. At this stage, with the continuous action of multiple stacked modules, the network gradually extracts higher-level semantic features. These features cover the driver's global profile information, such as the profile features of the head, torso and limbs, laying a solid and accurate foundation for subsequent specific category decoding, enabling DAN-GSFF to more accurately identify information such as the driver's behavior or status.

[0042] S7: Use the softmax activation function to perform feature decoding operations on the deep features extracted by DAN-GSFF, accurately map the features to the corresponding driver behavior categories, and output the final recognition results. The specific process is as follows: The deep features extracted by DAN-GSFF are first processed by the global average pooling layer, which effectively reduces the feature dimension by performing an average pooling operation on the feature map in a global range, while retaining global information, avoiding the risk of overfitting, and providing a more compact feature representation for subsequent processing. Next, the layer normalization layer is applied to the pooled features. This layer ensures the stability and consistency of data between different layers by standardizing the activation value of each sample, which helps to improve the training efficiency and performance of the model. Finally, the Linear layer, as the core part of the classifier, is responsible for mapping the normalized features to the predefined driver behavior category space. This layer realizes the linear transformation from feature space to category space through the weight matrix and bias vector, providing a basis for the final classification decision. After the Linear layer, the softmax activation function is used to calculate the probability distribution of each category, achieving accurate decoding of deep features, accurately mapping the features to the corresponding driver behavior categories, and finally outputting the recognition results to complete the accurate judgment of the driver's behavior.

[0043] In order to verify the recognition effect of each component of DAN-GSFF on driver behavior, ablation experiments were carried out. The experimental results are shown in Table 2, where Atten. represents the embedded MCA-DBR and Pose represents the fusion of the skeleton space features of the driver. In the SAA13 dataset, the basic network ConvNeXt achieved an accuracy of 82.99% and an F1-score of 83.52%. This experimental result shows that ConvNeXt can sensitively capture the driver's detailed information with its powerful feature extraction ability when facing complex and changeable driving scenes, thereby achieving accurate recognition of the driver's behavior. On the basis of the basic network, after embedding the designed multi-dimensional collaborative attention module MCA-DBR, the network's accuracy increased by 2.22%, the accuracy increased by 2.9%, the recall rate increased by 2.1% and the F1-score increased by 2.49%. These increases show that the embedding of visual attention can effectively enhance the network's ability to extract key features, such as driver profile, posture, hand-held items and other detailed features, while weakening the attention to irrelevant areas, such as complex global background features. In addition, the driver's backbone spatial features and the contextual geometric features of the image were integrated into the baseline model, and the network's accuracy, precision, recall and F1-score were improved by 5.36%, 5.62%, 5.35% and 5.48% respectively. This improvement in experimental results shows the effectiveness of the backbone spatial features in the driver behavior recognition task, and proves that the backbone spatial features can guide the network to focus more on extracting local area information closely related to the driver's behavior, thereby effectively improving the network's recognition performance of the driver's behavior. Finally, after integrating the driver's backbone spatial features and the contextual geometric features of the image as input and inputting them into the classification network strengthened by MCA-DBR, the network's recognition performance of the driver's behavior reached the optimal level, for example, the accuracy reached 90.11%, the accuracy reached 90.75%, the recall reached 90.23%, and the F1-score reached 90.49%.

[0044] Table 2 Overall ablation experiment results

[0045]

[0046] In order to verify the effectiveness of the method proposed in this invention, a comparative experiment with other literature methods was carried out. The experimental results are shown in Table 3 and Figure 3As shown. The proposed DAN-GSFF is a deep attention network that fuses image contextual geometric features and driver backbone spatial features as input. It first uses a posture estimator to detect the location of the driver's key points and generate the driver's backbone spatial features, and combines them with the image's contextual geometric features as input. In addition, this application adds a multi-dimensional collaborative attention module suitable for complex driving scenarios at the shallow position of the classification network to enhance the network's feature extraction ability for the driver's geometric information. In the end, DAN-GSFF achieved the best recognition performance on SAA13. Its precision, accuracy, recall rate and F1-score all exceeded 90%, reaching 90.11%, 90.75%, 90.23% and 90.49% respectively, and achieved an FPS of 96.2. Compared with other driver behavior recognition networks, it achieved the best performance in the trade-off between recognition accuracy and inference speed. In addition, it also achieved a recognition accuracy of 86.85% on a special sub-dataset under extreme low-light conditions, second only to BiRSWinT, but its parameter volume and FPS are far superior to BiRSWinT. Therefore, DAN-GSFF has more comprehensive generalization capabilities and robustness in complex driving scenarios.

[0047] Table 3 Comparative experimental results with other literature methods

[0048]

[0049]

[0050] In addition to the tabular results shown in Table 3, a variety of visualization methods are used to compare the recognition performance of the DAN-GSFF proposed in this application with that of five other well-performing networks, namely ResNet50, ResNeXt50, HCF, OLCMNet, and BiRSWinT. First, 10 cross-validation experiments were conducted, and 1,000 images were randomly selected from the test set for testing in each experiment. The recognition accuracy of the ten cross-validation results was plotted as a box plot, as shown in Figure 3. Figure 4 As shown in the figure, the DAN-GSFF of this application has the lowest lower quartile, which guarantees the recognition performance of DAN-GSFF in complex driving scenes, and the box change of DAN-GSFF is relatively small, which shows that DAN-GSFF has strong robustness in complex driving scenes. In addition, in order to better illustrate the recognition accuracy results of these six models, Figure 5The driving with confusion matrix gives more details. It can be seen that only the two networks DAN-GSFF and BiRSWinT have a recognition accuracy of more than 80% for talking with passengers (C9), and most of the classification errors are misclassified as safe driving (C0). This is because the difference between these two types of driving posture behaviors in some scenes is only reflected in the slight head posture, and the other features are basically similar. In addition, for the DAN-GSFF proposed in this application, the recognition accuracy of four types of driver behaviors exceeds 90%, among which the body stretching back (C7) is as high as 97.35%, indicating that the posture characteristics of this type of driver behavior are significantly different from other driving behaviors. However, the recognition accuracy of DAN-GSFF for the right-hand phone (C2) is only 78.54%, and most of them are misclassified as drinking water or eating (C6) and tidying hair or makeup (C8). This result reflects that when DAN-GSFF faces the action of the driver raising his right hand, it is difficult to identify whether the object in the right hand is a mobile phone, a water cup or nothing, indicating that DAN-GSFF has a weak ability to extract features of specific objects.

[0051] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A driver behavior recognition method based on a deep attention network with geometric spatial fusion features, characterized by: The steps include: S1: Construct a standard image dataset of driver behavior, including the SAA13 comprehensive dataset and the special extreme weak photon dataset; S2: Construct a YOLO-pose-based driver backbone spatial feature generation model; S3: Research on dual-stream fusion features that combine the contextual geometric features of the image and the backbone spatial features of the driver as input; S4: Research and design a multi-source collaborative attention MCA-DBR module specifically for complex driving scenarios; S5: Research on embedding the MCA-DBR module into the shallow layer of the network to enhance the network's ability to extract local detailed features of the driver; S6: Input the enhanced feature map into the classification network building module to extract deep semantic features, and gradually extract higher-level semantic features; S7: Use the softmax activation function to perform feature decoding operations on the deep features extracted by DAN-GSFF, accurately map the features to the corresponding driver behavior categories, and output the final recognition results.

2. The driver behavior recognition method based on the geometric space fusion feature deep attention network according to claim 1 is characterized by: The specific method of step S1 is as follows: S1-1: The unique scene characteristics of the five datasets State Farm, AUCV2, ASU, 100-Driver and 3MDAD are integrated with the driver, and this integrated dataset is named SAA13 according to the first letter; the scenes covered by the integrated dataset include different care conditions, different camera monitoring angles and cluttered dynamic background environments; SAA13 covers ten different driving behaviors, namely safe driving C0, sending messages with the left hand C1, making phone calls with the left hand C2, sending messages with the right hand C3, making phone calls with the right hand C4, operating the center console C5, drinking water or eating C6, stretching the body backwards C7, tidying hair or putting on makeup C8, and talking to passengers C9. The last nine are distracted driving behaviors that need to be detected; S1-2: In order to effectively reduce the repetition and similarity between adjacent images in the data set, the interval sampling strategy is used to perform secondary screening of the image data; the sampling interval is determined based on the stability or dynamics of the background environment, that is, whether the data collection vehicle is stationary or driving; specifically, when the vehicle is stationary, the global background of the image remains almost unchanged, so a larger sampling interval is used; when the vehicle is driving, the background continues to change, so the sampling interval will be correspondingly shortened compared to the stationary state; in order to increase the proportion of special scene images in the data set, a smaller sampling interval is used; after interval sampling, the data set is divided into training set, validation set and test set according to the driver division strategy, that is, the data of the training set, test set and validation set should be collected from different driver groups; S1-3: Separately collect images collected under extreme weak light conditions. These images are completely different from images taken under normal lighting conditions in terms of driver outline clarity, thereby constructing a special extreme weak photon dataset.

3. The driver behavior recognition method based on the geometric space fusion feature deep attention network according to claim 1 is characterized by: The specific method of step S2 is: using YOLO-pose as a posture estimator to accurately detect the key point positions of the driver in the image and obtain the key point coordinate information; considering that in the driving scene, the driver's lower body movements usually have no substantial effect on the recognition task, while the hand movements can often convey key information and play a vital role; Therefore, we only focus on detecting six key points of the driver's upper body, which include: {1: left wrist, 2: left elbow, 3: left shoulder, 4: right shoulder, 5: right elbow, 6: right wrist}; after obtaining these key points, we connect them in a specific order to generate and visualize the complete upper body skeleton posture information, that is, 1-2-3 forms the left arm, 4-5-6 forms the right arm, and 3 and 4 are retained to represent the connection of the two shoulders; this process not only captures the driver's body posture, but also retains the relative position relationship between the key points, providing a key basis for subsequent behavior analysis.

4. The driver behavior recognition method based on the geometric space fusion feature deep attention network according to claim 1 is characterized by: The specific method of step S3 is as follows: the dual-stream features input into the subsequent network fuse the contextual geometric features of the image and the backbone spatial features of the driver; wherein the contextual geometric features of the image are derived from the global information of the color image adjusted to a size of 224×224, which comprehensively covers the image details and constructs a rich scene and background context for the network; and the backbone spatial features of the driver are based on a skeleton framework generated by posture estimation technology, which not only outlines the key points of the driver and their spatial positional relationships, but also prompts the network to focus on local areas directly related to the driver's behavior, and deeply captures the spatial correlation of body posture, which is crucial for accurately identifying the driver's action patterns and intentions; the fusion of the two features not only greatly enhances the accuracy and stability of the network in driver behavior analysis, but also significantly improves the importance of local areas around the driver's key points, thereby enhancing the network's ability to recognize and extract these key features.

5. The driver behavior recognition method based on the geometric space fusion feature deep attention network according to claim 1 is characterized by: The specific method of step S4 is as follows: MCA-DBR is a lightweight multi-dimensional collaborative attention module, which models the width, height and channel dimensions at the same time and has a parallel three-branch structure; its workflow is as follows: the feature map with an input scale of C×H×W is replaced with the height and width dimensions in the channel dimension, and then the feature map is input to the left branch and the middle branch respectively; then, the same operation is performed with the right branch: first, the input feature F1 is compressed by global average pooling and global standard deviation pooling, and the two are fused by broadcast element-level summation. The first step is to extract the compressed features to aggregate cross-dimensional features, and this fused feature is represented as X×1×1, F2; secondly, the fused feature F2 is permuted to 1×1×X, F3, and processed by a two-dimensional convolution operation with a kernel size of 1×K; then, the convolved feature F4 is permuted back to X×1×1, F5; finally, F5 is processed by the sigmoid function and broadcast element-level multiplication operation is performed with the input feature F1; after this, the previously permuted feature map is permuted back to the original dimension, and finally the three feature maps are broadcast element-level averaged and summed, and the refined features are output.

6. The driver behavior recognition method based on the geometric space fusion feature deep attention network according to claim 1 is characterized by: The specific method of step S5 is as follows: the attention mechanism is a highly flexible and integrable module, and its embedding position has a wide selection space in the network architecture to adapt to different functional and performance requirements; the shallow layer of the neural network is mainly responsible for capturing the local details of the image, while the deep layer is more inclined to abstract semantically rich information; the local detail information of the driver is the most critical feature information of various driver behaviors; therefore, embedding the designed MCA-DBR module into the shallow position of the network can significantly enhance the network's ability to extract key local detail features, which are crucial for fine-grained driver behavior recognition tasks, and can also effectively suppress the interference of irrelevant features such as background and cockpit environment on the recognition process.

7. The driver behavior recognition method based on the geometric space fusion feature deep attention network according to claim 1 is characterized by: The specific content of step S6 is as follows: the feature map after MCA-DBR enhancement is then imported into the subsequent building module to perform a more in-depth and detailed feature extraction task; in this process, the large-kernel deep separable convolution technology not only greatly reduces the computational complexity of the model, but also enables the network to more efficiently capture the global context information in the image by increasing the size of the convolution kernel; In addition, the inverted residual structure achieves deep mining of image features by first expanding the number of feature channels, then applying deep convolution for feature extraction, and finally compressing the number of channels with the help of linear layers. This design not only improves the network's ability to capture complex features, but also significantly enhances the network's generalization performance, ensuring that the network can perform well in various complex scenarios. At this stage, with the continuous action of multiple stacked modules, the network gradually extracts higher-level semantic features. These features cover the driver's global profile information, laying a solid and precise foundation for subsequent specific category decoding, enabling DAN-GSFF to more accurately identify the driver's behavior or status information.

8. The driver behavior recognition method based on the geometric space fusion feature deep attention network according to claim 1 is characterized by: The specific process of step S7 is as follows: the deep features extracted by DAN-GSFF are first processed by the global average pooling layer, which effectively reduces the feature dimension by performing an average pooling operation on the feature map in a global range, while retaining global information, avoiding the risk of overfitting, and providing a more compact feature representation for subsequent processing; then, the layer normalization layer is applied to the pooled features, which ensures the stability and consistency of data between different layers by standardizing the activation value of each sample, and helps to improve the training efficiency and performance of the model; finally, the Linear layer, as the core part of the classifier, is responsible for mapping the normalized features to the predefined driver behavior category space. This layer realizes the linear transformation from the feature space to the category space through the action of the weight matrix and the bias vector, and provides a basis for the final classification decision; after the Linear layer, the softmax activation function is used to calculate the probability distribution of each category, so as to realize the accurate decoding of the deep features, accurately map the features to the corresponding driver behavior categories, and finally output the recognition results, completing the accurate judgment of the driver behavior.

Images