A driver distraction detection method based on quality-aware class activation mapping constraint

Abstract

The application provides a driver distraction detection method based on quality-aware class activation mapping constraint. Image samples with driver behavior class labels are obtained, features are extracted by a distraction detection model, and class activation mappings of each class are generated, the mapping corresponding to the true class is selected, the spatial focus score and the class discrimination score are calculated, and the sample-level quality score is obtained by adaptively fusing the class discrimination information in the training batch according to the maturity of the class discrimination information; the same sample quality score is used to construct a sample pair constraint weight, and quality-weighted semantic alignment is performed; and the target model is trained by combining the classification loss, the class activation mapping compactness constraint, the inter-class orthogonality constraint and the quality-weighted semantic alignment constraint. In the detection stage, the image of the driver to be detected is input, and the distraction behavior class result is output, without increasing the additional network structure and the calculation cost, so that the detection accuracy, stability and generalization ability can be improved.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent driving, computer vision and driver state monitoring technology, and particularly relates to a driver distraction detection method based on quality perception class activation mapping constraints. Background Technology

[0002] Distracted driving is a significant contributing factor to road traffic accidents. When drivers are engaged in activities such as making or receiving phone calls, sending text messages, drinking water, adjusting in-vehicle equipment, conversing with passengers, or turning to retrieve items while driving, their perception and reaction abilities to the road environment are easily reduced, thus increasing the risk of traffic accidents. Therefore, accurately, promptly, and reliably identifying driver distraction is a crucial technical challenge for intelligent driver assistance systems, in-vehicle active safety systems, and driver status monitoring systems.

[0003] Existing methods for driver distraction detection typically include those based on physiological signals, those based on vehicle state signals, and those based on visual images. Among these, visual image-based methods utilize onboard cameras to capture images of the driver and identify driver behavior categories through image classification models. These methods offer advantages such as being non-contact, easy to deploy, and highly compatible with existing onboard camera equipment, and are therefore widely used in driver distraction detection tasks.

[0004] With the development of deep learning technology, convolutional neural networks, lightweight neural networks, and visual attention models have been applied to driver distraction behavior recognition. For in-vehicle scenarios, driver distraction detection methods typically need to balance recognition accuracy, real-time detection, and computational efficiency; therefore, lightweight models have high practical value. Lightweight models can complete driver image feature extraction and behavior category prediction with lower parameter counts and computational complexity, which helps meet the real-time and computational efficiency requirements of driver distraction detection tasks.

[0005] However, in actual driver distraction detection, there are often significant fine-grained differences between different distracting behaviors. For example, the differences between actions such as making or receiving a phone call, drinking water, adjusting in-vehicle equipment, looking down to operate, or turning to retrieve an object are usually not reflected in significant overall posture changes, but rather in subtle differences in hand position, head orientation, local limb movements, and interactive objects such as mobile phones, water cups, steering wheels, and in-vehicle equipment. Furthermore, the real in-vehicle environment also presents factors such as changes in lighting, occlusion, motion blur, individual driver differences, background interference, and changes in shooting perspective. For lightweight networks with limited model capacity, when relying solely on global image representation for classification, the model is easily affected by irrelevant background areas and local noise, making it difficult to stably focus on truly discriminative key local regions, resulting in insufficient accuracy, stability, and generalization ability in driver distraction detection.

[0006] To enhance the model's ability to model key local regions, existing technologies include driver distraction detection methods based on keypoint detection, hand localization, object detection, or multi-region cropping. These methods typically use additional detection modules to explicitly locate the driver's head, hands, or interacting objects, and then combine local region features for behavior recognition. While these methods can improve fine-grained behavior recognition capabilities to some extent, they usually rely on additional annotations, upstream detectors, or multi-stage processing flows. If the upstream region localization results are offset or erroneous, subsequent distraction detection results are easily affected. Furthermore, multi-module, multi-branch, or multi-stage structures increase the computational overhead and system complexity of the detection stage, which is detrimental to meeting the real-time and computational efficiency requirements of driver distraction detection tasks.

[0007] Class activation mapping (NAM) can utilize the feature maps and classification layer weights of a classification network to generate spatial response maps associated with different categories, thus reflecting the image regions that the model focuses on when making category judgments. Unlike methods that rely on additional local annotations or external detectors, NAM-based constraint methods can use the class response maps as auxiliary supervision signals during the training phase, guiding the model to learn more discriminative local regions, while maintaining the original network structure during the detection phase. Therefore, NAM constraints provide a feasible approach to improve the detection accuracy and deployment efficiency of lightweight driver distraction detection methods.

[0008] However, existing driver distraction detection methods based on class activation map constraints typically assume that class activation maps generated from different training samples have similar reliability and impose the same level of constraint on all samples or sample pairs. This assumption is often difficult to hold in real-world driving scenarios. Influenced by occlusion, low illumination, motion blur, background interference, changes in driver posture, and subtle differences between categories, class activation maps corresponding to different samples may exhibit significant differences in spatial focus, category discrimination ability, and semantic reliability. For high-quality samples, class activation maps can usually focus accurately on key regions related to the current driving behavior; however, for low-quality or difficult samples, class activation maps may exhibit problems such as scattered responses, shifted regions of interest, or confusion with responses from other categories.

[0009] If the aforementioned differences in the quality of sample-level class activation maps are ignored during the construction of the driver distraction detection model, erroneous responses contained in low-quality class activation maps may be continuously reinforced as valid supervisory signals. Especially when semantic consistency or semantic alignment constraints are applied between samples of the same class, the erroneous response patterns of low-quality samples may propagate to other samples through cross-sample constraints, thereby weakening the model's ability to learn key local regions and affecting the accuracy, stability, and generalization ability of the final driver distraction detection.

[0010] Therefore, existing technologies still need a driver distraction detection method that can evaluate the reliability of activation mappings for different sample classes during model training without increasing the network structure and computational overhead in the detection phase, and dynamically adjust the cross-sample semantic constraint strength based on sample-level quality. This would improve the model's ability to model key discrimination regions such as the driver's hands, head, interactive objects, and local action areas, thereby enhancing the accuracy, stability, and generalization ability of driver distraction detection in complex in-vehicle scenarios. Summary of the Invention

[0011] This invention provides a driver distraction detection method based on quality-perception class activation mapping constraints.

[0012] The method includes: acquiring driver image samples with driver behavior category labels; inputting the driver image samples into a backbone network, extracting corresponding feature maps, and outputting driver behavior category prediction results through a classifier; generating class activation maps corresponding to each driver behavior category based on the feature maps and the classification layer weights of the classifier; selecting class activation maps corresponding to the real category from the class activation maps corresponding to each driver behavior category according to the real category labels of the driver image samples; calculating the spatial focusing score and category discrimination score of the sample attention map based on the class activation maps corresponding to the real category; and calculating the category discrimination information in the current training batch. The maturity level is assessed by adaptively fusing the spatial focusing score and the category discrimination score to obtain a sample-level quality score. Based on the sample-level quality scores among samples of the same category, sample pair constraint weights are constructed, and quality-weighted semantic alignment of similar samples is performed. The driver distraction detection model is trained by combining classification loss, class activation mapping compactness constraint loss, inter-class orthogonality constraint loss, and quality-weighted semantic alignment loss of similar samples to obtain a target driver distraction detection model. After training, an image of the driver to be detected is acquired, and the image is input into the target driver distraction detection model to output the driver distraction behavior category detection result.

[0013] The lightweight backbone network may include MobileNet, ShuffleNet, EfficientNet, or other lightweight convolutional neural networks, or it may include a lightweight visual neural network. The classifier may include a global average pooling layer and a linear classification layer. The class activation map may be generated based on the feature map and the weights of the linear classification layer, and is used to characterize the spatial response region of the model for different driver behavior categories.

[0014] The spatial focus score is used to characterize whether the class activation mapping corresponding to the true category responds concentratedly to a few key spatial locations. For the driver distraction detection task, these key spatial locations can correspond to the driver's hands, head, facial orientation, local limb movement areas, or local areas between the driver and interactive objects such as mobile phones, water cups, steering wheels, and in-vehicle devices. By calculating the spatial focus score, it can be determined whether the attention map of the current sample has good local focus characteristics.

[0015] The category discrimination score is used to characterize the discriminative advantage of the class activation map corresponding to the true class relative to the class activation maps of other classes. By comparing the difference between the true class response and the competing class response, it can be determined whether the attention map of the current sample has strong class specificity, thereby avoiding attention maps that are only spatially concentrated but have confused classes from being mistaken for high-quality attention maps.

[0016] The sample-level quality score does not use a fixed weight to fuse spatial focusing score and class discrimination score. Instead, it dynamically adjusts the fusion ratio based on the maturity of class discrimination information in the current training batch. In the early stages of training, when the model has not yet formed stable class boundaries, the class discrimination score may fluctuate significantly. In this case, the influence of the class discrimination score in the quality score is reduced. As training progresses and the class discrimination information matures, the fusion weight of the class discrimination score is increased. Therefore, the sample-level quality score can better adapt to the dynamic changes during model training.

[0017] When performing semantic alignment of samples of the same class, the constraint weights of the sample pairs are determined based on the sample-level quality scores of the two samples in the pair participating in the semantic alignment. When the quality of the attention map of either sample in a pair of samples of the same class is low, the semantic alignment constraint strength of that pair is reduced; when both samples in a pair of samples of the same class have high quality scores, the semantic alignment constraint strength of that pair is increased. This approach enhances the supervisory role of high-quality samples in model training and suppresses the propagation of cross-sample noise caused by low-quality attention maps.

[0018] The training loss of the driver distraction detection model includes classification loss, class activation map compactness constraint loss, inter-class orthogonality constraint loss, and quality-weighted semantic alignment loss for similar samples. Specifically, the classification loss ensures the model's basic classification ability; the class activation map compactness constraint loss promotes the concentration of class activation maps in a more compact spatial region; the inter-class orthogonality constraint loss reduces overlap and confusion between class activation maps of different categories; and the quality-weighted semantic alignment loss for similar samples dynamically adjusts the semantic alignment strength between similar samples based on the sample-level quality score.

[0019] The quality-aware class activation mapping constraint is used only during the training phase. During the inference phase, the auxiliary constraint branches used for quality assessment and semantic alignment are removed, retaining only the trained lightweight backbone network and classifier for predicting the behavior category of the input driver image. Therefore, this invention improves the recognition performance and stability of the lightweight driver distraction detection model without increasing the network structure or computational load during the inference phase.

[0020] Compared with the prior art, the present invention has at least the following beneficial effects:

[0021] First, this invention introduces a sample attention map reliability evaluation mechanism to measure the quality of class activation mapping from two dimensions: spatial focus and class discrimination ability. This can distinguish between high-quality and low-quality attention maps in training samples, and avoid treating all sample attention maps as equally reliable.

[0022] Second, this invention uses an adaptive sample quality scoring mechanism to dynamically adjust the fusion ratio of spatial focusing score and category discrimination score based on the maturity of category discrimination information in the current training batch. This enables sample quality estimation to adapt to different features in the early and late stages of model training, reducing the quality estimation bias caused by fixed fusion weights.

[0023] Third, this invention enables high-quality samples to play a stronger supervisory role in cross-sample semantic constraints through a quality-weighted semantic alignment mechanism for similar samples, while reducing the interference of low-quality samples on the optimization direction, thereby suppressing the propagation of noise from low-quality attention maps among similar samples.

[0024] Fourth, the quality-aware activation mapping constraint of the present invention only applies to the training phase. No additional detectors, local region annotations, key point models or multi-branch structures need to be introduced in the inference phase. It can maintain the original inference structure and computational complexity of the lightweight model and is suitable for driver state monitoring scenarios with high requirements for real-time performance and computational complexity.

[0025] Fifth, this invention can enhance the modeling ability of the lightweight driver distraction detection model for key discrimination regions such as hand position, head orientation, interactive objects, and local action areas, and improve the training stability and recognition reliability of the model in complex scenarios such as changes in lighting, occlusion, background interference, and individual differences among drivers. Attached Figure Description

[0026] The accompanying drawings are used to illustrate the technical process of the present invention, wherein:

[0027] Figure 1 This is a schematic diagram of the overall process of a driver distraction detection method based on quality-perception class activation mapping constraints according to an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of the training phase class activation mapping constraint branch and detection phase processing procedure according to one embodiment of the present invention. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. For ease of description, only the parts related to the present invention are shown in the drawings. Unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0030] Existing lightweight driver distraction detection methods typically use the driver image as the overall input to a classification network, predicting driver behavior categories through global features. While these methods offer advantages such as simple structure, fast inference speed, and ease of application, in real-world driving scenarios, different distraction behaviors often manifest only as subtle differences in hand position, head orientation, interactive objects, or localized limb movements. The models are easily affected by in-vehicle background, lighting changes, occlusion, motion blur, and individual driver differences, making it difficult to consistently focus on truly discriminative key local regions.

[0031] Existing training constraint methods based on class activation maps can guide models to focus on key local regions to some extent, but they typically assume that class activation maps generated from different training samples have similar reliability and impose constraints of uniform strength on all samples or sample pairs. However, in complex driving scenarios, class activation maps corresponding to different samples may differ significantly in terms of spatial focus, class discriminativeness, and semantic reliability. If these differences are ignored, erroneous responses in low-quality class activation maps may continue to propagate during cross-sample semantic alignment, thereby affecting the model's training stability and the learning effect of key local regions.

[0032] To address this, this invention proposes a driver distraction detection method based on quality-perceived class activation map constraints. The method first acquires driver image samples labeled with driver behavior categories; then, it inputs these driver image samples into a lightweight backbone network to extract feature maps, and outputs driver behavior category prediction results through a classifier; further, it generates class activation maps corresponding to each category based on the feature maps and classifier weights; during the training phase, it selects the class activation maps corresponding to the real categories, evaluates the reliability of sample attention maps from two dimensions: spatial focus and category discrimination ability, and adaptively fuses them according to the maturity of category discrimination information in the current training batch to obtain sample-level quality scores; subsequently, it constructs sample pair constraint weights based on the quality scores between samples of the same category, and performs quality-weighted semantic alignment of samples of the same category; finally, it trains the driver distraction detection model by combining classification loss, class activation map constraint loss, and quality-weighted semantic alignment loss.

[0033] This invention enhances the guiding role of high-quality attention maps in model training, suppresses noise propagation from low-quality attention maps in cross-sample constraints, and improves the lightweight model's ability to model key discrimination regions such as driver's hands, head, interactive objects, and local action areas. Furthermore, the quality-aware class activation mapping constraint of this invention only applies to the training phase; the inference phase retains only the trained lightweight backbone network and classifier, without introducing additional detectors, keypoint localization modules, multi-branch networks, or additional inference computation overhead. This makes it suitable for driver state monitoring scenarios with high requirements for real-time detection and computational efficiency.

[0034] Figure 1 This diagram illustrates the overall flow of a driver distraction detection method based on quality-perceived class activation mapping constraints according to one embodiment of the present invention. Figure 1 The method shown includes steps S110 to S170. Specifically, the method includes a training phase and a detection phase. The training phase is used to obtain a target driver distraction detection model, and the detection phase is used to output driver distraction behavior category detection results based on the driver image to be detected.

[0035] Figure 2 This illustrates a training phase class activation mapping constraint branch and detection phase processing procedure according to an embodiment of the present invention. Figure 2 As shown, during the training phase, driver image samples are processed by a lightweight backbone network, global average pooling, and a classifier to obtain driver behavior category predictions, forming a classification loss. Simultaneously, class activation maps for each category are generated based on feature maps and the classification layer weights of the classifier. After selecting the normalized class activation map corresponding to the true category, sample attention map reliability evaluation, adaptive quality scoring, and quality-weighted semantic alignment of similar samples are performed sequentially. The target driver distraction detection model is then trained by combining class activation map compactness constraint loss, inter-class orthogonality constraint loss, and quality-weighted semantic alignment loss of similar samples. During the detection phase, the trained lightweight backbone network and classifier output the driver distraction behavior category results, without using class activation map constraint branches.

[0036] In step S110, driver image samples with driver behavior category labels are obtained.

[0037] The driver image samples can be captured by in-vehicle cameras or derived from a pre-built driver behavior image dataset. The driver behavior categories can include safe driving, texting with the right hand, making or receiving calls with the right hand, texting with the left hand, making or receiving calls with the left hand, adjusting in-vehicle equipment, drinking water, turning to retrieve an item, fixing hair or applying makeup, and conversing with passengers. These categories are merely examples; in practical applications, they can be added, deleted, merged, or redefined according to the needs of the driver status monitoring task.

[0038] In step S120, the driver image sample is input into a lightweight backbone network to extract the corresponding feature map, and the driver behavior category prediction result is output through a classifier.

[0039] The lightweight backbone network can employ MobileNet, ShuffleNet, EfficientNet, or other lightweight visual neural networks, or it can employ a visual neural network with pruned, quantized, or lightweight structural designs. The classifier can include a global average pooling layer and a linear classification layer, used to generate driver behavior category prediction results based on the feature maps.

[0040] In one implementation, let the current training batch contain the first... One driver image sample Its actual category label is The driver image sample Inputting a lightweight backbone network yields high-level feature maps:

[0041]

[0042] in, This indicates a lightweight backbone network. This represents the backbone network parameters. The feature map can be represented as:

[0043]

[0044] in, Indicates the number of feature channels. and These represent the height and width of the feature map, respectively.

[0045] For the feature map Perform global average pooling to obtain the global feature vector:

[0046]

[0047] The global feature vector is input into the linear classification layer to obtain the driver behavior category prediction result:

[0048]

[0049] in, Represents the weights of the linear classification layer. Indicates the first The probability distribution of the predicted categories for each driver image sample.

[0050] In step S130, class activation maps corresponding to each driver behavior category are generated based on the feature map and the classification layer weights of the classifier.

[0051] The class activation mapping is used to represent the response intensity of the model to different spatial regions in the input image when judging a certain driver behavior category. Through class activation mapping, the model's attention to the hand region, head orientation region, interactive object region, and other local action regions can be obtained.

[0052] For the There are *n* driver behavior categories, and let the weight vector corresponding to each category in the linear classification layer be . According to the weight vector and feature map , generate the first Class activation mapping corresponding to each driver behavior category:

[0053]

[0054] in, Indicates spatial location on the feature map. Indicates the first The sample at the th individual channels, spatial locations The characteristic response at that location, Indicates the first The categories in the The classification layer weights on each channel.

[0055] To facilitate comparisons between different samples and categories, the class activation map is normalized to obtain a normalized class activation map:

[0056]

[0057] in, This is a numerically stable term. The normalized class activation mapping... Used for subsequent sample attention map reliability assessment and semantic alignment of similar samples.

[0058] In step S140, based on the true category label of the driver image sample, the class activation map corresponding to the true category is selected from the class activation maps corresponding to each driver behavior category, and the spatial focus score and category discrimination score are calculated based on the class activation map corresponding to the true category.

[0059] During the training phase, the true class labels of the samples are known. Therefore, this invention preferentially selects the normalized class activation mapping corresponding to the true class. As an object for evaluating the reliability of attention maps, the class activation maps corresponding to the true classes more directly reflect whether the model has formed an effective spatial response for the correct classes, compared to the class activation maps corresponding to the predicted classes.

[0060] This invention evaluates the reliability of sample attention maps from two dimensions: spatial focus and category discrimination ability.

[0061] First, the spatial focus score is calculated. The spatial focus score measures whether the class activation map corresponding to the true class concentrates the main response in a few key locations. For driver distraction detection tasks, discriminative visual cues are typically concentrated in the driver's hands, head orientation, localized limb movements, and areas of interacting objects. Therefore, a high-quality class activation map should produce strong responses in a few key areas, rather than generating scattered responses in a large background area.

[0062] In one implementation, the normalized class corresponding to the real category is activated for mapping. The response values ​​of all spatial locations are sorted in descending order, and the largest response value is selected first. These spatial locations form a set of spatial locations:

[0063]

[0064] Calculate the first value based on the average response value of each spatial location in the set of spatial locations. Spatial focus score for each sample:

[0065]

[0066] when When the value is large, it indicates that the class activation mapping corresponding to the true category is more concentrated on a small number of high-response regions, suggesting that the attention map of this sample is more likely to focus on key local regions related to the driver's behavior category.

[0067] Secondly, the category discrimination score is calculated. Spatial focus alone is insufficient to fully assess the quality of an attention map. For some samples, while the spatial response of the class activation map corresponding to the true class may be relatively concentrated, its high-response regions may highly overlap with the class activation maps corresponding to other classes. In such cases, although the attention map exhibits local concentration, it may not necessarily possess good class discrimination.

[0068] Therefore, this invention further calculates the category discrimination score. Specifically, at each spatial location, the normalized class activation map response value corresponding to the true class is compared with the maximum response value in the normalized class activation maps of other classes, and the positive difference between the true class response and the response of the strongest competing class is calculated. The class discrimination score of a sample is defined as:

[0069]

[0070] in, Indicates the first Each sample in category Normalized class activation mapping on spatial location The response value at that location, and These represent the height and width of the class activation map, respectively.

[0071] when When the value is large, it indicates that the class activation map corresponding to the true class not only has a strong response but also exhibits more significant class specificity compared to other competing classes. Therefore, the spatial focus score and the class discrimination score describe the quality of the sample attention map from the perspectives of local concentration and class discriminability, respectively, and the two have complementary effects.

[0072] In step S150, the spatial focusing score and the category discrimination score are adaptively fused according to the maturity of the category discrimination information in the current training batch to obtain a sample-level quality score.

[0073] In obtaining spatial focus score and category discrimination score Then, sample-level quality scores can be constructed. If fixed weights are used to fuse the two directly, it is assumed that the spatial focus score and the class discrimination score have the same stability throughout the training process. However, in the early stages of model training, the classification boundary has not yet fully formed, and the class discrimination score is prone to large fluctuations; while spatial focus information usually appears earlier and can provide an earlier reference for the quality of the attention map. As the training process progresses, the model gradually develops a clearer class discrimination ability, and the reliability of the class discrimination score gradually increases.

[0074] Based on the above considerations, the present invention dynamically adjusts the fusion ratio between spatial focusing score and category discrimination score according to the maturity of category discrimination information in the current training batch.

[0075] In one implementation, the samples with the highest class discrimination scores are first selected from the current training batch to form a sample set. Then, the sample set is calculated. The average class discrimination score of the middle samples is used as the class discrimination maturity of the current training batch:

[0076]

[0077] Furthermore, a dynamic fusion coefficient is calculated based on the maturity level determined by the category:

[0078]

[0079] in, and These represent the lower and upper limits of the dynamic fusion coefficient, respectively. This indicates that the maturity level for category discrimination is determined by an upper bound. This represents a truncation function used to restrict the input to a preset range.

[0080] When the class discrimination maturity of the current training batch is low, the dynamic fusion coefficient The dynamic fusion coefficient is relatively small, thus reducing the impact of the category discrimination score on the sample-level quality score; as the maturity of category discrimination gradually increases, the dynamic fusion coefficient... This increases the importance of category discrimination scores in quality scoring.

[0081] Based on the aforementioned dynamic fusion coefficient, calculate the first... Sample-level quality score for each sample:

[0082]

[0083] in, This indicates the lower limit of the sample-level quality score.

[0084] The purpose of setting a lower limit for quality scores is to prevent low-quality samples from being completely ignored during training. While the reliability of attention maps for difficult or low-quality samples is lower, their category supervision information still has training value. Therefore, by setting a lower limit... It can strike a balance between suppressing the propagation of noise in low-quality attention maps and preserving supervisory information from difficult samples.

[0085] Through the aforementioned adaptive sample quality scoring mechanism, this invention can dynamically adjust the quality estimation method based on the maturity of category discrimination information during training, making the sample-level quality score more consistent with the actual state during model training. The sample-level quality score will be further used for quality-weighted semantic alignment between samples of the same type.

[0086] In step S160, based on the sample-level quality scores between samples of the same category, sample pair constraint weights are constructed, and quality-weighted semantic alignment of samples of the same category is performed.

[0087] For driver distraction detection tasks, although different samples of the same driver behavior category may differ in driver identity, background, lighting, or shooting angle, their key local regions related to the category usually exhibit a certain degree of semantic consistency. For example, the category of making or receiving phone calls is usually associated with the area near the hands and ears, the category of drinking water is usually associated with the area of ​​the hands and cup-shaped objects, and the category of adjusting in-vehicle devices is usually associated with the hands and the central control area. Therefore, the semantic representations formed by samples of the same category under the guidance of class activation mapping should have high consistency.

[0088] However, if the same semantic alignment constraint is imposed on all similar sample pairs, erroneous attention patterns present in low-quality samples may propagate to other samples. Therefore, this invention dynamically adjusts the semantic alignment strength of similar sample pairs based on sample-level quality scores.

[0089] In one implementation, the total number of driver behavior categories is assumed to be... For the first For each sample, activation mapping is performed using the normalized class corresponding to each category. For feature maps Spatial location weighting is performed to obtain category-related weighted features. Then, spatial average pooling is performed on the category-related weighted features to obtain the first... Category-related semantic vectors for each category:

[0090]

[0091] in, This represents a weighted operation based on spatial location. This indicates a global average pooling operation.

[0092] The first The semantic vectors of multiple categories corresponding to a sample are concatenated in category order to obtain the overall semantic representation of the sample:

[0093]

[0094] in, This represents a vector concatenation operation. This represents the normalization operation. The overall semantic representation includes not only the semantic information in the feature map, but also the category response structure guided by the class activation mapping.

[0095] For the The first sample and the first For each sample, calculate the similarity between their overall semantic representations:

[0096]

[0097] Based on the semantic similarity between samples within a training batch, a predicted similarity distribution can be constructed. Simultaneously, based on the relationship between the true class labels of samples within the training batch, a target similarity distribution is constructed. When two samples belong to the same driver behavior category, the corresponding position in the target similarity distribution has a higher target value; when two samples belong to different driver behavior categories, the corresponding position in the target similarity distribution has a lower target value.

[0098] To adjust the semantic alignment strength of similar samples based on sample quality, this invention further constructs sample pair constraint weights. For the first... The first sample and the first For each sample, when the true class labels are the same, the constraint weight of the sample pair is determined based on the smaller of the sample-level quality scores; when the true class labels are different, the constraint weight of the sample pair is reset to zero.

[0099]

[0100] in, Indicates an indicator function, and They represent the first The first sample and the first Sample-level quality score for each sample.

[0101] The smaller of the two sample-level quality scores is used as the constraint weight for the sample pair because the reliability of semantic alignment between similar samples is usually limited by low-quality members. Even if one sample has a high quality score, if the attention map of the other sample shows significant shifts, dispersion, or class confusion, then the sample pair should not be subject to overly strong semantic alignment constraints. This weighting method allows for stronger constraints on high-quality similar sample pairs, while sample pairs containing low-quality samples receive weaker constraints.

[0102] Furthermore, regarding the first The constraint weights of the sample pairs between the i-th sample and other samples in the training batch are summed to obtain the j-th sample. The overall weight of each sample in the semantic alignment process of samples of the same type:

[0103]

[0104] Based on the overall weights, a quality-weighted semantic alignment loss for similar samples is constructed:

[0105]

[0106] in, Denotes KL divergence, This is a numerically stable term. Through the quality-weighted semantic alignment loss for similar samples, samples with higher quality have a greater weight in semantic alignment optimization, while the influence of samples with lower quality on the optimization direction is correspondingly weakened.

[0107] Therefore, this invention can reduce the propagation of erroneous supervision caused by low-quality attention maps while maintaining semantic supervision between similar samples, thereby improving the stability of the driver distraction detection model training process.

[0108] In step S170, the driver distraction detection model is trained by combining classification loss, class activation mapping compactness constraint loss, inter-class orthogonality constraint loss, and quality-weighted semantic alignment loss of similar samples.

[0109] In one implementation, the overall training objective of the driver distraction detection model includes classification loss, class activation map compactness constraint loss, inter-class orthogonality constraint loss, and quality-weighted semantic alignment loss for similar samples. The total training loss can be expressed as:

[0110]

[0111] in, Represents classification loss. The class activation mapping compactness constraint loss is represented. Represents the loss due to orthogonality constraints between classes. This represents the quality-weighted semantic alignment loss for similar samples. , and These represent the weighting coefficients of the corresponding loss terms.

[0112] The classification loss ensures the model's basic ability to recognize driver behavior categories. The class activation map compactness constraint loss promotes the concentration of class activation map responses in more compact key regions, reducing irrelevant background responses. The inter-class orthogonality constraint loss reduces response overlap between class activation maps of different categories, enhancing category specificity. The quality-weighted semantic alignment loss for similar samples adjusts the semantic alignment strength between similar samples based on sample-level quality scores.

[0113] In one implementation, to improve the stability of the training process, a hierarchical round-robin sampling strategy can be used to construct training batches, ensuring that each training batch covers as many driver behavior categories as possible, and that the same category has multiple samples within each training batch. This approach can improve the effectiveness of semantic alignment constraints for samples of the same type.

[0114] In one implementation, Xavier initialization can be used for the linear classification layer of the classifier to reduce the impact of uneven weight distribution in the early stages of training on the quality of class activation map generation. More stable classification layer initialization helps the class activation map constraint path enter an effective training state earlier.

[0115] In one implementation, the running statistics of the batch normalization layer can be frozen during training to reduce the interference of training batch sampling fluctuations on feature distribution estimation, thereby improving the convergence stability of the lightweight model in complex driver image scenarios.

[0116] It should be noted that the above sampling strategy, initialization strategy, and batch normalization processing strategy can be used as preferred training configurations and do not constitute a limitation on the scope of protection of this invention. The core of this invention lies in: constructing a sample attention map reliability evaluation mechanism based on class activation mapping, obtaining sample-level quality information through adaptive sample quality scoring, and introducing the sample-level quality information into the semantic alignment process of similar samples to dynamically adjust the cross-sample constraint strength.

[0117] After training, this invention no longer uses the sample attention map reliability evaluation module, the adaptive sample quality scoring module, and the quality-weighted semantic alignment module for similar samples during the inference phase. In other words, the quality-aware class activation map constraint path used during training is only used to optimize model parameters and is not a necessary network structure for the inference phase.

[0118] In the inference phase, the image of the driver to be identified is input into the trained lightweight backbone network to obtain the corresponding feature map; the driver behavior category prediction result is obtained through global average pooling and a classifier. The computational flow of the inference phase can be represented as follows:

[0119]

[0120] in, This represents the image of the driver to be identified. This represents the lightweight backbone network after training. This represents the classification layer weights after training. This indicates the predicted driver behavior category.

[0121] Since the inference phase only retains a lightweight backbone network and classifier, this invention does not introduce additional detectors, key point localization modules, multi-region pruning modules, or additional inference branches. Therefore, this invention can improve the learning ability and recognition stability of the driver distraction detection model for key local regions while maintaining low inference computation and low system complexity. It is suitable for applications such as in-vehicle driver state monitoring, intelligent assisted driving, fleet safety management, and driving behavior analysis.

[0122] As an alternative, the lightweight backbone network is not limited to MobileNet, ShuffleNet, or EfficientNet, and can be replaced by other lightweight convolutional neural networks, lightweight visual Transformer networks, or visual recognition networks with model compression. The generation method of the class activation map is not limited to the CAM method based on classification layer weights, but can also employ Grad-CAM, Score-CAM, or other attention mapping methods capable of characterizing category-related spatial responses. The spatial focus score is not limited to the top-K response mean, but can also be measured using metrics such as response entropy, response variance, compact region area, or high-response region connectivity. The category discrimination score is not limited to the average of the positive differences between the true category response and the strongest competing category response, but can also be calculated using methods such as inter-category response distance, category response ranking interval, or mutual information metrics. The adaptive fusion method is not limited to estimating maturity based on the highest few category discrimination scores within a batch, but can also dynamically determine the fusion coefficients based on the training epochs, validation set performance, inter-category boundary intervals, or exponential moving average statistics. The sample pair constraint weights are not limited to the smaller of the two sample quality scores; they can also be the average, product, harmonic mean, or weights mapped by a learnable function. The quality-weighted semantic alignment loss is not limited to KL divergence; it can also be mean squared error loss, contrastive learning loss, supervised contrastive loss, cosine distance loss, or other distribution alignment losses.

[0123] Furthermore, the technical solution of this invention, by introducing a sample attention map reliability evaluation mechanism, can distinguish between high-quality and low-quality class activation maps from two dimensions: spatial focus and category discrimination ability. Through an adaptive sample quality scoring mechanism, the quality estimation method can be dynamically adjusted according to the maturity of category discrimination information during training. Through a quality-weighted semantic alignment mechanism for similar samples, the supervisory role of high-quality samples in cross-sample constraints can be enhanced, while the interference of low-quality samples on the optimization direction can be weakened. Moreover, the quality-aware class activation map constraint is only used in the training phase and does not add additional network structure and computational overhead in the inference phase, thus balancing recognition performance, training stability, and computational efficiency in the detection phase.

[0124] In the description of this specification, the references to terms such as "an embodiment," "an implementation," "some embodiments," "some implementations," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, or characteristic described in connection with that embodiment, implementation, or example is included in at least one embodiment, implementation, or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment, implementation, or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments, implementations, or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments, implementations, or examples described in this specification, as well as the features of different embodiments, implementations, or examples.

[0125] Those skilled in the art should understand that the above embodiments are merely for illustrating the present invention and are not intended to limit the scope of the invention. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present invention.

Claims

1. A driver distraction detection method based on quality-perceived class activation mapping constraints, characterized in that, Includes the following steps: Obtain driver image samples with driver behavior category labels; The driver image samples are input into the driver distraction detection model, which includes a lightweight backbone network and a classifier; the corresponding feature maps are extracted through the lightweight backbone network, and the driver behavior category prediction results are output through the classifier; Based on the feature map and the classification layer weights of the classifier, a class activation mapping is generated for each driver behavior category. Based on the true category label of the driver image sample, select the class activation map corresponding to the true category from the class activation maps corresponding to each driver behavior category; Based on the class activation mapping corresponding to the true category, calculate the spatial focusing score and class discrimination score of the sample attention map; Based on the maturity of the category discrimination information in the current training batch, the spatial focusing score and the category discrimination score are adaptively fused to obtain a sample-level quality score. Based on the sample-level quality scores among samples of the same category, sample pair constraint weights are constructed, and quality-weighted semantic alignment of samples of the same category is performed. The driver distraction detection model is trained by combining classification loss, class activation mapping compactness constraint loss, inter-class orthogonality constraint loss, and quality-weighted semantic alignment loss of similar samples to obtain the target driver distraction detection model. During the detection phase, the trained target driver distraction detection model is used to perform forward inference on the driver image to obtain the corresponding driver distraction behavior category.

2. The method according to claim 1, characterized in that, The step of generating class activation maps corresponding to each driver behavior category based on the feature map and the classification layer weights of the classifier includes: The feature map output by the lightweight backbone network is denoted as... ; The classifier corresponding to the first The classification layer weights for each driver behavior category are denoted as follows: ; According to the feature map and the classification layer weights Generate the first Class activation mapping corresponding to each driver behavior category; The generated class activation maps are normalized to obtain normalized class activation maps.

3. The method according to claim 1, characterized in that, The calculation of the spatial focusing score of the sample attention map includes: The first The true category label of each driver image sample is denoted as Let the normalized class activation mapping corresponding to its true category be denoted as ; Sort the spatial positions in the normalized activation map according to the response value, and select the positions with the largest response values. Each spatial location forms a set of spatial locations. ; The spatial focusing score is calculated based on the average response value of each spatial location in the set of spatial locations:

4. The method according to claim 1, characterized in that, The calculation of the category discrimination score of the sample attention map includes: At each spatial location, calculate the positive difference between the normalized class activation map response value corresponding to the true class and the maximum response value of the normalized class activation map for other classes; The average of the positive differences at each spatial location is used to obtain the category discrimination score: in, and These represent the height and width of the class activation map, respectively. Indicates the first Driver image samples in category Normalized class activation mapping on spatial location The response value at that location.

5. The method according to claim 1, characterized in that, The spatial focusing score and the category discrimination score are adaptively fused based on the maturity level of the category discrimination information in the current training batch to obtain a sample-level quality score, including: Select the samples with the highest class discrimination scores from the current training batch to form a sample set. ; According to the sample set Calculate the class discrimination maturity of the current training batch by using the average class discrimination score of the samples: Based on the stated category, maturity level is determined, and the dynamic fusion coefficient is calculated: in, and These represent the lower and upper limits of the dynamic fusion coefficient, respectively. This indicates the upper bound of the maturity level for category discrimination; Based on the dynamic fusion coefficient, the spatial focus score and the category discrimination score are fused to obtain a sample-level quality score: in, This indicates the lower limit of the sample-level quality score.

6. The method according to claim 1, characterized in that, The quality-weighted semantic alignment of similar samples includes: The feature map is spatially weighted using the normalized class activation mapping corresponding to each category; Spatial average pooling is performed on the weighted feature map to obtain the category-related semantic vector; By concatenating multiple category-related semantic vectors in category order, a semantic representation of the sample is obtained. The semantic representation of the samples is normalized, and the semantic similarity between samples is calculated based on the normalized semantic representation.

7. The method according to claim 1, characterized in that, The construction of sample pair constraint weights based on sample-level quality scores among samples of the same category includes: For the The first sample and the first For each sample, when the true class labels are the same, the constraint weight of the sample pair is determined based on the smaller of the sample-level quality scores; when the true class labels are different, the constraint weight of the sample pair is reset to zero. in, Indicates an indicator function, and They represent the first The first sample and the first Sample-level quality score for each sample.

8. The method according to claim 1, characterized in that, The quality-weighted semantic alignment loss for similar samples is: in, This represents the target similarity distribution constructed based on the true class label relationships within the training batch. This represents the predicted similarity distribution constructed based on the semantic similarity between sample semantic representations. Denotes KL divergence, It is a numerically stable term; The total training loss of the driver distraction detection model is: in, Represents classification loss. The class activation mapping compactness constraint loss is represented. Represents the loss due to orthogonality constraints between classes. , and These represent the weighting coefficients of the corresponding loss terms.

Images