Driver distraction behavior detection method based on open set domain adaptation

Abstract

The invention discloses a driver distraction behavior detection method based on open set domain adaptation, and belongs to the field of driving detection. The method comprises a consistency maximization module and a subclass discriminator module. The method comprises a training stage and a testing stage. In the training stage, for a consistency maximization module, a randomly combined multi-modal data enhancement strategy is adopted, a Sentry Loss function is introduced, and the loss function effectively improves the stability and consistency of system output by explicitly minimizing an entropy value represented by a feature. The invention provides an innovative framework based on consistency maximization and a class level identification mechanism. According to the method, firstly, through multi-view feature learning and Sentry loss optimization, the recognition robustness of a model to a known category is improved; secondly, proposing a subclass discriminator module, designing an exclusive binary discriminator for each closed set class, constructing a closed set class rejection boundary through class-level adversarial training, and realizing efficient filtering and inter-class feature isolation of open class samples;

Description

Technical Field

[0001] The present invention relates to driving detection, and in particular to a driver distraction behavior detection method based on open set domain adaptation. Background Art

[0002] Distracted driving is one of the key factors leading to traffic accidents. Existing distracted behavior recognition is mainly based on deep learning methods, which identify distracted behaviors by analyzing the static features of images. However, there is significant uncertainty in driver behavior, and the labeled data used for algorithm training is difficult to exhaust all behavioral manifestations in real driving, which causes cross-domain label distribution shift (LDS), limiting the generalization ability of the model. Open-Set Domain Adaptation (OSDA) aims to achieve effective knowledge transfer under the condition that there is a difference in data distribution between the source domain and the target domain and the target domain contains samples of unknown categories. Traditional transfer learning methods often cannot effectively distinguish unknown samples from known categories due to their over-reliance on global alignment or entropy regularization, resulting in open class samples being incorrectly assimilated into the closed set, ultimately causing the performance degradation of the model in actual deployment.

[0003] Deep learning, as a powerful pattern recognition method, has achieved tremendous success in the field of machine vision and has been widely applied to object detection, semantic segmentation, and behavior recognition. Inspired by these successful applications, most researchers use deep learning methods based on CNN architectures to build behavior classification models by learning image features of various distracting behaviors. Existing work primarily focuses on improving recognition accuracy by optimizing deep learning architectures or pursuing more efficient input through feature fusion.

[0004] Although deep learning-based distraction behavior detection methods perform well on standard datasets, their application in actual driving scenarios still faces significant challenges. These challenges mainly come from two aspects: First, due to the high randomness and unpredictability of driver behavior in actual driving, the labeled categories of the training dataset (source domain) are often difficult to cover all possible behavior patterns in real scenarios (target domain). This inconsistency in label distribution between the source domain and the target domain, that is, cross-domain label distribution offset, will directly lead to a decrease in the performance of the model in actual applications. Secondly, there are differences in parameters such as the installation position and shooting angle of cameras in different vehicles, resulting in inconsistencies in the distribution of visual features between the target domain and the source domain. This cross-domain feature distribution difference will significantly affect the generalization ability of the model, making it difficult to adapt to actual driving data collected from different vehicles.

[0005] Open set domain adaptation aims to address the transfer learning problem when the source and target domains have different distributions and the target domain contains open-class samples. Existing methods typically rely on global feature alignment or entropy optimization, but these methods are inadequate for detecting open-class samples and isolating inter-class features. This can lead to open-class samples in the target domain being misclassified as closed-class samples, impacting transfer performance. Summary of the Invention

[0006] Purpose of the invention: The purpose of the present invention is to provide a driver distraction behavior detection method based on open set domain adaptation; to solve the problems existing in the above-mentioned background technology.

[0007] Technical Solution: A driver distraction behavior detection method based on open set domain adaptation, which includes a consistency maximization module and a subclass discriminator module; and includes two stages: training and testing;

[0008] Training Phase: For the consistency maximization module, a randomly combined multimodal data augmentation strategy is adopted, and the Sentry Loss function is introduced. This loss function effectively improves the stability and consistency of the system output by explicitly minimizing the entropy of the feature representation. For the subclass discriminator, fine-grained adversarial training is used to learn the exclusion boundary of each closed set category, forming a "class-level" open set detection mechanism. This can establish an isolation zone between classes during the feature extraction phase, so that open class samples trigger the joint filtering of the multi-level discriminator when migrating to the closed set feature space.

[0009] Testing phase: First, the target domain data is input into the system and undergoes various types of data augmentation. The augmented data is then input into the backbone model, which is responsible for extracting features and generating preliminary pseudo log-probabilities. These pseudo log-probabilities are then averaged. After the log-probabilities are averaged, the system checks whether the maximum value of the average log-probabilities exceeds a preset threshold. If so, the system selects the category with the maximum log-probability as the final classification result; if not, the input is classified as "other".

[0010] Preferably, the specific training process of the consistency maximization module is: given the model parameters θ, the input image x t After n types of data enhancement, multi-view features {a1(x t ),a2(x t ),...,a n (x t )}, given the input x and model parameters θ, the probability that the predicted value y is c is p Θ (y=c|x), c is the label category, and the consistency constraint is implemented through Sentry Loss:

[0011]

[0012] where i and j represent the indices of the last consistent and inconsistent transformation versions, respectively.

[0013] Preferably, the specific training process of the class discriminator is: for each closed set category k∈C s Designing a binary discriminator Constructing the exclusion boundary through class-level adversarial training, the feature generator G f The adversarial loss with the discriminator is:

[0014]

[0015] This loss forces the target domain samples to stay away from the feature space of the closed set category;

[0016]

[0017] The model is calculated on the source dataset Average loss on is the i-th sample in the source domain data, n s is the number of source domain samples, J is the cross entropy loss function, which measures the model prediction and the true label the differences between;

[0018]

[0019] in where d i represents the domain label of the i-th sample, which is used to identify whether the sample belongs to the source dataset or the target dataset; θ d is the global domain discriminator parameter, the feature generator G f , G d is the global domain discriminator;

[0020] Combining the classification loss with the global domain alignment loss, the final overall objective function is:

[0021]

[0022] Beneficial effects: The present invention proposes an innovative framework based on consistency maximization and class-level discrimination mechanism. First, through multi-perspective feature learning and Sentry loss optimization, the model's recognition robustness to known categories is improved. Secondly, a subclass discriminator module is proposed to design an exclusive binary discriminator for each closed set category, and the closed set category exclusion boundary is constructed through class-level adversarial training to achieve efficient filtering of open class samples and inter-class feature isolation. Finally, the overall objective function is designed by combining classification loss, global domain alignment loss and class-level discrimination loss to further optimize model performance and improve the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram of the method flow chart;

[0024] Figure 2 This is a schematic diagram of the testing phase. DETAILED DESCRIPTION

[0025] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example

[0027] like Figure 1 This paper presents a method for detecting driver distraction based on open-set domain adaptation. This method consists of two key modules: a consistency maximization module and a subclass discriminator module. These two modules enable effective open-set domain adaptation, significantly improving the model's performance in practical applications. The backbone network model follows the dual-channel model from Chapter 3.

[0028] This embodiment proposes a consistency maximization module based on multimodal feature enhancement, which aims to improve performance by enhancing the model's perception of essential features. The core design concept of this module is: assuming that the same picture should show high consistency and have a low entropy value after being enhanced in different ways. During the training process, a variety of different data enhancement methods are randomly used for optimization; in specific implementation, a randomly combined multimodal data enhancement strategy is adopted during the training process, and the Sentry Loss function is innovatively introduced. This loss function effectively improves the stability and consistency of the system output by explicitly minimizing the entropy of the feature representation. Experimental results show that this design significantly enhances the model's ability to extract the essential features of the image.

[0029] Given the model parameters θ, the input image x t After n types of data enhancement, multi-view features {a1(x t ),a2(x t ),...,a n (x t )}, given the input x and model parameters θ, the probability that the predicted value y is c is p Θ (y=c|x), c is the label category, and the consistency constraint is implemented through Sentry Loss:

[0030]

[0031] where i and j represent the indices of the last consistent and inconsistent transformation versions, respectively.

[0032] The subclass discriminator module innovates its architecture based on the classic domain adversarial network (DANN), designing a dedicated binary discriminator for each closed-set class. Unlike the global domain discriminator, which only distinguishes between the source and target domains, the subclass discriminator learns the exclusion boundary for each closed-set class through fine-grained adversarial training, forming a "class-level" open-set detection mechanism. This module establishes isolation zones between classes during the feature extraction phase, allowing open-class samples to trigger joint filtering by multiple discriminators as they migrate to the closed-set feature space.

[0033] Specifically, for each closed set category k∈C s Designing a binary discriminator Constructing the exclusion boundary through class-level adversarial training, the feature generator G f The adversarial loss with the discriminator is:

[0034]

[0035] This loss forces the target domain samples to stay away from the feature space of the closed set category;

[0036]

[0037] The model is calculated on the source dataset Average loss on is the i-th sample in the source domain data, n s is the number of source domain samples, J is the cross entropy loss function, which measures the model prediction and the true label the differences between;

[0038]

[0039] in where d i represents the domain label of the i-th sample, which is used to identify whether the sample belongs to the source dataset or the target dataset; θ d is the global domain discriminator parameter, the feature generator G f , G d is the global domain discriminator;

[0040] Combining the classification loss with the global domain alignment loss, the final overall objective function is:

[0041]

[0042] The above is the training phase. After the training phase is completed, it enters the testing phase. The test phase diagram is as follows Figure 2As shown in the figure, target domain data is first input into the system and subjected to various types of data augmentation (AugmentType1, AugmentType2, ..., AugmentTypeN) to increase data diversity and improve the model's generalization ability. The augmented data is then fed into a backbone model (BackboneModel), which extracts features and generates preliminary pseudo-log-odds (PseudoLogits). These pseudo-log-odds are then averaged to reduce the randomness introduced by data augmentation, resulting in more stable classification results. After averaging the log-odds, the system checks whether the maximum of the averaged log-odds exceeds a preset threshold. If so, the system selects the class with the maximum log-odds as the final classification result; if not, the input is classified as "Other." This step is key to open set recognition, allowing the model to identify and distinguish between known and unknown classes, thereby improving the model's robustness and accuracy when faced with unknown data. In this way, the system can effectively handle open set problems, i.e., identifying new classes that were not among the known classes during training during the testing phase.

[0043] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A driver distraction behavior detection method based on open set domain adaptation, characterized in that: The method includes a consistency maximization module and a subclass discriminator module; and includes two stages: training and testing; Training Phase: For the consistency maximization module, a randomly combined multimodal data augmentation strategy is employed, and the Sentry Loss function is introduced. This loss function effectively improves the stability and consistency of the system output by explicitly minimizing the entropy of the feature representation. For the subclass discriminator, fine-grained adversarial training is used to learn the exclusion boundary of each closed set category, forming a "class-level" open set detection mechanism. This establishes an isolation zone between classes during the feature extraction phase, allowing open class samples to trigger joint filtering by the multi-level discriminator when migrating to the closed set feature space. Testing phase: First, the target domain data is input into the system and undergoes various types of data augmentation. The augmented data is then fed into the backbone model, which extracts features and generates preliminary pseudo-log-odds. These pseudo-log-odds are then averaged. After averaging, the system checks whether the maximum value of the averaged log-odds exceeds a preset threshold. If so, the system selects the class with the maximum log-odds as the final classification result. If the threshold is not exceeded, the input is classified into the "Other" category.

2. The driver distraction behavior detection method based on open set domain adaptation according to claim 1, characterized in that: The specific training process of the consistency maximization module is: given the model parameters θ, the input image x t After n types of data enhancement, multi-view features {a1(x t ),a2(x t ),...,a n (x t )}, given the input x and model parameters θ, the probability that the predicted value y is c is p Θ (y=c|x), c is the label category, and the consistency constraint is implemented through Sentry Loss: where i and j represent the indices of the last consistent and inconsistent transformation versions, respectively.

3. The driver distraction behavior detection method based on open set domain adaptation according to claim 1, characterized in that: The specific training process of the subclass discriminator is: for each closed set category k∈C s Designing a binary discriminator Constructing the exclusion boundary through class-level adversarial training, the feature generator G f The adversarial loss with the discriminator is: This loss forces the target domain samples to stay away from the feature space of the closed set category; The model is calculated on the source dataset Average loss on is the i-th sample in the source domain data, n s is the number of source domain samples, J is the cross entropy loss function, which measures the model prediction and the true label the differences between; in where d i represents the domain label of the i-th sample, which is used to identify whether the sample belongs to the source dataset or the target dataset; θ d is the global domain discriminator parameter, the feature generator G f , G d is the global domain discriminator; Combining the classification loss with the global domain alignment loss, the final overall objective function is:

Images