Selective deep transfer learning method based on driving style difference

Abstract

The application discloses a kind of selective deep migration learning methods based on driving mark difference, consider the damage of driver individual difference to fatigue driving identification model universality, aim at solving the problem of low precision in prior art to unknown driver identification fatigue driving state.By this method, system can extract driving mark feature knowledge from the model of known driver, and migrate to new driver, form personalized fatigue driving identification model.

Description

Technical Field

[0001] This invention relates to the fields of machine learning, transfer learning, and deep learning, and in particular to a fatigue driving identification model based on driver's footprint (driving behavior pattern) data. Specifically, it is a selective deep transfer learning method based on driver's footprint differences to improve the accuracy of identifying the fatigue driving state of unknown drivers. Background Technology

[0002] Existing fatigue driving detection technologies typically rely on driver physiological data or facial expression analysis via in-vehicle cameras to identify fatigue states. While these methods can achieve some success in fatigue state identification, they suffer from inconsistent accuracy, dependence on pre-acquired fatigue calibration data, and high requirements for the driving environment. In particular, they do not adequately consider individual differences among drivers, resulting in poor generalization ability of existing models for new drivers and difficulty in effectively adapting to the fatigue states of each individual driver. Furthermore, as individual driver differences and driving environments change, existing models exhibit poor generalization ability for new drivers, making it difficult to effectively adapt to the fatigue states of each individual driver.

[0003] In recent years, driver fingerprint-based fatigue driving identification methods have gradually emerged. Driver fingerprints refer to personalized driving behavior features extracted through analysis of driver operation data. However, due to significant differences in driver fingerprints among different drivers, existing models cannot accurately identify the fatigue state of new drivers. Therefore, there is an urgent need for a method that can improve the generalization ability of fatigue driving identification models when fatigue data of unknown drivers is lacking. Summary of the Invention

[0004] This invention provides a selective deep transfer learning method based on driver fingerprint differences, aiming to solve the problem of low accuracy in identifying fatigue driving states of unknown drivers in existing technologies. Through this method, the system can extract driver fingerprint feature knowledge from models of known drivers and transfer it to new drivers to form a personalized fatigue driving identification model.

[0005] To achieve the above objectives, the specific solution of the present invention is as follows:

[0006] Step A: Randomly assign multiple known subjects (KS) and multiple unknown subjects (US);

[0007] Step B: Establish a pre-trained model library for deep transfer learning;

[0008] Step C: Select the best known subjects (BKS) for deep transfer learning;

[0009] Step D: Retrain the pre-trained model using data from the unknown subject US to complete deep transfer learning and establish a fatigue driving identification model for the unknown subject US.

[0010] In step A:

[0011] The data for known subjects KS include: pre-acquired awake driving dataset and fatigue driving dataset;

[0012] The data for unknown subjects (US) includes: the initial phase dataset and the remaining dataset;

[0013] The initial phase dataset consists of driving data within the initial driving time (IPT); the remaining dataset consists of the data remaining after removing the initial driving time (IPT), representing driving data collected after the initial driving time (IPT) period.

[0014] Initial driving time (IPT) ≤ 30 min.

[0015] In step B, a fatigue driving identification model DRMD based on driving patterns is established for each known subject KS.

[0016] By combining principal component analysis (PCA) with a one-dimensional convolutional neural network (1D-CNN), driving behavior features are automatically extracted and learned, without relying on manually labeled features, thereby identifying whether the driver is fatigued.

[0017] For the i-th known subject KS, the entire sample set is first divided into a training set and a test set using 5-fold cross-validation. Then, a fatigue driving identification model DRMD based on a combination of principal component analysis (PCA) and a 1D convolutional neural network (1D-CNN) is trained using the training set. Finally, the fatigue driving identification model DRMD with the highest identification accuracy is tested using the test set and selected as the fatigue driving identification model DRMD for transfer learning of the known subject KS, denoted as fatigue driving model KSDRMD. i .

[0018] In step C, the process of selecting the best known subject (BKS) for deep transfer learning includes: calculating the representation of the unknown subject (US) and one of the known subjects (KS). i Selection factor CF for differences in driving patterns i We selected the best known subject BKS with the smallest difference from the unknown subject US driving pattern for deep transfer learning;

[0019] The initial phase dataset of the unknown subject (US) was mixed with the test data of each known subject (KS), and then the mixed test data was input into the fatigue driving identification model KSDRMD for each known subject (KS). i The accuracy of driving state identification for each mixed test dataset was calculated; the fatigue driving identification model KSDRMD for each known subject KS was compared. iThe accuracy of driving state identification on a test set containing initial data of unknown subjects was assessed using the selection factor CF. i Calculation method: Determine the best known subjects (BKS) for deep transfer learning based on the selection factor scores.

[0020] Using the best known BKS results, a fatigue driving identification model USDRMD for unknown subjects (US) is built using deep transfer learning.

[0021] The USDRMD model for identifying fatigued driving in unknown subjects (US) was built using deep transfer learning.

[0022] Step 1: Retrieve the best known subject BKS fatigue driving identification model BKSDRMD and use it as a pre-training model for deep transfer learning.

[0023] Step 2: Construct a training dataset for deep transfer learning; compensate the unknown subject US's awake driving fingerprint knowledge to the unknown subject US's fatigue driving identification model USDRMD; supplement the training set of the best known subject BKS to form a training dataset.

[0024] Step 3: Use deep transfer learning to process the fatigue driving identification model BKSDRMD of the best known subject BKS, and establish the fatigue driving identification model USDRMD of the unknown subject US. The fatigue driving identification model BKSDRMD is a fatigue driving identification model based on deep learning and has a strong ability to mine fatigue features. Using the fatigue driving identification model BKSDRMD of the best known subject BKS as the pre-trained model, the pre-trained model is retrained using the training dataset. The network parameter adjustment coefficient AC is designed. Based on the deep transfer learning method, the fatigue driving identification model BKSDRMD of the best known subject BKS is transferred to the fatigue driving identification model USDRMD of the unknown subject US.

[0025] Step 4: Use the remaining data of unknown subjects (US) to verify the effect of selective deep transfer learning based on driving pattern differences, and analyze the influence of network parameter adjustment coefficient AC and initial driving time IPT.

[0026] Selection factor CF i The calculation formula is as follows:

[0027]

[0028] In the formula, CF i KSDRMD represents the selection factor score of the i-th known subject's KS, 1 ≤ i ≤ nk; nk is the number of known subjects' KS; i Given the known subjects KS i The most accurate KSDRMD; mixed iGiven the known subjects KS i Mixed data of test data and initial driving data of unknown subjects (US); ACCDS (KSDRMD) i ,mixd i ) for KSDRMD i For mixd i The accuracy of driving status recognition.

[0029] Using the above method, this invention can significantly improve the generalization ability of the fatigue driving identification model in the absence of unknown driver fatigue data. Experiments show that, through selective deep transfer learning, the model's accuracy, sensitivity, specificity, and F1 score are improved by 21.32%, 8.05%, 30.18%, and 20.18%, respectively, compared to before transfer learning. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the method of the present invention;

[0031] Figure 2 This is a schematic diagram of the fatigue driving identification model based on the combined model of principal component analysis (PCA) and 1D convolutional neural network (1D-CNN) of the present invention.

[0032] Figure 3 This invention demonstrates the accuracy of the fatigue driving identification model KSDRMD for each known subject (ACCDS). KS and accuracy ACCDS KSUS Schematic diagram;

[0033] Figure 4 This invention demonstrates the sensitivity of the fatigue driving identification model KSDRMD for each known subject KS. KS and sensitivity SENDS KSUS Schematic diagram;

[0034] Figure 5 The specificity of the fatigue driving identification model KSDRMD for each known subject KS in this invention is SPEDS. KS and specificity SPEDS KSUS Schematic diagram;

[0035] Figure 6 This invention demonstrates the F1 score (F1DS) of the fatigue driving identification model KSDRMD for each known subject (KS) in identifying their own driving state. KS F1DS scores for identifying unknown test driver conditions KSUS A schematic diagram;

[0036] Figure 7This is a schematic diagram of the accuracy of the KSDRMD fatigue driving identification model before and after migration in identifying the unknown driving state of the subject US, according to the present invention.

[0037] Figure 8 This is a schematic diagram of the sensitivity of the KSDRMD fatigue driving identification model before and after migration of the present invention to identify the unknown driving state of the subject US.

[0038] Figure 9 This is a schematic diagram of the specificity SPEDS of the KSDRMD fatigue driving identification model before and after migration of the present invention in identifying the unknown subject US driving state.

[0039] Figure 10 This is a schematic diagram of the F1 score F1DS of the fatigue driving identification model KSDRMD before and after migration of the present invention for identifying the unknown driving state of the subject US. Detailed Implementation

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but this is not to limit the scope of the invention to this.

[0041] Example 1:

[0042] like Figure 1 As shown in this embodiment, a selective deep transfer learning method based on driving pattern differences is implemented through the following steps:

[0043] Step A: Randomly assign multiple known subjects (KS) and multiple unknown subjects (US);

[0044] Step B: Establish a pre-trained model library for deep transfer learning;

[0045] Step C: Select the best known subject (BKS) for deep transfer learning;

[0046] Step D: Retrain the pre-trained model using data from unknown subjects to complete deep transfer learning and establish a fatigue driving identification model for unknown subjects.

[0047] In step A:

[0048] The data for known subjects KS include: sufficient sober driving datasets and fatigue driving datasets obtained in advance;

[0049] The data for unknown subjects (US) includes: the initial phase dataset and the remaining dataset;

[0050] The initial phase dataset consists of driving data within the initial driving period (IPT); the remaining dataset consists of the data remaining after removing the IPT.

[0051] Initial driving time (IPT) ≤ 30 min. For example, driving data within the first 30 minutes. Previous studies have shown that, provided drivers are adequately rested before driving, they have higher attention and alertness during the initial short driving time (within the first 30 minutes) and are generally in a more awake state.

[0052] Assume that the data collected within the initial driving time (IPT) (0 < initial driving time IPT ≤ 30 min) represents driving data under conscious conditions, but driving data under fatigue conditions for unknown subjects is unknown. Therefore, the experimental data for unknown subjects (US) are divided into two parts: the initial phase dataset (driving data within the initial driving time IPT) and the remaining dataset (data remaining after removing the initial driving time IPT).

[0053] The initial dataset is important training data for Deep Transfer Learning (DTL), while the remaining dataset serves as a test set (DTL_test dataset) to verify the effectiveness of Deep Transfer Learning.

[0054] In step B, a fatigue driving identification model DRMD based on driving patterns is established for each known subject KS.

[0055] Combining Principal Component Analysis (PCA) with a 1D Convolutional Neural Network (1D-CNN), this method automatically extracts and learns driving behavior features under fatigue conditions, without relying on manually labeled features, to identify whether a driver is fatigued. For the i-th known subject KS, the entire sample set is first divided into a training set and a test set using 5-fold cross-validation. Then, the training set is used to train a fatigue driving identification model DRMD based on a combined PCA+1D-CNN model. Finally, the test set is used to test and select the fatigue driving identification model DRMD with the highest identification accuracy as the fatigue driving identification model DRMD for transfer learning of the known subject KS, denoted as fatigue driving model KSDRMD. i (with Known Subject's DRMD).

[0056] Given the advantages of deep learning algorithms, a driver fatigue recognition model based on driver fingerprints is established using a 1D-CNN (One-Dimensional Convolutional Neural Network) as its core. The overall structure of the model is as follows: Figure 2 Training was conducted using known data on the subjects' wakefulness and fatigue levels (KS). Figure 2The fatigue driving identification model based on driver fingerprints uses deep learning algorithms to mine the mapping relationship between the driver fingerprint feature matrix (DF) and driving state labels, thereby realizing fatigue driving identification based on driver fingerprints.

[0057] In step C, the process of selecting the best known subject BKS for deep transfer learning includes: calculating the selection factor CF representing the difference in driving patterns between the unknown subject US and one of the known subjects KSi. i We selected the best known subject BKS with the smallest difference from the unknown subject US driving pattern for deep transfer learning;

[0058] The initial phase dataset of the unknown subject (US) was mixed with the test data of each known subject (KS), and then the mixed test data was input into the fatigue driving identification model KSDRMD for each known subject (KS). i Calculate the fatigue driving identification model KSDRMD one by one. i Accuracy of driving state identification on mixed test datasets; comparison with fatigue driving identification models KSDRMD for known subjects KS. i For the driving state identification accuracy of the test set containing initial stage data of unknown subjects, the selection factor representing the difference between the driving patterns of unknown subjects (US) and known subjects (KS) is calculated; based on the selection factor score, the best known subject (BKS) for deep transfer learning is determined.

[0059] Using the best known subject BKS fatigue driving identification model BKSDRMD, we used deep transfer learning to build an unknown subject US fatigue driving identification model USDRMD (Unknown Subject's DRMD).

[0060] USDRMD uses deep transfer learning to build a fatigue driving identification model for unknown subjects, including:

[0061] Step 1: Retrieve the best known subject's fatigue driving identification model BKSDRMD (Best Known Subject's DRMD) and use it as a pre-trained model for deep transfer learning;

[0062] Step 2: Construct the training dataset (DTL_training dataset) for deep transfer learning; the initial stage data of the unknown subject US contains rich driving fingerprint features in a conscious state, which is the basis for constructing the training dataset. The conscious driving fingerprint knowledge of the unknown subject US is used to compensate the fatigue driving identification model USDRMD of the unknown subject; the training set of the best known subject BKS is used as a supplement to form the training dataset.

[0063] Step 3: Use deep transfer learning to process the best known subject KS fatigue driving identification model BKSDRMD and establish the unknown subject US fatigue driving identification model USDRMD. The fatigue driving identification model BKSDRMD is a deep learning-based fatigue driving identification model with strong fatigue feature mining capabilities. Using the best known subject KS fatigue driving identification model BKSDRMD as the pre-trained model, the pre-trained model is retrained using the training dataset. The network parameter adjustment coefficient AC (Adjustment Coefficient) is designed. Based on the deep transfer learning method, the best known subject KS fatigue driving identification model BKSDRMD is transferred to the unknown subject US to establish the unknown subject US personalized fatigue driving identification model USDRMD.

[0064] Step 4: Use the remaining data of unknown subjects (US) to verify the effect of selective deep transfer learning based on driving pattern differences, and analyze the influence of network parameter adjustment coefficient AC and initial driving time IPT.

[0065] Selective deep transfer learning refers to selecting the best known subject (KS) for deep transfer learning to build a fatigue driving identification model (USDRMD). This includes a selection process and a deep transfer learning process. The selection process determines which known subject (KS) to use for deep transfer learning based on a selection factor (CF). i The Choice Factor is crucial for selective deep transfer learning. The more similar the datasets of the source and target tasks are, the better the transfer learning will be. Therefore, known subjects (KS) with similar driving patterns to the unknown subject (US) should be selected for deep transfer learning. The smaller the driving pattern difference between the known subject (KS) and the unknown subject (US), the better the fatigue driving identification model (DRMD) of the known subject (KS) will perform in identifying driving states on mixed test sets containing samples from the unknown subject (US), and the easier it will be to perform transfer learning.

[0066] Selection factor CF i The calculation formula is as follows:

[0067]

[0068] In the formula, CF i KSDRMD represents the selection factor score of the i-th KS, 1 ≤ i ≤ nk; nk is the number of known subject KS; i Given the known subjects KS i The most accurate fatigue driving identification model is KSDRMD; mixed i Given the known subjects KS i Mixed data of test data and initial driving data of unknown subjects (US); ACCDS (KSDRMD) i,mixd i ) for KSDRMD i For mixd i The accuracy of driving state identification. Under the condition that only driving data from the initial driving phase of an unknown subject (US) can be obtained, it reflects, to some extent, the driving state identification accuracy of a known subject (KS). i Differences in driving patterns between the unknown subject US and the known subject KS i The larger the value, the higher the KSDRMD (Kardrow Driver Detection Model) is. i The stronger the generalization ability to unknown subjects (US), the stronger the generalization ability to known subjects (KS). i The smaller the difference in driving patterns between the known subject KS and the unknown subject US, the better. Therefore, the known subject KS is ranked according to the selection factor score CF. i Sort in descending order, select factor CF i The largest known subject KS is used as the known subject BKS for deep transfer learning to establish the fatigue driving identification model USDRMD.

[0069] Deep transfer learning methods include:

[0070] according to Figure 1 The methodology involves selecting known subjects (BKS) and then applying deep transfer learning to transfer the fatigue driving identification model (BKSDRMD) to unknown subjects (US), thus establishing the fatigue driving identification model (USDRMD). Deep transfer learning refers to the process of applying a model from a source task based on a deep learning algorithm to a target task. Deep transfer learning first trains a deep learning model using data from the source task, using this model as a pre-trained model. Then, it fine-tunes the pre-trained model using data from the target task, leveraging the general features and patterns learned by the pre-trained model to improve its ability to solve the target task.

[0071] Deep transfer learning offers several advantages. For example, it transfers data from models based on deep learning algorithms, possessing powerful nonlinear modeling capabilities and a stronger ability to handle large-scale, complex data. Furthermore, deep transfer learning uses large-scale data for pre-training, learning richer general features, requiring less data for new tasks, and achieving higher transfer efficiency. Therefore, we use deep transfer learning, combined with limited driving data from the initial driving phase of an unknown subject (US), to establish a fatigue driving identification model, USDRMD.

[0072] The deep transfer learning in this study belongs to network-based deep transfer learning, and its process is as follows: Figure 1As shown in Part 4. Network-based deep transfer learning refers to using a portion of the network layers of a pre-trained model in the source domain, and then retraining it to transfer it into a deep learning model for solving a target task. Network-based deep transfer learning assumes that the front layers of the deep learning network can be viewed as a relatively general feature extractor, and its feature representation patterns are applicable to other related target tasks. The main steps and mathematical expressions for performing deep transfer learning are as follows:

[0073] Assume the dataset of the source domain is The index vector of the i-th sample in the source domain. The corresponding driving status labels. The dataset for the target domain is... The index vector of the j-th sample in the target domain. These are the corresponding driving state labels. ns and nt are the sample sizes in the source and target domains, respectively. The source domain dataset is the dataset of known subjects (BKS), and the target domain dataset is the dataset of unknown subjects (US).

[0074] First, in D s Model M, trained on the source task based on the combined model PCA+1DCNN, is used for training. s M s The modeling process is as follows:

[0075]

[0076] In equation (1-2), argmin represents the minimum value; θ s For model parameters; M s This is the parameter space of the model. The meaning of this formula is that in D... s The goal is to find the average loss between the model's output driving state value and the actual driving state value, and then find the model parameter θ that minimizes the average loss. s To establish an optimal fatigue driving identification model M based on driving patterns of known subjects. s .

[0077] Then, with M s For pre-training the model, the dataset D of the target domain is used. t Retraining M s Fine-tuning M s Based on the model parameters, a fatigue driving identification model M suitable for the target task is established. t M t The formula is:

[0078]

[0079] In equation (1-3), argmin represents the minimum value; θ t Fine-tuning M in the target domains The model parameters obtained later; M t The model is M t The parameter space; l is the loss function. After fine-tuning the model, in D t The goal is to find the average loss between the model's output driving state value and the actual driving state value, and then find the model parameter θ that minimizes the average loss. t Establish M t That is, the fatigue driving identification model USDRMD for unknown subjects US.

[0080] The convolutional layers of the BKSDRMD fatigue driving identification model can learn low-level, general fatigue feature representations. These general fatigue feature knowledge (the parameters of the convolutional layers) should be applied to unknown subjects as much as possible. However, the fully connected layers of the BKSDRMD model typically contain high-level features highly correlated with known subjects (BKS), and should be significantly adjusted to quickly adapt to the features of unknown subjects (US). Therefore, during deep transfer learning, by setting a large learning rate for the fully connected layers and a smaller learning rate for other layers, the layers in the BKSDRMD model other than the fully connected layers are frozen, and the focus is on training and adjusting the fully connected layers.

[0081] Example 2: According to Figure 1 The procedure shown is to randomly select subject NO.20 as the unknown subject US, and the remaining 23 subjects as the known subjects KS, and use this as an example for the study.

[0082] Existing research indicates that the KSDRMD fatigue driving identification model has a good recognition effect on the driving state of subject KS. However, due to differences in driving patterns among subjects, the effectiveness of the KSDRMD model in recognizing the driving state of unknown subjects needs further testing to analyze whether it is necessary to consider driving pattern differences in transfer learning. Therefore, the remaining data of unknown subjects were used as test data and input into the KSDRMD fatigue driving identification model for each known subject KS to obtain the recognition effect of the KSDRMD model on the driving state of unknown subject US before transfer learning. The recognition effect of the KSDRMD model on the driving state of unknown subject US was comprehensively evaluated using the accuracy of driving state recognition (ACCDS), sensitivity of driving state (SENDS), specificity of driving state (SPEDS), and F1 score (F1DS).

[0083] Figure 3The accuracy of the fatigue driving identification model KSDRMD for each known subject's KS in identifying their own driving state is demonstrated by ACCDS. KS And the accuracy of ACCDS in identifying unknown test subject driving conditions KSUS .

[0084] like Figure 3 As shown, the accuracy of KS for each known subject in ACCDS KS All reached 100%, however, the accuracy of KS for each known subject was ACCDS. KSUS All were relatively small. Among them, the accuracy of subject No. 1 in ACCDS was [missing information]. KSUS The minimum accuracy was 41.68%, with subject No. 18 achieving the highest accuracy in ACCDS. KSUS The maximum is only 62.52%. For the same precision, ACCDS... KS Compared to ACCDS, the accuracy of KS for each known subject is... KSUS All showed a significant decrease. Among them, the accuracy of ACCDS for subject No. 1 was [missing data]. KSUS The largest decrease was 58.32%. Test results indicate that the fatigue driving identification model KSDRMD for known subjects performs very well in identifying their own driving state. However, due to the differences in driving patterns between the known subject KS and the unknown subject US, the fatigue driving identification model KSDRMD for the known subject KS performs poorly overall in identifying the driving state of the unknown subject US.

[0085] Figure 4 This demonstrates the sensitivity of the fatigue driving identification model KSDRMD for each known subject (KS) in identifying their own driving state. KS Sensitivity to identify unknown test subject driving conditions SENDS KSUS .

[0086] like Figure 4 As shown, the sensitivity of all known subjects' KS is SENDS KS All reached 100%, indicating that the fatigue driving identification model KSDRMD for known subjects KS has excellent accuracy in identifying their own fatigue samples. Compared to sensitivity SENDS KS Sensitivity of KS for each known subject (SENDS) KSUS All decreased. Among them, the sensitivity of subject No. 1 decreased. KSUS The decrease was the largest (51.37%), and the sensitivity of the No. 11 subject was SENDS KSUS The decrease was the smallest (14.96%). The results indicate that differences in driving patterns caused varying degrees of decrease in the ability of the fatigue driving identification model KSDRMD for known subjects KS to detect fatigue samples of unknown subjects US, and an increase in the false negative rate for fatigue samples of unknown subjects US.

[0087] Figure 5 The specificity of KSDRMD in identifying one's own driving state for each known subject KS is shown in SPEDS. KS and the specificity of identifying unknown test subject driving conditions (SPEDS) KSUS .

[0088] like Figure 5 As shown, the specificity SPEDS of all known subjects' KS is... KS All achieved 100%, indicating that the fatigue driving identification model KSDRMD for known subjects KS can accurately identify their own conscious samples. However, the specificity SPEDS for each known subject KS is... KSUS The specificity (SPEDS) was relatively low. Specifically, subjects No. 5 and No. 9 had the lowest specificity. KSUS It is the smallest, with a value of 13.16%. The specificity SPEDS for subject No. 23 is... BKSUS It is the largest, with a value of 64.66%. The specificity of KS for all known subjects is SPEDS. KSUS Average Bit Dissimilarity SPEDS KS There was a decrease. Among them, subjects No. 5 and No. 9 experienced the largest decreases, at 86.84%. (Comparison) Figure 4 and Figure 5 The results show that, compared to specificity, SENDS KSUS Specificity SPEDS for each known subject's KS KSUS The error is even lower. This indicates that the fatigue driving identification model KSDRMD for known subjects KS performs worse in identifying samples of awake driving state from unknown subjects US, with a large number of awake samples being misclassified as fatigue samples. Therefore, a deep transfer learning approach is needed to adjust the fatigue driving identification model KSDRMD for known subjects KS using awake driving data from the initial driving phase of unknown subjects US, thereby obtaining compensation for the awake driving characteristics of unknown subjects US and improving the ability to identify awake samples from unknown subjects US.

[0089] Figure 6 This displays the F1 score (F1DS) of the fatigue driving identification model KSDRMD for each known subject (KS) in identifying their own driving state. KS F1DS scores for identifying unknown test driver conditions KSUS .like Figure 6 As shown, the F1DS of each known subject KS KS All reached 100%, indicating that the fatigue driving identification model KSDRMD for each known subject KS has a strong comprehensive identification ability of the driving state of its own samples. However, compared with the F1 score F1DS, it is still lower. KS F1 scores F1DS for each known subject's KS KSUS All decreased. The F1 score (F1DS) of the No. 1 participant... KSUSThe lowest score was 50.06%, with the No. 1 participant achieving the highest F1 score (F1DS). KSUS The decrease was also the largest, at 49.94%. Subject No. 18's F1 score, F1DS... KSUS The highest score was 72.53%, the F1 score of subject No. 18 was F1DS. KSUS The decrease was the smallest, at 27.47%. This shows that... Figure 3 Consistent with the results, the fatigue driving identification model KSDRMD for known subjects KS does not perform well in comprehensively identifying the driving state of unknown subjects, and there are many cases of missed or false alarms.

[0090] Depend on Figures 3 to 6 The comparison of different model evaluation metrics shows that the fatigue driving identification model KSDRMD for each known subject KS performs well in identifying its own driving state, but performs poorly in identifying the driving state of unknown subjects. This phenomenon reflects both the personalized characteristics of the fatigue driving identification model KSDRMD and the fact that due to differences in driving patterns, the fatigue driving identification model KSDRMD for known subjects KS has poor generalization performance for unknown subjects US, and cannot be directly used to identify the driving state of unknown subjects US. Therefore, it is necessary to fine-tune the fatigue driving identification model KSDRMD for known subjects KS through deep transfer learning to improve the identification performance for the driving state of unknown subjects US.

[0091] Factor selection results: Due to significant differences in driving patterns among subjects, the KSDRMD fatigue driving identification model performs poorly in identifying the driving state of unknown subjects (US). Therefore, the fatigue feature knowledge learned from the KSDRMD model should be utilized, combined with the initial driving data of unknown subjects (US), and deep transfer learning techniques should be applied to establish a fatigue driving identification model adapted to unknown subjects (US). Figure 1 When performing selective deep transfer learning, the first step is to calculate the selection factor CF of the known subject KS according to formula (1-1). i The best known subject (BKS) for deep transfer learning was determined based on the selection factor scores. Table 2-1 shows the selection factor scores of all known subject KS, with each known subject KS arranged in ascending order of selection factor scores.

[0092] Table 2-1 Known selection factor scores of participants' KS

[0093]

[0094]

[0095] ACCDS(KSDRMD i ,ksd i ) is KSi KSDRMD i The accuracy of identifying one's own driving status, ACCDS (KSDRMD) i ,mixd i The meaning of ) is shown in equation (1-1). As shown in Table 2-1, the ACCDS (KSDRMD) of all known subjects KS i ,ksd i All reached 100.00%, however, the ACCDS (KSDRMD) of each known subject KS i ,mixd i All are more accurate than ACCDS (KSDRMD) i ,ksd i Low. Subject No. 18's ACCDS (KSDRMD) i ,mixd i The largest was ACCDS (KSDRMD), with a value of 86.42%, and the fifth subject's ACCDS was the largest. i ,mixd i The smallest value is 51.11%. From equation (1-1), it can be seen that the selection factor CF... i With ACCDS (KSDRMD) i ,mixd i Proportional to the selection factor CF of subject No. 18. i The largest value was 1.33, representing the selection factor CF of subject No. 5. i The minimum value is 0.79. Therefore, the selection factor CF is chosen. i The largest participant, No. 18, was selected as the best known participant (BKS) for deep transfer learning.

[0096] To compare CF with different selection factors i The deep transfer learning performance of the known subjects KS is analyzed by assigning each known subject KS to a different subject. Figure 1 Part 4 involves deep transfer learning to establish the fatigue driving identification model USDRMD. During deep transfer learning, all known participants used the fatigue driving identification model KSDRMD as their pre-trained model. Remaining data from unknown participants were used to test the pre-trained models before and after the transfer learning. Accuracy (ACCDS), Sensitivity (SENDS), Specificity (SPEDS), and F1 score (F1DS) were used to comprehensively evaluate the model's driving state identification performance. To control for variables, all known participants (KS) used the same initial driving time IPT = 30 min and network adjustment coefficient AC = 1 during deep transfer learning.

[0097] Figure 7This demonstrates the accuracy of the KSDRMD fatigue driving identification model, based on known subject KS, in identifying unknown subject driving states before and after transfer learning. (ACCDS accuracy) KSUS and accuracy ACCDS KSUST The left vertical axis represents the accuracy of the fatigue driving identification model KSDRMD in identifying unknown subject driving states before and after deep transfer learning, respectively. i The right vertical axis represents the precision of the ACCDS (Acceptable Correction and Discrimination Dataset). The horizontal axis represents the known participant KS numbers, sorted in ascending order of selection factor scores. All data in the graph are approximate results rounded to two decimal places.

[0098] like Figure 7 As shown, for all known subjects' KS, after deep transfer learning, the ACCDS accuracy of each known subject's KS is... KSUST All are higher than the accuracy of ACCDS KSUS This demonstrates that, based on deep transfer learning, fine-tuning the fatigue driving identification model KSDRMD of known subjects KS using initial-stage driving data from unknown subjects US can improve the identification accuracy for unknown subjects' driving states. Furthermore, the accuracy of each known subject KS (ACCDS) is shown. KSUST With selection factor CF i The trend is increasing and improving. Among them, the selection factor CF... i The largest was the selection factor CF of subject No. 18. i (1.33), Precision of No. 18 subject in ACCDS KSUST It is also the largest, with a value of 92.80%. Selection factor CF i The smallest was the selection factor CF of subject No. 5. i (0.79), the accuracy of the No.5 subject in ACCDS KSUST It is the smallest, with a value of 68.37%. This indicates that the selection factor CF was chosen. i The fatigue driving identification model USDRMD, built using deep transfer learning with a large known subject KS, shows higher accuracy in identifying the driving state of unknown subjects US. Therefore, the selection factor CF should be used preferentially. i A fatigue driving identification model USDRMD was established by using deep transfer learning on a large known subject KS.

[0099] Figure 8 and Figure 9 The sensitivity of the fatigue driving identification model KSDRMD for each known subject KS, namely SENDS and SPEDS, is presented before and after the transfer learning process for identifying the driving state of unknown subjects. Figure 8 In the middle, the right vertical axis represents sensitivity SENDS. KSUS and sensitivity SENDSKSUST SENDS represent the sensitivity of the fatigue driving identification model KSDRMD in identifying the unknown driving state of the subject (US) before and after deep transfer learning. Figure 9 In the middle, the right vertical axis represents the specificity SPEDS. KSUS and specificity SPEDS KSUST SPEDS represents the specificity of the fatigue driving identification model KSDRMD in identifying the unknown driving state of the subject US before and after deep transfer learning. Figure 8 and Figure 9 The meanings of the left vertical axis and horizontal axis are as follows: Figure 7 same.

[0100] Depend on Figure 8 and Figure 9 It is evident that, compared to accuracy-based ACCDS, the effect of deep transfer learning on sensitivity-based SENDS and specificity-based SPEDS is more complex. Figure 8 In the study, after deep transfer learning, the sensitivity SENDS of the fatigue driving identification model KSDRMD for each known subject KS showed different trends. For example, the sensitivity SENDS of subjects No.6, No.7, No.11, and No.17 were different. KSUST Less than sensitivity SENDS KSUS The model's sensitivity decreases after deep transfer learning. The sensitivity of the remaining known subjects' KS is... KSUST All greater than the sensitivity SENDS KSUS Sensitivity SENDS KSUST With selection factor CF i The trend is one of increasing slowly, but not strictly monotonically increasing. The results indicate that the selection factor CF... i The fatigue driving identification model KSDRMD with a large number of known subjects KS typically shows enhanced ability to identify fatigue samples of unknown subjects US after deep transfer learning. However, unilaterally pursuing high sensitivity can lead to increased misjudgments; therefore, the effect of transfer learning needs to be comprehensively analyzed in conjunction with specificity SPEDS.

[0101] exist Figure 9 In the study, the specificity of each known subject's KS (SPEDS) was calculated. KSUST All were higher than the specificity of SPEDS KSUS This indicates that deep transfer learning can increase the specificity (SPEDS) of the fatigue driving identification model KSDRMD in recognizing the driving state of unknown subjects (US), and improve the ability of KSDRMD to identify conscious samples of unknown subjects (US). (Specificity SPEDS) KSUST There is also the option of using the selection factor CF i The increasing trend, and the selection factor CF iLarger specificity of known subjects' KS (SPEDS) KSUST Typically high, its fatigue driving identification model KSDRMD exhibits strong recognition ability for conscious samples of unknown subjects (US) after deep transfer. Although the specificity of subject No. 14, SPEDS, is... KSUST Achieving a specificity of 100% higher than that of subject No. 18 (SPEDS) KSUST (98.50%), however, the sensitivity of subject No. 14 was SENDS KSUST (83.08%) lower than the sensitivity of subject No. 18. SENDS KSUST (89.03%). This indicates that compared to subject No. 18, selecting subject No. 14 for deep transfer learning to build the fatigue driving identification model USDRMD can identify all conscious samples from the unknown subject US, but some fatigue samples are incorrectly classified as conscious samples, resulting in a high false alarm rate. Therefore, the overall sensitivity SENDS KSUST and specificity SPEDS KSUST Selecting subject No. 18 as the best known subject (BKS) and using deep transfer learning to build USDRMD yields better results.

[0102] In addition, in comparison Figure 8 and Figure 9 It can be seen that, compared to the model performance before deep transfer learning, the specificity SPEDS of all known subjects' KS is significantly higher. KSUST The average increase (49.97%) was higher than the sensitivity of KS in all known subjects. KSUST The average improvement was 10.42%. This result indicates that deep transfer learning improves specificity SPEDS better than sensitivity SENDS. After deep transfer learning, the fatigue driving identification model KSDRMD's ability to identify conscious samples of unknown subjects (US) is greatly improved, and the false alarm rate decreases. The reason may be that the driving data of unknown subjects (US) in the initial driving stage reflects the driving fingerprint features of the conscious state. Retraining the fatigue driving identification model KSDRMD with the conscious driving data of unknown subjects (US) allows the fatigue driving identification model KSDRMD to obtain the conscious driving fingerprint feature information of unknown subjects (US), which significantly improves the fatigue driving identification model KSDRMD's ability to identify conscious samples of unknown subjects (US).

[0103] Figure 10 This demonstrates the performance of the fatigue driving identification model KSDRMD, based on a known subject KS, in identifying the F1 score F1DS of an unknown subject US driving state before and after transfer learning. (F1 score F1DS) KSUS F1 scores and F1DS KSUSTThe left and right axes represent the F1 scores (F1DS) of the fatigue driving identification model KSDRMD before and after deep transfer learning, respectively, for identifying the unknown driving state of the subject (US). The right vertical axis represents the F1 score (F1DS), and the left vertical and horizontal axes have the same meanings as... Figure 7 Same. For example... Figure 10 As shown, the F1 scores F1DS of each known subject's KS are... KSUST All are higher than the F1 score F1DS KSUS This indicates that deep transfer learning can improve the comprehensive identification ability of the fatigue driving identification model KSDRMD for known subjects KS to the driving state of unknown subjects US. Furthermore, with the selection factor CF... i The increase in F1 scores, F1DS KSUST There is a tendency to increase, so the selection factor CF i The highest F1 score (F1DS) of the No. 18 participant. KSUST The highest percentage was 93.70%. This indicates that selecting subject No. 18 for deep transfer learning to establish the fatigue driving identification model USDRMD resulted in good overall performance, reducing both false negative and false positive rates.

[0104] according to Figure 1 The method involves treating each participant as an unknown participant (US) and performing selective deep transfer learning, repeated 24 times. Figure 1 The process involves selecting the best known subject (KS) for deep transfer learning to establish a fatigue driving identification model for the unknown subject (US). The fatigue driving identification model BKSDRMD, based on the best known subject (BKS), is then used to establish the fatigue driving identification model USDRMD after deep transfer learning. The accuracy, sensitivity, specificity, and F1 score of this model for identifying the driving state of the unknown subject (US) are represented by the accuracy (ACCDS). BKSUST Sensitivity SENDS BKSUST Specificity SPEDS BKSUST F1 score F1DS BKSUST The average accuracy of repeated modeling results in ACCDS BKSUST Sensitivity SENDS BKSUST Specificity SPEDS BKSUST F1 score F1DS BKSUST The accuracy rates were 89.26%, 87.74%, 90.28%, and 86.74%, respectively, representing improvements of 21.32%, 8.05%, 30.18%, and 20.18% compared to before deep transfer learning. In summary, the fatigue driving identification model USDRMD, built using deep transfer learning with different known subjects KS, exhibits varying performance in identifying unknown subjects' US driving states. The selection factor CF was chosen. i Deep transfer learning with a larger known subject KS typically achieves better recognition of driving states. This may be due to the selection factor CF. iThe larger the value, the smaller the driving pattern difference between the unknown subject (US) and the known subject (KS), the more similar the driving pattern features under different driving states, the less difficult it is to perform deep transfer learning to the unknown subject (US), and the better the recognition effect of the unknown subject (US) driving state after transfer.

[0105] The above description is only a preferred embodiment of the present invention. Therefore, any equivalent changes or modifications made to the structure, features and principles described in the claims of this patent application are included within the protection scope of this patent application.

Claims

1. A selective deep transfer learning method based on driving pattern differences, characterized in that, This method is implemented through the following steps: Step A: Randomly assign multiple known subjects (KS) and multiple unknown subjects (US); Step B: Establish a pre-trained model library for deep transfer learning; Step C: Select the best known subjects (BKS) for deep transfer learning; Step D: Retrain the pre-trained model using data from the unknown subject US to complete deep transfer learning and establish a fatigue driving identification model for the unknown subject US. In step B, a fatigue driving identification model DRMD based on driving patterns is established for each known subject KS. By combining principal component analysis (PCA) with a one-dimensional convolutional neural network (1D-CNN), the driver can be identified as fatigued by automatically extracting and learning driving behavior features. The process of selecting the best known subject (BKS) for deep transfer learning includes: calculating the representation of the unknown subject (US) and one of the known subjects (KS). i Selection factor CF for differences in driving patterns i We selected the best known subject BKS with the smallest difference from the unknown subject US driving pattern for deep transfer learning; The initial phase dataset of unknown subjects (US) was mixed with the test data of each known subject (KS). The mixed test data was then input into the fatigue driving identification model (DRMD) of each known subject (KS), and the driving state identification accuracy for the mixed test dataset was calculated for each subject. The accuracy was then compared with the fatigue driving identification models (KSDRMD) of each known subject (KS). i The accuracy of driving state identification on a test set containing initial data of unknown subjects was assessed using a selection factor. Calculation method: Determine the best known subjects (BKS) for deep transfer learning based on the selection factor scores; The USDRMD model for identifying fatigued driving in unknown subjects (US) was built using deep transfer learning. Step 1: Retrieve the best known subject BKS fatigue driving identification model BKSDRMD and use it as a pre-training model for deep transfer learning. Step 2: Construct a training dataset for deep transfer learning; compensate the unknown subject US's awake driving fingerprint knowledge to the unknown subject US's fatigue driving identification model USDRMD; supplement the training set of the best known subject BKS to form a training dataset. Step 3: Use deep transfer learning to process the fatigue driving identification model BKSDRMD of the best known subject BKS, and establish the fatigue driving identification model USDRMD of the unknown subject US; the fatigue driving identification model BKSDRMD is a fatigue driving identification model based on deep learning; using the fatigue driving identification model BKSDRMD of the best known subject BKS as the pre-trained model, the pre-trained model is retrained using the training dataset, and the network parameter adjustment coefficient AC is designed. Based on the deep transfer learning method, the fatigue driving identification model BKSDRMD of the best known subject BKS is transferred to the fatigue driving identification model USDRMD of the unknown subject US. Step 4: Use the remaining data of unknown subjects (US) to verify the effect of selective deep transfer learning based on driving pattern differences, and analyze the influence of network parameter adjustment coefficient AC and initial driving time IPT. Selection factor The calculation formula is as follows: ； In the formula, Let K be the selection factor score of the i-th known subject, 1≤i≤nk; nk is the number of known subjects with KS. For known subjects The most accurate KSDRMD; For known subjects The data is a mixture of test data and driving data from the initial driving phase of an unknown subject (US). for right The accuracy of driving status recognition.

2. The selective deep transfer learning method based on fingerprint differences according to claim 1, characterized in that, In step A: The data for known subjects KS include: pre-acquired awake driving dataset and fatigue driving dataset; The data for unknown subjects (US) includes: the initial phase dataset and the remaining dataset; The initial phase dataset consists of driving data within the initial driving time (IPT); the remaining dataset consists of the data remaining after removing the initial driving time (IPT). Initial driving time (IPT) ≤ 30 min.

3. The selective deep transfer learning method based on fingerprint differences according to claim 2, characterized in that, For the i-th known subject KS, the full sample set is first divided into a training set and a test set using 5-fold cross-validation. Then, the fatigue driving identification model DRMD, which is a combination model of principal component analysis (PCA) and 1D convolutional neural network (1D-CNN), is trained using the training set. Finally, the fatigue driving identification model DRMD with the highest identification accuracy is tested using the test set and selected as the fatigue driving identification model DRMD for transfer learning of the known subject KS, denoted as fatigue driving model KSDRMD.

4. The selective deep transfer learning method based on fingerprint differences according to claim 3, characterized in that, Using the best known BKS results, a fatigue driving identification model USDRMD for unknown subjects (US) is built using deep transfer learning.

Images