Night vision equipment user cognitive state evaluation method and storage medium

Abstract

The invention discloses a night vision equipment user cognitive state evaluation method and a storage medium, and the method is based on five classic machine learning classification evaluation models, and integrates multiple classifiers according to the multi-modal physiological data of night vision equipment users under night vision conditions in combination with a Bayesian estimation method, thereby improving the cognitive state evaluation accuracy of the night vision equipment users. A more complete and accurate data mapping model is constructed, the cognitive state of the night vision equipment personnel is evaluated, and the final purpose is to effectively improve the accuracy of classification evaluation of the multi-modal physiological data. According to the method, the accuracy of a classification evaluation result is effectively improved, a powerful guarantee is provided for subsequent data analysis, and the method has important reference value in the aspects of operator interface system design, intelligent upgrading of a man-machine interaction mode and man-machine early warning and has a remarkable promotion effect on development of man-machine interaction.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning and ergonomics, and in particular to a classification, evaluation and optimization method for multimodal physiological data of night vision equipment users, and specifically to a cognitive state evaluation method and storage medium for night vision equipment users. Background Art

[0002] By combining and integrating several weak classifiers through some method, a strong classifier can be obtained, i.e., a multi-classifier ensemble, which helps improve accuracy. Common multi-classifier ensemble methods include averaging, voting, and learning. Averaging and voting are relatively simple, as they simply average or aggregate the results without adding any additional information, and thus fail to demonstrate the advantages of high-performance classifiers. Learning, on the other hand, is relatively complex and requires pre-training and optimization to obtain a new strong classifier before performing classification predictions.

[0003] Currently, most cognitive status assessment methods for personnel wearing night vision equipment (such as pilots) use a single machine learning algorithm to construct a mapping relationship model from physiological data to cognitive status. Such an algorithm structure may not be able to accurately classify and evaluate multimodal physiological data, and therefore cannot meet the needs of accurately assessing the cognitive status of night vision equipment personnel during missions. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for evaluating the cognitive state of night vision equipment users. The method is based on five classic machine learning classification evaluation models. It targets the multimodal physiological data of night vision equipment users under night vision conditions, combines the Bayesian estimation method to integrate multiple classifiers, and constructs a more complete and accurate data mapping model to realize the evaluation of the cognitive state of night vision equipment users. The ultimate goal is to effectively improve the accuracy of the classification evaluation of multimodal physiological data.

[0005] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0006] Step 1: Select an appropriate machine learning classification and evaluation model, perform appropriate parameter tuning, perform classification and evaluation on the pilot's multimodal physiological dataset, and compile statistical results. This paper selects five classic machine learning classification and evaluation models: support vector machine, K-nearest neighbor, decision tree, random forest, and BP neural network.

[0007] Step 2: Calculate the confusion matrix of each machine learning classification evaluation model, and then calculate the evaluation indicators of each model through the confusion matrix, including error rate, accuracy, precision, recall rate and F-value.

[0008] Step 3: Based on the evaluation indicators of each classification evaluation model and combined with Bayesian statistical theory, a combined classifier mathematical model is constructed:

[0009]

[0010] Where P(A k |B1,B2,...,B n ) is the final classification evaluation result, P(A j |B1,B2,...,B n ) is the combined fusion probability of each classification evaluation model, P(B1,B2,...,B n |A j ) is the uncertainty of the target classification description, that is, the accuracy of the evaluation indicators of each classification evaluation model, P(A j ) is the ratio of the jth true classification sample to the total samples P(B1,B2,...,B n ).

[0011] A computer-readable storage medium stores a computer program, which is executed by a processor to implement the steps of a method for evaluating the cognitive state of a night vision equipment user according to the present invention.

[0012] Beneficial effects of the present invention:

[0013] The multi-classifier integration method based on Bayesian estimation of the present invention can give higher conditional probabilities to classifiers with high classification performance, build a more complete and accurate data mapping model, thereby effectively improving the accuracy of classification evaluation results and providing strong guarantees for subsequent data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 : Schematic diagram of confusion matrix.

[0015] Figure 2 :Flowchart of multi-classifier ensemble based on Bayesian estimation method.

[0016] Figure 3 : Test set confusion matrix of each classification model.

[0017] Figure 4 : Histogram of evaluation index results of each classification model.

[0018] Figure 5 : Confusion matrix diagram of multiple classifiers based on Bayesian estimation.

[0019] Figure 6 : Histogram of evaluation index results of each classification model. DETAILED DESCRIPTION

[0020] The following describes in detail embodiments of the present invention. The embodiments are exemplary and intended to explain the present invention, but are not to be construed as limiting the present invention.

[0021] A method for evaluating the cognitive state of night vision equipment personnel based on Bayesian estimation comprises the following steps:

[0022] Step 1: Classical Machine Learning Classification Evaluation Model

[0023] (1) Support Vector Machine

[0024] Support Vector Machine (SVM) is a common machine learning algorithm primarily used for classification and regression problems. The basic idea behind SVM is to find a dividing line, or maximum margin hyperplane, that maximizes the margin between different classes, separating data samples from different categories. The data samples in an SVM are called support vectors; they are the data samples closest to the boundary between different classes on the decision boundary.

[0025] The basic steps of SVM are as follows:

[0026] (a) Input the training data set and select an appropriate kernel function;

[0027] (b) Calculate the similarity matrix between training samples based on the selected kernel function;

[0028] (c) Solve the optimal hyperplane according to the selected optimization objective function;

[0029] (d) Classify new samples.

[0030] (2) K nearest neighbors

[0031] K-nearest neighbor (KNN) is a simple and classic machine learning classification algorithm that classifies samples by calculating the distance (usually Euclidean distance) between the data to be classified and the classified samples. KNN performs the following steps for each point to be classified:

[0032] (a) Calculate the distance between the point to be classified and all classified points;

[0033] (b) Sort by distance, usually in ascending order;

[0034] (c) Select the first k classified points that are closest to the point to be classified, where the value of k affects the classification performance of the classifier;

[0035] (d) Count the total number of each category in the k points selected above;

[0036] (e) The category with the highest frequency among the above k points is used as the category of the point to be classified.

[0037] (3) Decision Tree

[0038] Decision Tree (DT) is a common machine learning method. Decision Tree, also known as Decision Tree, is an important classification and regression method in data mining technology. It is a predictive analysis model expressed in the form of a tree structure (including binary tree and multi-branch tree). Decision Tree is an algorithm for supervised learning. The generation process of a decision tree is as follows:

[0039] (a) Starting from the root node, the information purity of the relationship between each feature of the sample and the classification result is calculated;

[0040] (b) Node splitting. Based on the purity of each feature in the node, the feature with the highest purity is selected as the decision rule, and a node is split into two child nodes based on the value of the feature;

[0041] (c) Repeat step 2 for the internal child nodes, and the decision tree continues to grow;

[0042] (d) Stop splitting. When the purity of the information in the node is very low or the classification results within the node are consistent and no classification is required, the node stops splitting. When all nodes no longer split, the decision tree stops growing and the decision tree training is completed.

[0043] (4) Random Forest

[0044] A random forest (RF) is essentially a collection of decision trees where each tree is slightly different from the other trees. The idea behind random forests is that some trees may do relatively well in prediction, but some trees may overfit part of the data. If many trees are built, some trees may perform well and some trees may overfit, and the amount of overfitting can be reduced by averaging the results while retaining the predictive power of the tree. To implement this strategy, many decision trees need to be built. Each tree should do an acceptable job of predicting the target, and the trees are different from each other. The trees in a random forest are random in two ways: by selecting the data points used to build the tree and by selecting the features in each split test. The algorithmic steps of a random forest are as follows:

[0045] (a) Extract training sets from the original sample set. In each round, n training samples are extracted from the original sample set using the bootstrapping method (sampling with replacement). A total of k rounds of extraction are performed to obtain k training sets. (The k training sets are independent of each other)

[0046] (b) Each time a training set is used to obtain a model, k training sets are used to obtain a total of k models.

[0047] (c) For classification problems: use voting to obtain the classification result of the k models obtained in the previous step; for regression problems, calculate the mean of the above models as the final result.

[0048] (5) BP neural network

[0049] The BP (Back Propagation, BP) neural network is currently the most widely used artificial neural network. It is a type of artificial neural network. The BP neural network is a multi-layer feedforward network trained by error back propagation. Its algorithm is called the BP algorithm. Its basic idea is the gradient descent method, which uses gradient search technology to minimize the mean square error between the network's actual output value and the expected output value. The BP neural network process is mainly divided into two stages. The first stage is the forward propagation of the signal, from the input layer through the hidden layer, and finally to the output layer. The second stage is the backward propagation of the error, from the output layer to the hidden layer, and finally to the input layer, sequentially adjusting the weights and biases from the hidden layer to the output layer, and from the input layer to the hidden layer.

[0050] Step 2: Classification model evaluation metrics

[0051] The confusion matrix indicator is a method for evaluating the effectiveness of classification models. For a classification problem, instances are divided into positive (Positive) or negative (Negative). When using a classifier for classification, there are four situations: if the instance is positive and is evaluated as positive, it is recorded as a true positive (True Positive, TP); if the instance is positive but is evaluated as negative, it is recorded as a false negative (False Negative, FN); if the instance is negative but is evaluated as positive, it is recorded as a false positive (False Positive, FP); if the instance is negative but is evaluated as negative, it is recorded as a true negative (True Negative, TN). A matrix can be used to express the indicator parameters in all binary problems, namely the confusion matrix (such as Figure 1 shown).

[0052] Based on the above four values, we can calculate four confusion matrix indicators: error rate (ERR), accuracy (ACC), precision (PRE), recall (REC), and F-value. The specific definitions and explanations are as follows:

[0053] (1) Accuracy: Accuracy is defined as the ratio of the number of correctly evaluated samples to the total number of evaluated samples. Its expression is:

[0054]

[0055] (2) Error rate: The error rate is defined as the ratio of the number of samples with evaluation errors to the total number of samples evaluated. Its expression is:

[0056]

[0057] (3) Recall rate: Compared with precision rate, recall rate is based on samples. It is defined as the number of positive samples that are evaluated correctly. The expression is:

[0058]

[0059] (4) Precision: Precision refers to the evaluation results. It is defined as the number of samples evaluated as positive that are truly positive samples. The expression is:

[0060]

[0061] (5) F1 score: The F1 score is an evaluation metric that combines precision and recall. It is calculated based on the harmonic mean of precision and recall, and it gives greater weight to lower values. In this way, only when both precision and recall are high can the classification achieve a high F1 score. The calculation formula of F1 score is as follows:

[0062]

[0063] Step 3: Multi-classifier ensemble method based on Bayesian estimation

[0064] Bayesian statistical theory believes that people's estimates of the occurrence of an event are different before and after the test, and different test results have different impacts on people's final estimates. The Bayesian estimation process is the continuous correction of prior knowledge to a posteriori knowledge during the re-testing process. The basic theoretical formula of the Bayesian statistical estimation method is as follows.

[0065] Prior knowledge: P(A1), P(A2), ..., P(A n ) represents events A1, A2, ..., A n The probability of occurrence, which is knowledge before the experiment is called "prior knowledge".

[0066] A posteriori knowledge: Due to the appearance of a test result B, people’s understanding of events A1, A2, ..., A n The knowledge of what happened after the test is called "posterior knowledge". n The probability of occurrence is expressed as conditional probability: P(A1|B), P(A2|B),…, P(A n |B), and obviously P(A i |B)≥0,∑P(A i|B)=1.

[0067] The conditional probability formula is:

[0068]

[0069] The total probability formula is:

[0070]

[0071] Where A i is a partition of the sample space, namely A i are mutually exclusive events and ∑P(A i )=1.

[0072] Bayesian formula: for a set of mutually exclusive events A1, A2, ..., A n When the measurement result is B, A i The probability of occurrence is:

[0073]

[0074] Combined with the above Bayesian formula, the evaluation indicators of the classic machine learning classification evaluation model are used to complete the integrated fusion results of multiple classifiers. The process is as follows: Figure 2 shown.

[0075] First, different classifiers are used to classify and evaluate the unknown cognitive state level. j is the cognitive state level evaluation result given by the jth classifier. The actual cognitive state level target is composed of A1, A2, ..., A n Indicates that its prior probability P(A i ) represents the ratio of the actual number of samples of the i-th cognitive state level to the total number of samples, which can be obtained through the confusion matrix, and A i satisfy n is the total number of classifiers. Each classifier gives a classification description of the cognitive state level based on the evaluation results using the classifier's evaluation index. According to the confusion matrix in the classifier's evaluation index, the uncertainty P(B j |A i ). P(B j |A i ) indicates that the actual cognitive status of the classification is A i Under the conditions of evaluating B j The probability of B is evaluated in the confusion matrix. j The actual category of the class sample is A i The proportion of all samples in the dataset. From this, the target fusion probability can be calculated through Bayesian estimation, thereby obtaining the combined classification result of cognitive state registration. The steps are as follows:

[0076] (1) Obtain the classification results B1, B2, ..., B of the cognitive state level output by each classifier n ;

[0077] (2) The accuracy calculated by the evaluation index of the model is the uncertainty of each classifier's description of the cognitive state level, which is P(B j |A i ),i=1,2…,n。

[0078] (3) Calculate the combined fusion probability of cognitive state levels:

[0079]

[0080] And because the classification evaluation results of each classifier are independent of each other, we have:

[0081]

[0082] (4) Select the maximum Bayesian estimated probability to obtain the cognitive state combination classification result:

[0083]

[0084] Example 1

[0085] This example uses the background of pilot combat missions under night vision conditions to collect multimodal physiological data of pilots wearing night vision equipment in a simulated cockpit. This data is combined with cognitive state level labels to form a cognitive state dataset for night vision equipment personnel. The method of the present invention is used to improve the accuracy of cognitive state level prediction for night vision equipment personnel. The method specifically includes the following steps:

[0086] Step 1: Calculate the parameters of each classic machine learning classification evaluation model

[0087] Based on the night vision equipment personnel cognitive status dataset, various classic machine learning classification evaluation models were used for parameter tuning and optimization. The parameters of each classic machine learning classification evaluation model are as follows:

[0088] (1) Support vector machine model parameters (as shown in the following table)

[0089]

[0090]

[0091] (2) K-nearest neighbor model parameters (as shown in the table below)

[0092] Parameter name Parameter value Search Algorithms auto Number of leaves 30 Number of nearest neighbors 5 Neighbor sample weight function uniform Vector distance algorithm euclidean

[0093] (3) Decision tree model parameters (as shown in the table below)

[0094] Parameter name Parameter value Node split evaluation criteria gini Feature partition point selection criteria best The maximum feature ratio considered when splitting None Minimum number of samples for internal node splitting 2 Minimum number of samples for a leaf node 1 The minimum weight of samples in the leaf node 0 Maximum number of leaf nodes 50 Maximum depth of the tree 10 Node partition impurity threshold 0

[0095] (4) Random forest model parameters (as shown in the table below)

[0096]

[0097]

[0098] (5) BP neural network model parameters (as shown in the following table)

[0099] Parameter name Parameter value Activation Function logistic Solver lbfgs Learning rate 0.001 L2 regularization term 1 Number of iterations 2000 Number of neurons in the first hidden layer 60

[0100] Step 2: Calculate the evaluation indicators of each classification model

[0101] Usually, the "80 / 20" split is used when dividing the test set and training set in the model, that is, 80% of the data is randomly selected as the training set for model training, and the remaining 20% of the data is used as the test set to evaluate the model results. After training each model, combined with the test set for classification evaluation, the test set confusion matrix of each classification model can be obtained, such as Figure 3 shown.

[0102] By calculating the confusion matrix, we can obtain the results of various evaluation indicators. The results of the five classification model evaluation indicators are shown in the following table.

[0103] method Accuracy ACC Error Rate ERR Recall rate REC Precision F1 Support Vector Machine 0.773 0.227 0.773 0.811 0.777 KNN 0.886 0.114 0.886 0.886 0.88 DT 0.869 0.131 0.869 0.874 0.868 RF 0.897 0.103 0.897 0.9 0.89 BP 0.904 0.096 0.904 0.904 0.903

[0104] Draw the above table into a histogram, as shown below: Figure 4 shown.

[0105] Step 3: Calculate the Bayesian estimation results of the multi-classifier ensemble model

[0106] Combining the above five classification model evaluation indicators, the test set confusion matrix obtained by the multi-classifier integration method based on Bayesian estimation is as follows Figure 5 shown.

[0107] By calculating the confusion matrix, we can integrate the evaluation index results of the multi-classifier model and then compare them with the evaluation index results of the above five classification models. The classification model evaluation index results are shown in the following table:

[0108]

[0109]

[0110] Draw the classification evaluation results in the above table into a histogram, as shown in the following example: Figure 6 shown.

[0111] As can be seen from the figure, among the evaluation results of each individual classification model, the BP model has the highest accuracy of 90.4%, followed by RF at 89.7%, DT at 86.9%, KNN at 88.6%, and SVM at 77.3%. However, after evaluation of the classifier ensemble model, the accuracy is as high as 94%, which is higher than the evaluation results of each individual classifier model, and the other indicators of the classifier ensemble model are higher than the evaluation results of each individual classifier model. Therefore, the method proposed in the present invention effectively improves the accuracy of the classification evaluation results and provides a strong guarantee for subsequent data analysis.

Claims

1. A method for assessing the cognitive status of a person using night vision equipment, characterized in that: The following steps are involved: Step 1: Select several machine learning classification evaluation models; Step 2: Calculate the confusion matrix of each machine learning classification evaluation model, and then calculate the evaluation index of each model through the confusion matrix; Step 3: Based on the evaluation indicators of each classification evaluation model and combined with Bayesian statistical theory, a combined classifier mathematical model is constructed: Among them, P(A k |B1,B2,...,B n ) is the final classification evaluation result, P(A j |B1,B2,...,B n ) is the combined fusion probability of each classification evaluation model, P(B1,B2,...,B n |A j ) is the uncertainty of the target classification description, P(A j ) is the ratio of the jth true classification sample to the total samples P(B1,B2,...,B n ); Step 4: Calculate the target fusion probability through Bayesian estimation to obtain the cognitive state level combination classification result.

2. The evaluation method according to claim 1, wherein: The machine learning classification evaluation models include support vector machine, K nearest neighbor, decision tree, random forest and BP neural network classification evaluation models.

3. The evaluation method according to claim 3, wherein: The parameters for tuning the machine learning classification evaluation model are shown in the following table:

4. The evaluation method according to claim 3, wherein: The evaluation indicators include error rate, accuracy rate, precision rate, recall rate and F1 value.

5. The evaluation method according to claim 4, characterized in that In step 2, the confusion matrix of each classification result is obtained by combining the machine learning classification evaluation model after tuning and parameter adjustment. The evaluation indicators of each model are calculated through the confusion matrix as follows: Among them, TP is the true positive class, FN is the false negative class, FP is the false positive class, and TN is the true negative class.

6. The evaluation method according to claim 5, wherein: In step 3, different classifiers are used to classify and evaluate the unknown cognitive state level. j is the cognitive state level evaluation result given by the jth classifier, and the actual cognitive state level target is composed of A1, A2, ..., A n Indicates that its prior probability P(A i ) represents the ratio of the actual number of samples of the i-th cognitive state level to the total samples, which can be obtained through the confusion matrix of the classification evaluation results of each classification evaluation model, and A i satisfy n is the total number of classifiers.

7. The evaluation method according to claim 5, wherein: In step 3, each classifier uses the evaluation index of the classifier to give a classification description of the cognitive state level according to the evaluation results. According to the confusion matrix in the classifier evaluation index, the uncertainty P(B j |A i ), where P(B j |A i ) indicates that the classification is at the level of A in actual cognitive status i Under the conditions of evaluating B j The probability of B is evaluated in the confusion matrix. j The actual category of the class sample is A i The proportion of all samples.

8. The evaluation method according to claim 7, characterized in that In step 3, according to the conditional probability formula, total probability formula and Bayesian formula in Bayesian theory, it is:

9. The evaluation method according to claim 8, characterized in that The target fusion probability is calculated through Bayesian estimation to obtain the cognitive state level combination classification results, including: (1) Obtain the classification results B1, B2, ..., B of the cognitive state level output by each classifier n ; (2) The accuracy calculated by the evaluation index of the model is the uncertainty of each classifier's description of the cognitive state level is P(B j |A i ),i=1,2…,n; (3) Calculate the combined fusion probability P(A) of the cognitive state level i |B1,B2,…,B n ): (4) Select the maximum Bayesian estimated probability to obtain the cognitive state combination classification result P(A k |B1,B2,…,B n ):

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of a method for assessing the cognitive state of a night vision equipment user as described in any one of claims 1 to 9.

Images