A real-time cumulative workload assessment method for ramp controllers

Abstract

The present invention discloses a method for evaluating the real-time cumulative workload of an apron controller, comprising the following steps: (1) obtaining non-invasive data and preprocessing the data; (2) classifying samples according to an unsupervised clustering algorithm; and dividing the load levels; (3) using feature weights to perform feature selection, and selecting the fusion feature index combination with the highest classification accuracy as the real-time evaluation of the workload of the apron controller; (4) performing feature fusion in multiple ways on the selected features, and obtaining the feature combination with the highest classification accuracy as the feature index combination for the real-time evaluation of the workload of the apron controller; (5) performing numerical conversion on the selected feature index combination, obtaining the numerical representation of the real-time cumulative workload of personnel, and corresponding to the load level represented by it, and analyzing the threshold of the workload of personnel; the present invention improves the evaluation accuracy of the existing workload numerical expression method and improves the scientific nature of the decision-making of load management.

Description

Technical Field

[0001] The present invention relates to the technical field of civil aviation airport operation safety, and in particular to a method for evaluating the real-time cumulative workload of a ramp controller. Background Art

[0002] At present, the field of apron control technology is facing rapid development and challenges. With the transfer of aircraft apron control to domestic airports with a passenger volume of tens of millions, apron control has gradually attracted attention as an emerging field. In terms of technology, some advanced apron control systems have been put into use, including real-time monitoring and coordination systems, automated flight sequencing systems, and decision support systems based on artificial intelligence.

[0003] However, the field of apron control technology still faces some key issues. First, the apron control system needs to better adapt to the special scenarios of different airports, including differences in airport layout, operation scale, and ground resource allocation. Second, the requirements of real-time and accuracy make the system's ability to handle multimodal features a key challenge, requiring more effective data fusion and processing methods. In addition, the workload assessment problem of apron controllers also highlights the technical needs for human-machine collaboration in a complex information environment. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a real-time cumulative workload assessment method for apron controllers. In view of the subjectivity and inability to track the existing apron control load assessment in real time, a real-time assessment method for apron control workload based on multi-type machine learning algorithms is provided in combination with multimodal feature data. The method aims to achieve real-time measurement, tracking and management of the cumulative workload of apron controllers in actual work scenarios, thereby providing a theoretical basis for improving the scientific nature of civil aviation personnel load management strategies.

[0005] Technical solution: The method for evaluating the real-time cumulative workload of a ramp controller described in the present invention comprises the following steps:

[0006] (1) Obtain non-intrusive data of apron controllers in civil aviation airports during their working status and pre-process the data;

[0007] (2) Classify the samples according to the unsupervised clustering algorithm and divide the load levels;

[0008] (3) Use feature weights to select features and eliminate features with small weight values; input the remaining features into multiple machine learning classifiers, select the classifier or ensemble learning method with better evaluation performance and stability, and select the one with the highest classification accuracy as the fusion feature indicator combination for real-time workload evaluation of ramp controllers;

[0009] (4) The selected features are fused in various ways to obtain the feature combination with the highest classification accuracy as the feature indicator combination for real-time workload evaluation of ramp controllers;

[0010] (5) The selected characteristic indicator combination is converted into a numerical value to obtain the numerical representation of the real-time cumulative workload of the personnel, and the threshold of the personnel workload is analyzed by corresponding to the load level represented by it.

[0011] Furthermore, the non-invasive data of step (1) includes: facial video, eye movement features and respiratory waveform; the preprocessing includes aligning data samples and extracting features that change significantly during work and can reflect the changes in the controller's workload.

[0012] Furthermore, in step (1), the facial video includes a real-time fatigue monitoring system and an eye movement data recording and analysis system;

[0013] Among them, the fatigue real-time detection system identifies the eye aspect ratio in real time and records the number of times the alarm information appears and the duration of each alarm; the proportion of frames with an eye aspect ratio less than 0.3 in each 300-frame smoothing window is used to represent the eye closure percentage; the eye aspect ratio and eye closure percentage are weighted to obtain the value M representing the eye feature; the formula is as follows:

[0014]

[0015] M=0.3*EAR+0.7*PERCLOS

[0016] Among them, PERCLOS represents the percentage of eye closure, N represents the number of frames, EAR represents the eye aspect ratio, and M is the numerical representation of eye features.

[0017] Furthermore, the step (2) is specifically as follows: first, determine the optimal number of clusters for sample classification, and cluster the samples accordingly; then, for the clustered results, based on the principle of cumulative increase in personnel workload during the same continuous working period, observe the distribution of sample collection time and the relative size of each feature data in each cluster, and add a load level mapping for the data samples in the cluster.

[0018] Furthermore, the step (3) is specifically as follows: using a random forest algorithm to calculate the weights of all features, and determining the feature retention threshold as 0.04-0.10 based on the calculation results of the feature weights; selecting six basic models including support vector machine, decision tree, random forest, KNN, logistic regression, and neural network, and four integrated learning methods including voting, stacking, bagging, and boosting to form four integrated models, using a 5-fold cross-validation method for training, leaving out 20% of the samples as a test set for horizontal comparison, drawing a learning curve, and using the test set to calculate the classification accuracy, precision-recall ratio, and F1-score of each classifier, and comparing the effects and excellence of each classification algorithm.

[0019] Furthermore, the step (4) is specifically as follows: using a linear early fusion method, a deep joint autoencoder model based on feature selection, a neural network high-dimensional feature transformation, maximum fusion, average fusion, and Bayesian rule fusion to perform feature pairwise coupling and multi-feature coupling, and input them into the better classification algorithm obtained in step (3), compare the classification and prediction accuracies respectively, and obtain the most accurate fusion feature combination.

[0020] Furthermore, the step (5) is specifically as follows: the fusion feature combination with the highest accuracy is subjected to data standardization using the maximum-minimum standardization method, and then numerical conversion is performed, and the formula is as follows:

[0021] Workload=∑(N(C n ))

[0022] Among them, C n Represents the feature values ​​of each dimension after feature fusion, and N represents the normalization method.

[0023] The present invention provides a real-time cumulative workload assessment system for apron controllers, comprising:

[0024] Data preprocessing module: used to obtain non-intrusive data of the apron controllers in the working state of civil aviation airports and pre-process the data;

[0025] Classification module: used to classify samples according to unsupervised clustering algorithm and divide the load levels;

[0026] Selection module: used to select features using feature weights and remove features with smaller weight values; input the remaining features into multiple machine learning classifiers, select the classifier or ensemble learning method with better evaluation performance and stability, and select the one with the highest classification accuracy as the fusion feature indicator combination for real-time workload evaluation of ramp controllers;

[0027] Fusion module: used to perform multiple feature fusions on the selected features to obtain the feature combination with the highest classification accuracy as the feature indicator combination for real-time workload evaluation of apron controllers;

[0028] Conversion module: used to convert the selected characteristic indicator combination into numerical values, obtain the numerical representation of the real-time cumulative workload of personnel, correspond it with the load level it represents, and analyze the threshold of personnel workload.

[0029] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the methods described in the preceding paragraph for evaluating the real-time cumulative workload of an apron controller is implemented.

[0030] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the methods for real-time cumulative workload assessment of a ramp controller.

[0031] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by screening multimodal personnel behavior feature data, focusing on using non-a priori means to complete the mapping of load levels for the data set, and using a variety of combinations and algorithms to deeply integrate multimodal features. It improves the evaluation accuracy of the existing workload numerical expression method, makes personnel workload easier to measure and track, facilitates the identification of potential problems, and improves the scientific nature of load management decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of the present invention;

[0033] Figure 2 It is a result schematic diagram of the present invention. DETAILED DESCRIPTION

[0034] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0035] like Figure 1-2 As shown, an embodiment of the present invention provides a method for evaluating the real-time cumulative workload of an apron controller, comprising the following steps:

[0036] (1) Obtain non-invasive data of apron controllers at civil aviation airports during their working state and pre-process the data; the non-invasive data includes: facial video, eye movement features, and breathing waveforms; pre-processing includes aligning data samples and extracting features that change significantly during work and can reflect the changes in the controller's workload. Facial video includes a real-time fatigue monitoring system and an eye movement data recording and analysis system;

[0037] Among them, the fatigue real-time detection system identifies the eye aspect ratio in real time and records the number of times the alarm information appears and the duration of each alarm; the proportion of frames with an eye aspect ratio less than 0.3 in each 300-frame smoothing window is used to represent the eye closure percentage; the eye aspect ratio and eye closure percentage are weighted to obtain the value M representing the eye feature; the formula is as follows:

[0038]

[0039] M=0.3*EAR+0.7*PERCLOS

[0040] Among them, PERCLOS represents the percentage of eye closure, N represents the number of frames, EAR represents the eye aspect ratio, and M is the numerical representation of eye features.

[0041] The number of consecutive frames with M greater than 0.605 is counted, and the fatigue levels are divided into mild, moderate and severe with 5, 15 and 50 consecutive frames as the interval boundaries; the number of occurrences of the three levels is recorded and displayed in real time; when the number of consecutive frames is greater than or equal to 50, that is, the fatigue level reaches severe, the automatic alarm function is triggered, and a red alarm message will appear at the top of the interface until the M value returns to below the critical value. Data alignment is to align the indicators in the time series and eliminate the indicators that change less over time. The equipment for collecting eye movement characteristics and respiratory waveforms includes eyeglass eye trackers and physiological monitoring equipment.

[0042] (2) Classify the samples according to the unsupervised clustering algorithm and divide the load levels as follows: First, determine the optimal number of clusters for sample classification and cluster the samples accordingly; then, for the clustered results, observe the distribution of sample collection time and the relative size of each feature data in each cluster based on the principle of cumulative increase in personnel workload during the same continuous working period, and add a load level mapping for the data samples in the cluster.

[0043] (3) Use feature weights for feature selection and eliminate features with smaller weight values; input the remaining features into multiple machine learning classifiers, select the classifier or ensemble learning method with better evaluation performance and stability, and select the one with the highest classification accuracy as the fusion feature indicator combination for real-time evaluation of the workload of apron controllers; specifically: use the random forest algorithm to calculate the weights of all features, and determine the feature retention threshold as 0.04 based on the calculation results of the feature weights; select six basic models including support vector machine, decision tree, random forest, KNN, logistic regression, and neural network, and four ensemble learning methods including voting, stacking, bagging, and boosting to form four ensemble models, use the 5-fold cross-validation method for training, set aside 20% of the samples as the test set for horizontal comparison, draw the learning curve, and use the test set to calculate the classification accuracy, precision-recall ratio, and F1-score of each classifier, and compare the effects and excellence of each classification algorithm.

[0044] (4) The selected features are fused in various ways to abstract high-dimensional features that are concise and can present multiple differences, and obtain feature combinations of different dimensions and forms. The feature combinations are input into the classifiers respectively, and the feature combination with the highest classification accuracy is selected as the feature indicator combination for real-time evaluation of the workload of the apron controller; specifically, the linear early fusion method, the deep joint autoencoder model based on feature selection, the neural network high-dimensional feature conversion, the maximum fusion, the average fusion, and the Bayesian rule fusion are used to couple the features in pairs and multiple features, and input them into the better classification algorithm obtained in step (3), and the classification and prediction accuracies are compared respectively to obtain the fusion feature combination with the highest accuracy.

[0045] (5) Perform numerical conversion on the selected feature index combination to obtain the numerical representation of the real-time cumulative workload of the personnel, and analyze the threshold of the workload of the personnel in accordance with the load level represented by it. The details are as follows: The fusion feature combination with the highest accuracy is normalized using the maximum-minimum normalization method, and then numerically converted. The formula is as follows:

[0046] Workload=∑(N(C n ))

[0047] Among them, C n Represents the feature values ​​of each dimension after feature fusion, and N represents the normalization method.

[0048] After obtaining the real-time cumulative value of the workload of each sample, combined with the corresponding workload level, excluding abnormal values, sorting the data in time series, and selecting the median of the overlapping part of the previous level and the next level as the threshold of the load level. In this example, a numerical result is calculated for each unit of workload. Take the values ​​of the first and last 8% of the adjacent clusters, and according to the above method and process the values ​​into percentages, select 25.2% and 65.6% as the critical values ​​of low and medium workloads, medium and high workloads, respectively. In the samples of this experiment, the critical values ​​appear at the 21st minute and the 60th minute, respectively.

[0049] The embodiment of the present invention further provides a system for evaluating the real-time cumulative workload of a ramp controller, comprising:

[0050] Data preprocessing module: used to obtain non-intrusive data of the apron controllers in the working state of civil aviation airports and pre-process the data;

[0051] Classification module: used to classify samples according to unsupervised clustering algorithm and divide the load levels;

[0052] Selection module: used to select features using feature weights and remove features with smaller weight values; input the remaining features into multiple machine learning classifiers, select the classifier or ensemble learning method with better evaluation performance and stability, and select the one with the highest classification accuracy as the fusion feature indicator combination for real-time workload evaluation of ramp controllers;

[0053] Fusion module: used to perform multiple feature fusions on the selected features to obtain the feature combination with the highest classification accuracy as the feature indicator combination for real-time workload evaluation of apron controllers;

[0054] Conversion module: used to convert the selected characteristic indicator combination into numerical values, obtain the numerical representation of the real-time cumulative workload of personnel, correspond it with the load level it represents, and analyze the threshold of personnel workload.

[0055] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, any one of the methods for real-time cumulative workload assessment of a ramp controller is implemented.

[0056] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any one of the methods for real-time cumulative workload assessment for apron controllers.

Claims

1. A method for evaluating the real-time cumulative workload of an apron controller, characterized in that: The following steps are involved: (1) Obtain non-intrusive data of apron controllers at civil aviation airports during their working status and pre-process the data; Non-invasive data includes: facial video, eye movement features and respiratory waveforms; pre-processing includes aligning data samples and extracting features that change significantly during work and can reflect the changes in the controller's workload; facial video includes a real-time fatigue monitoring system and an eye movement data recording and analysis system; Among them, the fatigue real-time detection system identifies the eye aspect ratio in real time and records the number of times the alarm information appears and the duration of each alarm; the proportion of frames with an eye aspect ratio less than 0.3 in each 300-frame smoothing window is used to represent the eye closure percentage; the eye aspect ratio and eye closure percentage are weighted to obtain the value M representing the eye feature; the formula is as follows: ； ； Among them, PERCLOS represents the percentage of eye closure, N represents the number of frames, EAR represents the aspect ratio of the eye, and M is the numerical representation of the eye feature; (2) Classify the samples according to the unsupervised clustering algorithm and divide the load levels. The details are as follows: First, determine the optimal number of clusters for sample classification and cluster the samples accordingly. Then, for the clustered results, observe the distribution of sample collection time and the relative size of each feature data in each cluster according to the principle of cumulative increase in personnel workload during the same continuous working period, and add load level mapping for the data samples in the cluster. (3) Use feature weights for feature selection and remove features with smaller weight values; input the remaining features into multiple machine learning classifiers, select the classifier or ensemble learning method with better evaluation performance and stability, and select the one with the highest classification accuracy as the fusion feature indicator combination for real-time evaluation of the workload of apron controllers; specifically: use the random forest algorithm to calculate the weights of all features, and determine the feature retention threshold as 0.04-0.10 based on the calculation results of the feature weights; select six basic models including support vector machine, decision tree, random forest, KNN, logistic regression, and neural network, and four ensemble learning methods including voting, stacking, bagging, and boosting to form four ensemble models, use the 5-fold cross-validation method for training, set aside 20% of the samples as the test set for horizontal comparison, draw the learning curve, and use the test set to calculate the classification accuracy, precision-recall ratio, and F1-score of each classifier, and compare the effects and excellence of each classification algorithm; (4) The selected features are subjected to feature fusion in a variety of ways to obtain the feature combination with the highest classification accuracy as the feature indicator combination for real-time workload evaluation of the ramp controller; specifically: linear early fusion method, deep joint autoencoder model based on feature selection, neural network high-dimensional feature conversion, maximum fusion, average fusion, Bayesian rule fusion are used to couple features in pairs and multiple features, and the features are input into the better classification algorithm obtained in step (3), and the classification and prediction accuracies are compared to obtain the fusion feature combination with the highest accuracy; (5) Perform numerical conversion on the selected feature index combination to obtain the numerical representation of the real-time cumulative workload of the personnel, and analyze the threshold of the workload of the personnel by corresponding to the load level represented by it; the details are as follows: the fusion feature combination with the highest accuracy is normalized using the maximum-minimum normalization method, and then numerical conversion is performed, the formula is as follows: ； in, Represents the feature values ​​of each dimension after feature fusion, and N represents the normalization method.

2. A real-time cumulative workload assessment system for ramp controllers, characterized in that: include: Data preprocessing module: used to obtain non-intrusive data of the apron controllers in the working state of civil aviation airports and pre-process the data; Non-invasive data includes: facial video, eye movement features and respiratory waveforms; pre-processing includes aligning data samples and extracting features that change significantly during work and can reflect the changes in the controller's workload; facial video includes a real-time fatigue monitoring system and an eye movement data recording and analysis system; Among them, the fatigue real-time detection system identifies the eye aspect ratio in real time and records the number of times the alarm information appears and the duration of each alarm; the proportion of frames with an eye aspect ratio less than 0.3 in each 300-frame smoothing window is used to represent the eye closure percentage; the eye aspect ratio and eye closure percentage are weighted to obtain the value M representing the eye feature; the formula is as follows: ； ； Among them, PERCLOS represents the percentage of eye closure, N represents the number of frames, EAR represents the aspect ratio of the eye, and M is the numerical representation of the eye feature; Classification module: used to classify samples according to the unsupervised clustering algorithm and divide the load levels. The details are as follows: first, determine the optimal number of clusters for sample classification, and cluster the samples accordingly; then, for the clustered results, based on the principle of cumulative increase in personnel workload during the same continuous working period, observe the distribution of sample collection time and the relative size of each feature data in each cluster, and add load level mapping for the data samples in the cluster; Selection module: used to select features using feature weights and remove features with smaller weight values; input the remaining features into multiple machine learning classifiers, select classifiers or ensemble learning methods with better evaluation performance and stability, and select the one with the highest classification accuracy as the fusion feature indicator combination for real-time evaluation of the workload of apron controllers; specifically as follows: use the random forest algorithm to calculate the weights of all features, and determine the feature retention threshold as 0.04-0.10 based on the calculation results of the feature weights; select six basic models including support vector machine, decision tree, random forest, KNN, logistic regression, and neural network, and four ensemble learning methods including voting, stacking, bagging, and boosting to form four ensemble models, use the 5-fold cross-validation method for training, set aside 20% of the samples as a test set for horizontal comparison, draw a learning curve, and use the test set to calculate the classification accuracy, precision-recall ratio, and F1-score of each classifier, and compare the effects and excellence of each classification algorithm; Fusion module: used to fuse the selected features in various ways to obtain the feature combination with the highest classification accuracy as the feature indicator combination for real-time workload evaluation of apron controllers; specifically: linear early fusion method, deep joint autoencoder model based on feature selection, neural network high-dimensional feature conversion, maximum fusion, average fusion, Bayesian rule fusion are used to couple features in pairs and multiple features, and input them into the better classification algorithm obtained by the selection module, and the classification and prediction accuracies are compared to obtain the fusion feature combination with the highest accuracy; Conversion module: used to convert the selected feature index combination into numerical values, obtain the numerical representation of the real-time cumulative workload of personnel, and correspond it to the load level represented by it, and analyze the threshold of personnel workload; specifically: the fusion feature combination with the highest accuracy is normalized using the maximum-minimum normalization method, and then converted into numerical values. The formula is as follows: ； in, Represents the feature values ​​of each dimension after feature fusion, and N represents the normalization method.

3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the method for real-time cumulative workload assessment for apron controllers according to claim 1 is implemented.

4. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for real-time cumulative workload assessment for apron controllers according to claim 1 is implemented.

Images