Fatigue prediction method for tower controllers based on work type and circadian rhythm

Abstract

The present application relates to the technical field of air traffic management, and discloses a tower controller fatigue prediction method based on work type and circadian rhythm, comprising: obtaining a face video frame of a controller in a tower, and calculating a PERCLOS time sequence representing an eyelid closure proportion; obtaining an exogenous factor related to a control task; encoding the PERCLOS time sequence and the exogenous factor, and inputting them into a trained time sequence neural network model, wherein the time sequence neural network model is configured to apply different weights to the work type, the circadian rhythm and the work demand respectively for differential modeling; and based on the output of the time sequence neural network model, generating a fatigue level curve covering a preset prediction time window. The present application aims to solve the technical problem of the existing fatigue monitoring method, which has a lagging response and is insufficiently adaptable to dynamic working scenarios, thereby providing a method capable of actively and accurately predicting the future fatigue development trend of a controller.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of air traffic management, and in particular to a method for predicting the fatigue of an on-duty controller based on work type and circadian rhythm. BACKGROUND

[0002] Air traffic control (ATC) is a key link to ensure the safety and efficiency of national airspace, and controllers are the core of the system. Due to high work intensity, high mental stress, and irregular shift system, the fatigue of controllers on duty has become an important hidden danger affecting air traffic safety. Therefore, it is of great practical significance to objectively and real-time monitor and manage the fatigue state of controllers.

[0003] In related technologies, there are schemes that use computer vision technology to monitor personnel fatigue. For example, a common method (as described in the public document "A Data-Driven Approach for Fatigue Detection of Air Traffic Controllers") is to capture the facial video of the operator through a camera and calculate the percentage of eyelid closure over the pupil (PERCLOS) as a physiological indicator of fatigue. When the instantaneous value of this indicator exceeds a pre-set fixed threshold, the system will trigger a fatigue alarm.

[0004] However, in the related art, how to achieve more accurate and forward-looking fatigue management has been a persistent technical problem. Specifically, existing technical solutions often face the following challenges:

[0005] 1. Response lag: Existing methods are mostly "passive" detection, i.e., an alarm is issued only after the fatigue state has occurred. This response mode lacks lead time, and for the air traffic control field which requires high forward-looking safety management, the window period for risk intervention is very limited.

[0006] 2. Lack of adaptability: The occurrence and development of fatigue is a complex process influenced by multiple factors. For example, night shifts (circadian rhythm), high-density flights (work demand), and different control positions (work type) have significantly different effects on controller fatigue. Existing methods generally use a "one-size-fits-all" fixed threshold judgment standard, failing to combine these dynamic changing context information for differentiated analysis, leading to false negatives or false positives in complex and variable real-world scenarios, and the prediction accuracy and reliability need to be improved.

[0007] Therefore, there is an urgent need in the art for a new fatigue prediction technology that can overcome the above-mentioned defects, realize the transition from passive detection to active prediction, and adapt to different work scenarios and individual differences. SUMMARY

[0008] In order to solve the problems existing in the prior art, the present application aims to solve the technical problems of response lag and insufficient adaptability to dynamic working scenarios in the prior art fatigue monitoring method, thereby providing a method capable of actively and accurately predicting the future fatigue development trend of the controller.

[0009] To achieve the above-mentioned purpose, the present application provides a controller fatigue prediction method based on working type and circadian rhythm, which comprises the following steps:

[0010] Obtaining the face video frame of the controller in the post, and calculating the PERCLOS time series representing the proportion of eyelid closure;

[0011] Obtaining exogenous factors related to the control task, the exogenous factors including one or more of working type, circadian rhythm and working demand;

[0012] Encoding the PERCLOS time series and the exogenous factors to obtain input features, and inputting the input features into a trained time series neural network model, the time series neural network model being configured to apply different weights to the working type, the circadian rhythm and the working demand respectively for differential modeling;

[0013] Based on the output of the time series neural network model, generating a fatigue level curve covering a preset prediction time window.

[0014] Preferably, the encoding of the exogenous factors comprises:

[0015] The working type is one-hot encoded;

[0016] The circadian rhythm is encoded using a periodic function;

[0017] The working demand is numerically encoded using at least one of the number of flights and / or the count of conflict-related events in a preset time window as a proxy feature.

[0018] Preferably, the time series neural network model comprises a factor gating structure, which generates channel-level weights based on a factor vector of at least one of the exogenous factors, and differentially weights the time series part in the input features before time series modeling.

[0019] Preferably, the time series neural network model further comprises an individualized adaptation layer, which uses initial parameters based on population stratification prior in cold start, and performs online fine-tuning with few samples to optimize the prediction of individual controllers when the collected individual PERCLOS time series and exogenous factor data meet the preset conditions.

[0020] Preferably, it further comprises:

[0021] compare the fatigue level curve with a conditioned threshold determined according to the work type, circadian rhythm and work demand;

[0022] generate an early warning information when the value of the fatigue level curve at a future time point will exceed the conditioned threshold.

[0023] Preferably, the calculation of the PERCLOS time series comprises: quality scoring of the face video frames, performing frame skipping and state caching on low-quality frames when the quality score is below a first threshold; suspending or delaying the generation of the fatigue level curve when the quality score is below a second threshold within a continuous preset time length.

[0024] Preferably, the exogenous factors further comprise at least one of individual characteristics, the individual characteristics comprising bucket encoding of service length, frequency or length of night shift in the past seven days, and discrete encoding of pre-shift sleep self-evaluation, and the individual characteristics are collectively input as factor-gated inputs to achieve differential modeling together with the work type, circadian rhythm and work demand.

[0025] Preferably, the training of the time series neural network model comprises:

[0026] training based on historical data of multiple controllers to obtain a general fatigue prediction model;

[0027] fine-tuning the general fatigue prediction model using personal historical data of the controller to be tested to generate the time series neural network model.

[0028] Advantages

[0029] 1. The present application converts the traditional post-event warning into pre-event warning by generating a fatigue level curve for a future period of time, which can provide fatigue trend prediction for a certain period of time in the future, leaving a sufficient decision-making window for managers to intervene in task allocation, rest arrangement, etc., thereby moving the safety management to an earlier stage and greatly improving the safety margin.

[0030] 2. The present application combines physiological indicators (PERCLOS) with exogenous factors related to work scenarios (work type, circadian rhythm, work demand, etc.) and processes them through a neural network capable of differential modeling. This enables the prediction model to dynamically adapt to different working conditions, for example, to learn that fatigue accumulation under high-load night work is much faster than under low-load daytime work, thereby significantly improving the accuracy of the prediction and the adaptability to real work scenarios, effectively reducing the false positive and false negative rates.

[0031] 3. By using a preferred individualized adaptation layer or fine-tuning training method, this invention can build a unique prediction model for each controller, fully considering the physiological differences between individuals, making the prediction results more consistent with individual fatigue patterns, and further enhancing the reliability and practical value of the prediction. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating a preferred embodiment of the present invention for a method to predict the fatigue of controllers on duty based on work type and circadian rhythm. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0034] Example 1

[0035] like Figure 1 As shown, this embodiment provides a complete process for a method to predict controller fatigue, which can be executed by a software program deployed on a control center server or an edge computing device.

[0036] S1. Obtain facial video frames from controllers and calculate the PERCLOS time series representing the percentage of eyelid closure.

[0037] In this embodiment, "PERCLOS" (Percentage of Eyelid Closure over the Pupilover Time) is defined as the percentage of time the eyelids are closed relative to the total time within a unit of time. It is an internationally recognized objective physiological indicator that is highly correlated with drowsiness and fatigue. Specifically, the system continuously captures a video stream of the controller's face at a rate of, for example, 24 frames per second using a camera installed in front of the controller's seat. For each video frame, an image processing module first uses techniques such as Multi-Task Convolutional Network (MTCNN) to perform face detection and eye key point localization. Subsequently, a lightweight convolutional neural network (CNN) classifier analyzes the eye area to determine whether it is open, partially closed, or fully closed. The system aggregates time in minutes, calculates the average PERCLOS value within that minute, and stores it along with a timestamp in a time-series database, thereby continuously generating and updating the controller's PERCLOS time series.

[0038] To cope with the poor video quality that may occur in real environment, a preferred implementation is to introduce a robust processing mechanism in the process of calculating the PERCLOS time series. Specifically, the system can perform real-time quality scoring on each frame of the human face video frame, which can comprehensively consider factors such as image clarity, deflection angle of facial posture, and uniformity of illumination. When the quality score of a frame is lower than a preset first threshold, it indicates that the frame data may be unreliable, and the system will perform frame skipping operation and use state caching technology, i.e. using the last valid eye state to fill the current time point, to ensure the continuity of the PERCLOS time series. Further, when the average quality score of the video frame is continuously lower than a more stringent second threshold for a continuous preset time length (for example, 10 seconds), the system will judge that the current data stream has failed seriously, at which time the generation of the fatigue level curve will be paused or delayed, and a prompt will be sent to the system administrator, thereby avoiding generating false predictions based on contaminated data.

[0039] S2, obtain exogenous factors related to the control task, the exogenous factors at least including one or more of work type, circadian rhythm, and work demand.

[0040] These exogenous factors are key external variables that affect fatigue evolution. In the present application, the exogenous factor is a superior concept, which at least includes one or more of work type, circadian rhythm, and work demand.

[0041] Work type refers to the nature of the task undertaken by the controller, for example, terminal control area (TCU) post or area control area (ACU) post, and the post information of the current controller can be obtained from the control scheduling system through the API interface

[0042] Circadian rhythm refers to the natural physiological rhythm cycle in a day, which is usually directly related to the current work shift time, such as day shift (e.g. 08:00-16:00), evening shift (16:00-24:00), or night shift (00:00-08:00), which can be obtained by obtaining the standard time of the current system.

[0043] Work demand refers to the instantaneous workload or task complexity, which can be obtained from the flight flow management system through the API interface to obtain real-time data related to the current seat, such as the number of flights handled in a preset time window (e.g. past 15 minutes), the alarm count of potential conflict events, etc.

[0044] The purpose of the core technology of this step is to provide "context" for the physiological indicators, solving the uncertainty of the model. A single physiological indicator time series can only describe "what happened" (What), i.e. the fatigue level is increasing or decreasing, but cannot explain "why" (Why).

[0045] While the exogenous factors provided by the present embodiment provide a basis for the model to make judgments, help it to eliminate ambiguity, and make more accurate predictions. The following will be further described according to a scene example:

[0046] Suppose the model observes that the PERCLOS value of a controller has a slight upward trend in the past 10 minutes. With only this information, the model faces great uncertainty:

[0047] Possibility A: This is just a normal physiological fluctuation that will soon recover.

[0048] Possibility B: This is an early sign of fatigue accumulation, which will soon enter a rapid growth period.

[0049] After introducing the exogenous factor, if the exogenous factor at this time is {work type: ACU, circadian rhythm: day shift, work demand: low}, the model learns from historical data that in this relaxed scenario, the fatigue accumulation rate is slow, so it is more likely to judge as "Possibility A" and predict that the future fatigue curve will tend to be flat; if the exogenous factor is {work type: TCU, circadian rhythm: second half of night shift, work demand: high}, the model will identify that this is a high-risk combination, and the probability of judging as "Possibility B" is extremely high, and it will predict a steep and rapidly rising fatigue curve.

[0050] The introduction of exogenous factors can bring four key improvements to the model:

[0051] 1. Improve prediction accuracy; this is the most direct contribution. Fatigue is a complex nonlinear process driven by multiple factors. An autoregressive model based only on historical PERCLOS values has very limited patterns to capture. Exogenous factors provide the model with additional, strongly explanatory feature dimensions, enabling it to learn more complex and accurate mapping relationships, which greatly improves the model's fitting and generalization capabilities, and thus improves the accuracy of the prediction.

[0052] 2. Enhance the model's interpretability: In critical areas such as aviation safety, a "black box" model is difficult to trust and apply. The introduction of exogenous factors makes the model's decision-making process partially transparent. When the system issues a warning, it can not only say "there is a fatigue risk in the next 30 minutes", but also provide more insightful explanations: "due to the current second half of night shift and the increase in flight conflict events, the model predicts that the fatigue risk will rise significantly". This interpretability is crucial for gaining the trust of air traffic controllers, effective risk communication, and post-mortem analysis.

[0053] 3. Support deep personalization: Individual characteristics (e.g. seniority, sleep habits, etc.) are a special but crucial subset of exogenous factors. They are the key to explain "why two people have completely different fatigue responses in the exact same work scenario".

[0054] In summary, the technical consideration of exogenous factors is the core of the evolution of the present invention from a simple "physiological signal monitor" to an "intelligent fatigue prediction system with context awareness, explainability, adaptability, and personalization". It is not just a technical choice, but a fundamental strategic layout to build a truly effective, reliable, and trustworthy on-duty fatigue management solution.

[0055] In a specific implementation, in order to convert these heterogeneous exogenous factors into a numerical format that the model can handle, they are encoded. For example, the work type is one-hot encoded (TCU is represented as [1, 0] and ACU is represented as [0, 1]); in order for the model to understand the periodicity of time, the circadian rhythm is encoded using a periodic function, for example, converting the hour h in a day into a two-dimensional vector [sin(2πh / 24), cos(2πh / 24)]; the work demand is numerically encoded using one or more proxy features, for example, counting the number of flights within a preset time window (e.g. the past 15 minutes), or counting the number of conflict-related events recorded by the automation system, and then normalizing these numerical values.

[0056] In order to further improve the personalization and accuracy of the prediction, in a more preferred embodiment, the exogenous factors further include individual characteristics. Individual characteristics are relatively stable attributes that characterize the individual differences of the controller. For example, they can include seniority (which can be bucketed, e.g. divided into "junior", "intermediate", and "senior" levels), the frequency or duration of night shifts in the past seven days (reflecting the degree of recent work-rest disorder), and the discrete encoding of pre-shift sleep self-evaluation (e.g. a 1-5 point sleep quality score submitted by the controller through a simple questionnaire before work). These individual characteristics will be encoded and processed together with the work type, circadian rhythm, and work demand data described above.

[0057] S3, encode the PERCLOS time series and the exogenous factors to obtain input features, which are input into a trained time series neural network model configured to apply different weights to the work type, circadian rhythm, and work demand for differential modeling.

[0058] The time series neural network model is a broad concept that can adopt various specific network architectures, such as architectures based on gated recurrent units (GRU), long short-term memory networks (LSTM), or Transformers.

[0059] The key of the present application is that the time series neural network model is configured to apply different weights to the work type, the circadian rhythm and the work demand respectively for differential modeling. This means that the model does not simply treat all input information equally, but can learn and understand the differences in fatigue evolution patterns under different external conditions. For example, the model can learn through training that under the circadian rhythm condition of "night shift", the same "high work demand" has a much greater promoting effect on fatigue accumulation than under the condition of "day shift".

[0060] The differential modeling refers to constructing a neural network model that can dynamically adjust its internal information processing method and judgment logic according to the input external conditions (i.e. exogenous factors). In order to efficiently realize this differential modeling, a preferred implementation is that the time series neural network model includes a factor gating structure. Specifically, this structure inputs the encoded exogenous factors (including work type, circadian rhythm, work demand, and optional individual characteristics) as a whole "factor vector" into a small gating network (e.g. one or more fully connected layers followed by an activation function such as Sigmoid). The output of this gating network is a set of channel-level weights, which are then used for element-wise multiplication with the input PERCLOS time series features. In this way, the model can dynamically and adaptively adjust its attention to historical fatigue time series features according to the current external conditions, i.e. differentially weight the input time series part, and then perform subsequent time series modeling, thereby achieving high flexibility and precision of scene adaptation.

[0061] In order to solve the individual difference problem and optimize the model performance, the time series neural network model of the present application can adopt a two-stage training and deployment strategy. First, based on the historical data of multiple controllers, a general fatigue prediction model is trained. This general model learns the general fatigue rules of the controller group. Then, when the system serves a specific controller to be tested, the general fatigue prediction model can be fine-tuned using the personal historical data of the controller to be tested. This transfer learning approach enables the final time series neural network model used for prediction to quickly adapt to the unique characteristics of the individual, thereby significantly improving the prediction accuracy.

[0062] In actual deployment, in order to handle the "cold start" problem (i.e. new users or users with insufficient data), the time series neural network model can further include an individual adaptation layer. When the system cannot obtain sufficient personal historical data for fine-tuning, the adaptation layer will use initial parameters based on population stratification priors. For example, the system can pre-cluster all historical users according to characteristics such as position, length of service, etc. to form several group portraits. For new users, the system will classify them into the most similar group according to their characteristics, and use the average model parameters of the group as the initial configuration of their prediction. As the individual controller's personal data accumulates, when the data volume meets the preset conditions (e.g. more than 20 complete shifts of data have been recorded), the adaptation layer will automatically trigger the online fine-tuning process of the small sample, iteratively optimize the model with newly collected data, and thus continuously improve the prediction ability of individual controllers.

[0063] S4, based on the output of the time series neural network model, generate a fatigue level curve covering a preset prediction time window. For example, the output of the model is a vector containing 60 numerical values, corresponding to the predicted PERCLOS value of each minute in the next 60 minutes. On the user interface, these points are connected to form a smooth curve, which intuitively shows the fatigue evolution trend of the controller in the next hour.

[0064] Finally, in order to convert the prediction into effective safety intervention, the method of the present application also includes the step of generating an early warning information. The system compares the generated fatigue level curve with a conditional threshold. The threshold here is not fixed, but dynamically determined according to the current type of work, circadian rhythm and work demand. For example, the system can have a built-in rule engine or a small model to calculate a reasonable, adaptive fatigue risk threshold according to the current exogenous factors. When the system detects that the predicted fatigue level curve will exceed this dynamic threshold at a certain time point in the future, it will immediately generate an early warning information and push it to the controller supervisor. The information not only contains the alarm, but also can be accompanied by the time when the threshold is expected to be reached, for example: "Warning: seat A controller is expected to enter a high-risk state of fatigue in 28 minutes, please pay attention." This forward-looking warning provides valuable decision-making time for managers to take preventive measures (such as adjusting task allocation, arranging temporary rest, etc.), thereby eliminating safety risks at the embryonic stage.

[0065] In summary, the present application successfully upgrades the traditional passive fatigue detection to active and forward-looking fatigue prediction by acquiring and fusing physiological time series data and multi-dimensional exogenous factors, using an advanced time series neural network model capable of differential modeling and personalized adaptation. The present application not only significantly improves the accuracy and scenario adaptability of the prediction, but also provides strong technical support for the safety management of high-risk posts by generating future fatigue curves and early warning information, which has extremely high practical value and popularization prospect. Those skilled in the art can understand that, without departing from the core idea of the present application, various modifications or equivalent replacements can be made to the specific details in the above embodiments, such as the selection of network structure, the way of feature encoding, the threshold setting strategy, etc. These changes shall fall within the protection scope of the present application.

Claims

1. A method for predicting fatigue of a tower controller based on a work type and a circadian rhythm, characterized by, The method comprises the steps of: obtaining facial video frames of a controller on duty, and calculating a PERCLOS time sequence representing the proportion of eyelid closure; obtaining exogenous factors related to the control task, the exogenous factors including one or more of work type, circadian rhythm, and work demand; encoding the PERCLOS time sequence and the exogenous factors to obtain input features, and inputting the input features into a trained time sequence neural network model configured to apply different weights to the work type, the circadian rhythm, and the work demand for differential modeling; generating a fatigue level curve covering a preset prediction time window based on the output of the time sequence neural network model; The method further comprises the steps of: comparing the fatigue level curve with a conditional threshold determined according to the work type, circadian rhythm, and work demand; generating early warning information when the value of the fatigue level curve at a future time point will exceed the conditional threshold; The time sequence neural network model comprises a factor gating structure that generates channel-level weights based on a factor vector of at least one of the exogenous factors, differentially weights the time sequence part in the input features, and then performs time sequence modeling; The time sequence neural network model further comprises an individualized adaptation layer that uses initial parameters based on population stratification prior in cold start, and performs online fine-tuning with a small sample to optimize the prediction of the individual controller when the collected individual PERCLOS time sequence and exogenous factor data meet the preset conditions.

2. The work type and circadian rhythm based fatigue prediction method for a tower controller according to claim 1, wherein, The encoding of the exogenous factors comprises: one-hot encoding the work type; encoding the circadian rhythm using a periodic function; numerically encoding at least one of the number of flights and / or the count of conflict-related events in a preset time window as a proxy feature. 3.The work type and circadian rhythm based fatigue prediction method for tower controllers according to claim 1, wherein, The calculation of the PERCLOS time sequence comprises: quality scoring the facial video frames, performing frame skipping and state caching on low-quality frames when the quality score is below a first threshold, and suspending or delaying the generation of the fatigue level curve when the quality score is below a second threshold for a continuous preset duration. 4.The work type and circadian rhythm based fatigue prediction method for tower controllers according to claim 1, wherein, The exogenous factors further include at least one individual feature, including bucketed encoding of service length, frequency or duration of night shifts in the past seven days, and discrete encoding of pre-shift sleep self-assessment, and the individual features are input to the factor gating together with the work type, circadian rhythm, and work demand for differential modeling. 5.The work type and circadian rhythm based fatigue prediction method for tower controllers according to claim 1, wherein: The training of the time sequence neural network model comprises: training a general fatigue prediction model based on historical data of multiple controllers; fine-tuning the general fatigue prediction model using personal historical data of the controller to be tested to generate the time sequence neural network model.

Images