Air traffic controller competency and psychological competency evaluation method and system

Abstract

The invention relates to the field of data processing methods for management purposes, and discloses an air traffic controller competency and psychological competency evaluation method and system, and the method comprises the steps: constructing a competency and psychological competency evaluation model; acquiring a historical training data set; respectively training a first neural network model and a second neural network model; collecting objective data reflecting the current performance of the controller to be evaluated in the standardized simulation training scene, and respectively inputting the objective data into the first neural network model and the second neural network model to obtain prediction scores of the first observation index and the second observation index of the controller to be evaluated; determining a wind competency evaluation result based on the prediction score of the first observation index and a wind competency evaluation model; and determining a psychological competency evaluation result based on the predicted score of the second observation index and a psychological competency evaluation model. According to the method, subjective prejudice caused by individual differences such as experience and state of scoring persons is remarkably reduced, and the objectivity, consistency and repeatability of scoring are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing methods for management purposes, and in particular to a method and system for evaluating the style competence and psychological competence of air traffic controllers. Background Art

[0002] Air traffic controllers (hereinafter referred to as "controllers") shoulder the key responsibility of ensuring air traffic safety and efficiency. Their work style and psychological competence directly affect flight safety. Currently, the evaluation methods for controllers' work style and psychological competence mainly rely on the following methods: 1. Manual observation and subjective evaluation: During simulator training or actual work, experienced instructors or experts observe and qualitatively evaluate controllers' performance based on their experience, or conduct interviews. This method relies heavily on the evaluator's expertise and experience, and the results are susceptible to subjective bias, individual emotional state, and inconsistent evaluation criteria, making it difficult to ensure objectivity and consistency.

[0003] 2. Traditional Questionnaires and Psychological Tests: Air traffic controllers are tested using standardized psychological questionnaires or work style inventories. While these tools generally have a certain degree of reliability and validity, their content can be too general and lack direct and in-depth relevance to air traffic controllers' specific work situations and behaviors. Furthermore, the evaluation results may be influenced by the social desirability effect or misinterpretation of the subjects.

[0004] 3. Simulator training assessment: Controllers' skills are assessed through the use of specific simulated training scenarios. While these can replicate work scenarios, the assessment of the work style and psychological aspects involved still largely relies on instructors' subjective judgment and scoring based on operational records. This lacks a refined and objective approach to measuring the underlying competency factors behind complex behaviors.

[0005] In the existing technology, although some studies have attempted to introduce data analysis or machine learning methods, there are often the following shortcomings: First, there is a lack of a competency evaluation model that has been constructed and verified by scientific methods (such as psychometric theory) and can clearly define the relationship between evaluation indicators (latent variables) and specific observation indicators as a basis; second, even if objective data is collected, it is difficult to directly and effectively establish clear and explainable associations with abstract style or psychological competency indicators and perform objective scoring; third, for the two different but related competency dimensions of style and psychology, there is a lack of an integrated solution that can conduct targeted modeling and evaluation separately while utilizing a common objective data foundation.

[0006] Therefore, there is still room for improvement in existing technologies in achieving a comprehensive, objective, efficient and insightful evaluation of air traffic controllers' work style and psychological competence. Summary of the Invention

[0007] In order to solve the problems existing in the above-mentioned prior art, the technical solutions provided by the present invention include: The evaluation method for the style competence and psychological competence of air traffic controllers includes the following steps: S1. Construct a work style competency evaluation model and a psychological competency evaluation model. The work style competency evaluation model defines a first evaluation indicator and a corresponding first observation indicator, and the psychological competency evaluation model defines a second evaluation indicator and a corresponding second observation indicator. S2. Obtaining a historical training dataset generated in a standardized simulated training scenario associated with the first and second observation indicators, the historical training dataset including objective data reflecting the historical controller's performance in the scenario and corresponding expert scores for the first and second observation indicators; S3. training a first neural network model for predicting the first observation indicator score and a second neural network model for predicting the second observation indicator score based on the historical training data set; S4. Collect objective data reflecting the current performance of the controller to be evaluated in at least one standardized simulated training scenario, and input the data into the first and second neural network models, respectively, to obtain predicted scores for the first and second observation indicators of the controller to be evaluated; S5. Based on the predicted score of the first observation indicator and the work style competency evaluation model, determine the work style competency evaluation result of the controller to be evaluated; based on the predicted score of the second observation indicator and the psychological competency evaluation model, determine the psychological competency evaluation result of the controller to be evaluated.

[0008] Preferably, the step of constructing the work style competency evaluation model and the psychological competency evaluation model in S1 includes: deconstructing the work style competency evaluation elements and the psychological competency evaluation elements through the Delphi method, and defining the first evaluation indicator, the first observation indicator and the second evaluation indicator, the second observation indicator.

[0009] Preferably, the step of constructing the work style competency evaluation model and the psychological competency evaluation model in S1 further includes: The structural equation model was used to verify the correlation between the first evaluation index and the first observation index, as well as the correlation between the second evaluation index and the second observation index, and the maximum likelihood estimation method was used for parameter estimation and model fit test to optimize and finally determine the first observation index and the second observation index.

[0010] Preferably, the objective data includes the operation record data and response data of the air traffic controller in the standardized simulation training scenario, and the questionnaire data filled in by the controller after completing the standardized simulation training scenario.

[0011] Preferably, the first neural network model and the second neural network model adopt the same network architecture, including: A first input processing module, configured to process time series features derived from the operation record data and response reaction data; A second input processing module, configured to process semantic features derived from the questionnaire data; a feature fusion module, fusing the temporal features output by the first input processing module and the semantic features output by the second input processing module through an attention mechanism to generate a fused feature; The prediction output module predicts the corresponding observation indicator score based on the fusion features through the fully connected layer and the corresponding activation function.

[0012] Preferably, the method for processing the time series features by the first input processing module includes: Arranging the operation record data in chronological order to form an event sequence, and converting the event record data into a first numerical vector sequence by one-hot encoding according to the event category; The response reaction data are arranged into a response sequence according to the response category, and the time features in the response sequence are calculated, and converted into a second numerical vector sequence through one-hot encoding according to the response category.

[0013] Preferably, the method for processing semantic features by the second input processing module includes: Performing one-hot encoding processing on the multiple-choice question data in the questionnaire data; Normalizing the scoring question data in the questionnaire data; Extract text semantic feature vectors from the text question data in the question-answer sheet data.

[0014] The present invention also discloses an air traffic controller style and psychological competence evaluation system, comprising: A model building module, used to build a work style competency evaluation model and a psychological competency evaluation model, wherein the work style competency evaluation model defines a first evaluation indicator and a corresponding first observation indicator, and the psychological competency evaluation model defines a second evaluation indicator and a corresponding second observation indicator; a data acquisition module, configured to acquire a historical training data set generated in a standardized simulated training scenario associated with the first observation indicator and the second observation indicator, the historical training data set comprising objective data reflecting the performance of historical controllers in the scenario and corresponding expert scores on the first observation indicator and the second observation indicator; A model training module, configured to train a first neural network model for predicting the first observation indicator score and a second neural network model for predicting the second observation indicator score based on the historical training data set; a score prediction module, configured to collect objective data reflecting the current performance of the controller to be evaluated in at least one of the standardized simulated training scenarios, and input the objective data into the trained first neural network model and the second neural network model to obtain a predicted score for the first observation indicator and the second observation indicator of the controller to be evaluated; A result determination module is used to determine the work style competency evaluation result of the controller to be evaluated based on the predicted score of the first observation indicator and the work style competency evaluation model; and to determine the psychological competency evaluation result of the controller to be evaluated based on the predicted score of the second observation indicator and the psychological competency evaluation model.

[0015] Beneficial effects 1. By constructing a work style competency assessment model and a psychological competency assessment model, we scientifically define the assessment indicators (latent variables) and their corresponding observational indicators, and validate their interrelationships, ensuring the theoretical foundation and construct validity of the assessment framework. The independent assessment models and the neural network model that predicts their observational indicators enable specialized assessments of these two distinct but critical competency dimensions, producing independent evaluation results that facilitate more refined analysis and personalized development recommendations.

[0016] 2. Utilizing the first and second neural network models, trained based on historical objective data and expert ratings, the system automatically predicts the scores for the first and second observation indicators based on objective data collected from the controllers being evaluated in standardized simulated training scenarios. This replaces subjective human judgment with machine intelligence, significantly reducing subjective bias caused by individual differences in raters' experience and status, and improving the objectivity, consistency, and repeatability of the ratings.

[0017] 3. Integrate and process various types of objective data, including simulator operation records, response data, and questionnaire data. Through specific feature engineering processing and the fusion mechanism within the neural network, more comprehensive multi-dimensional information reflecting the controller's competency performance is captured, thereby making more accurate score predictions.

[0018] 4. It can conduct specialized assessments of two distinct but critical competency dimensions: work style and psychology. The output of independent evaluation results facilitates more refined analysis and personalized development recommendations. Furthermore, both are based on a unified framework for objective data collection and processing, demonstrating the integrated nature of the evaluation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1The present invention provides a flow chart of a method for evaluating the work style competency and psychological competency of air traffic controllers in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0020] Example 1 The present invention relates to the technical field of air traffic controller (hereinafter referred to as "controller") competency assessment. Specifically, it relates to a method and system for comprehensively evaluating controllers' work style and psychological competence. In air traffic management, a controller's rigorous work style and psychological stability are crucial to their ability to safely and efficiently handle air traffic situations. Existing techniques for evaluating controllers' work style and psychological competence often rely on manual observation and subjective scoring by expert instructors during simulated training or actual work, or on the use of general psychological assessment scales. For example, after a complex simulator training session, an instructor might, based on their own experience and overall impression of a controller's performance, give evaluations such as "not decisive enough" or "needs to improve their awareness of rules," and assign scores corresponding to certain observational dimensions. A drawback of this approach is that the evaluation results are easily influenced by various non-technical factors, such as the evaluator's personal experience, subjective preferences, fatigue, and even their interpersonal relationship with the person being evaluated. This makes it difficult to fully guarantee the objectivity, consistency, and repeatability of the evaluation. Furthermore, while traditional psychological questionnaires are highly standardized, their content is often generic and may not fully capture the specific work styles and psychological traits required for regulatory work. Furthermore, participants may also exhibit a degree of social desirability bias in their responses. These issues limit the accuracy of assessment results and their effective application in personnel selection, targeted training, and risk management.

[0021] This invention aims to overcome the shortcomings of the prior art by providing a method and system for evaluating the work style and psychological competence of air traffic controllers in a more objective, detailed, and efficient manner. Its core innovation lies in the development of a collaborative working model that deeply integrates a competency theory model based on expert knowledge and scientific methods with objective behavioral data analysis technology based on machine learning. This model enables automated, standardized, and multi-dimensional evaluation of controller performance in simulated training scenarios.

[0022] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0023] In specific embodiments of the present invention, the "first evaluation indicator" and "second evaluation indicator" refer to relatively abstract, difficult-to-measure core competency latent variable dimensions defined in the competency model, such as "responsibility" and "rigor" under work style competencies, or "emotional stability" and "stress tolerance" under psychological competencies. The "first observation indicator" and "second observation indicator" are specific, observable or measurable indicators corresponding to these latent variable evaluation indicators, which can be manifested through specific behaviors or reactions in specific situations. For example, the evaluation indicator "responsibility" may be reflected through multiple observation indicators, such as "actively identifying and reporting potential conflicts" and "strictly adhering to standard operating procedures (SOPs)."

[0024] A method for evaluating the style competence and psychological competence of air traffic controllers, such as Figure 1 As shown, its core process and key technical features are elaborated in detail below.

[0025] S1. Construct work style competency evaluation model and psychological competency evaluation model.

[0026] The method of this invention first requires the establishment of scientific and rational work style competency assessment models and psychological competency assessment models as the foundation for subsequent evaluation. The work style competency assessment model defines a first evaluation indicator (i.e., a latent variable dimension related to work style) and a corresponding first observation indicator (i.e., a measurable work style behavior manifestation point). Similarly, the psychological competency assessment model defines a second evaluation indicator (i.e., a latent variable dimension related to psychology) and a corresponding second observation indicator.

[0027] In a preferred embodiment, the model construction process, particularly the deconstruction of evaluation elements and the preliminary definition of evaluation and observation indicators, can utilize the Delphi method. The Delphi method is a structured decision support method that uses multiple rounds of anonymous expert questionnaires and feedback to gradually converge expert opinions. Specifically, a panel of experts, including senior air traffic control experts, instructors, and psychology experts, can be invited to form a panel. First, through open-ended questionnaires or brainstorming sessions, the experts gather their opinions on the key elements of air traffic controller style and psychological competence. These elements are then organized into items. Through subsequent rounds of questionnaires, the experts are asked to rate and rank each element based on its importance, clarity, and observability, and to provide feedback for revisions. After each round of questionnaires, the organizers will provide feedback to the expert panel on the statistical results and key opinions for reference in the next round of decision-making, until the expert opinions reach a predetermined convergence standard. This method allows for the systematic and scientific extraction and definition of a system of evaluation and observation indicators that align with the practicalities of air traffic control work, drawing upon expert experience.

[0028] After initially defining the evaluation indicators and observational indicators, a further preferred approach to ensure the scientificity and validity of the model is to employ structural equation modeling (SEM) to verify the correlations between the first evaluation indicator and the first observational indicator, and between the second evaluation indicator and the second observational indicator, respectively. This approach can then optimize and ultimately determine the observational indicators. Structural equation modeling is a multivariate statistical method that can simultaneously process latent variables and their observational indicators, analyzing the complex relationships between them. Specifically, a sample of data can be collected (for example, by having experts rate controllers' performance on various observational indicators from historical cases) and then a hypothesized measurement model (i.e., a model of the relationship between the latent variable evaluation indicators and their corresponding observational indicators) can be constructed. To verify these correlations, maximum likelihood estimation (MLE) can be used to estimate model parameters. A series of model fit tests (such as chi-square, RMSEA, CFI, TLI, etc.) can be used to assess the fit between the theoretical model and the actual data. If the fit is poor, or the factor loadings of certain observation indicators are not ideal (i.e., the correlation with the corresponding evaluation indicators is not strong), the model can be modified, such as deleting or modifying inappropriate observation indicators, until the model achieves an acceptable goodness of fit. This step ensures that the observation indicators ultimately included in the evaluation model can effectively and reliably reflect the corresponding latent variable evaluation indicators, thereby ensuring the structural validity and content validity of the entire evaluation model.

[0029] In some preferred embodiments, the first evaluation indicators are set based on a systematic analysis of regulatory characteristics, relevant rules and regulations formulated by competent authorities and units, relevant domestic and foreign research results, and one-on-one interview survey results, and include: Professional Value Perception: An individual's understanding and selection of professional value elements. Positive perceptions can enhance proactive and reflective adherence to professional requirements. The corresponding first observation indicator might be something like, "Frequency of proactively seeking instructions or clarification from superiors when faced with non-standard instructions or potential rule conflicts during simulation training" or "Expression of recognition of the importance of safety regulations during debriefing."

[0030] Emotional management ability: An individual's ability to perceive, control, and regulate their emotions to maintain a stable state, adapt to complex environments, and make rational decisions. Corresponding primary observation indicators may include "the fluctuation range of physiological indicators (such as heart rate) when encountering continuous special situations or high-load operation during simulation training," "the degree to which one maintains calmness and professionalism during verbal communication when encountering communication breakdowns or misunderstood instructions," and "the accuracy of self-assessment of one's emotional control under high pressure after simulation training."

[0031] Behavioral adaptability: An individual's ability to respond to environmental stimuli to meet their needs, effectively adjust their behavior, and complete tasks correctly largely determines compliance with rules. The corresponding first observation indicators could be "the speed and accuracy with which new procedures or equipment are mastered during simulation training when standard operating procedures change or new equipment is introduced," "the ability to timely adjust control strategies and effectively implement them in the face of dynamically changing and complex traffic situations," and so on.

[0032] Interpersonal interaction patterns: Individuals' behavioral patterns in interpersonal interactions. Favorable interaction patterns help individuals adhere to social norms and influence their perceptions of regulatory agencies. Examples of corresponding first-level observation indicators might include "the proactiveness, clarity, and smoothness of communication in multi-person collaborative scenarios during simulated training (such as with adjacent sector controllers and shift supervisors)" and "how they react to feedback (positive or negative) from colleagues or pilots."

[0033] Social support system: The tools, emotional support, and professional support provided by an individual's environment (family, organization, etc.) helps them integrate resources to respond to incidents and reduce the likelihood of violations. While this is primarily a contextual factor, its corresponding first observation indicator in the evaluation model might be assessed through questionnaires to assess "the individual's perceived level of support from colleagues, supervisors, and family," and "the individual's willingness and behavior to proactively seek help when encountering work difficulties."

[0034] For each of the above-mentioned first evaluation indicators, several specific first observation indicators will be further refined and verified through methods such as the Delphi method and the structural equation model. These observation indicators constitute the target scoring items predicted by the subsequent neural network model.

[0035] In other preferred embodiments, the second evaluation indicators selected by expert evaluation based on international standards, comprehensive reference to domestic and foreign literature, interviews, and general psychological assessment tools include: Personality traits refer to relatively stable and enduring personality traits within an individual's psychological competence, which influence how they respond to various situations. They are uniquely selected to address the high-pressure, high-responsibility nature of the air traffic control profession, ensuring that the assessment results are highly aligned with aviation safety requirements. These traits are further subdivided into core dimensions such as "stability," "responsibility," "rationality," "self-discipline," and "prudence." Each core dimension corresponds to a set of secondary observation indicators. For example, "stability" might be measured through indicators such as "the increase in operational error rates when faced with sudden, high-stress events during simulated training" and "the degree of fatigue experienced during continuous tasks." "Responsibility" might be measured through indicators such as "the completeness and accuracy of recitation of air traffic control instructions" and "the comprehensiveness and detail of information conveyed during shift handovers."

[0036] Ability traits: The psychological characteristics that enable individuals to effectively complete tasks, including cognitive abilities and interpersonal skills, are key factors in determining individual work or task performance. These traits encompass four dimensions: cognitive thinking, adaptability, interpersonal skills, and general cognition. For example, secondary observation indicators for cognitive thinking might include "the number and speed of identifying potential conflict points in complex traffic situations" and "the ability to simultaneously monitor and manage the dynamics of multiple aircraft." Adaptability might be measured through, for example, "the efficiency and accuracy of executing emergency response procedures for sudden equipment failures in simulated training."

[0037] Abnormality Assessment: This involves a broad range of psychiatric symptoms, such as thinking, emotions, behavior, interpersonal relationships, and lifestyle habits, providing a comprehensive understanding of the subject's mental health. Secondary observation indicators here are often derived directly from standardized psychological assessment tools, such as the scores for various factors in the SCL-90 scale, such as somatization, obsessive-compulsive disorder, interpersonal sensitivity, depression, and anxiety. These can be directly used as scores for observation indicators.

[0038] Similarly, for each evaluation indicator under personality traits and ability traits, scientific methods will be used to verify and determine several specific second observation indicators.

[0039] After constructing the evaluation model in step S1 and determining the first and second observation indicators, these observation indicators do not exist in isolation, but need to be closely and logically associated with one or more standardized simulation training scenarios. This association is the basis for subsequent objective scoring using simulation training data through neural network models. Specifically, if observation indicators related to "emotional management ability" are to be evaluated, the designed simulation training scenarios will include high-pressure and high-load elements such as high traffic volume, multiple concurrent special situations, and communication interference, in order to observe the controller's emotional stability and decision-making quality under these stimuli. If observation indicators related to "behavioral adaptability" are to be evaluated, the scenarios may include sudden rule changes, the activation of new equipment, unexpected airspace user requests, etc., in order to observe their learning and adaptation speed and the effectiveness of behavioral adjustments.

[0040] By associating observational indicators with specific scenarios, we provide concrete, observable "anchors" for the subsequent scoring of these observational indicators. That is, when experts score historical controllers' observational indicators in step S2, they base their scores on the controllers' specific behaviors and responses in these specific, designed, standardized simulation training scenarios. For example, experts' scores for the observational indicator "Emergency Response Speed" would be based on the time it took the controller to receive the emergency situation report and issue the first valid instruction in a simulated scenario involving an "engine failure" emergency situation.

[0041] In summary, the first and second observation indicators are scientifically defined and validated measurement points that specifically characterize latent variables of work style and psychological competence. The close connection between these observation indicators and standardized simulation training scenarios is a key step in ensuring that the evaluation content is aligned with actual work, that data collection is targeted, and that the subsequent neural network can effectively learn and objectively score. This connection gives the entire evaluation method both theoretical depth and practical applicability.

[0042] S2. Obtain historical training dataset.

[0043] After the evaluation model is constructed, a historical training dataset is required to train the neural network model used for objective scoring. This dataset is derived from controller performance in standardized simulated training scenarios associated with the first and second observational indicators determined in step S1. This historical training dataset consists of two core components: objective data reflecting the controller's performance in the scenarios, and expert scores for the first and second observational indicators.

[0044] In a specific embodiment, the “objective data” include but are not limited to: 1. Records of air traffic controllers' operations in the standardized simulated training scenarios: This data can refer to the specific operations performed by the controller on the simulator, such as keystrokes, mouse clicks, command input sequences, flight plan modification records, and system parameter adjustments. This data is typically automatically and in detail recorded by the simulator system, providing a high degree of objectivity and traceability.

[0045] 2. Response data: This refers to the controller's responses to system alerts, calls, or instructions from other virtual characters (e.g., pilots or other controllers). Examples include response time to specific alerts, the content and speed of responses to pilot requests, and adherence to standard phraseology used in ground-to-air communications. Response data can be obtained through simulator event logs and voice-to-text conversion.

[0046] 3. Questionnaires completed by air traffic controllers after completing the standardized simulation training scenarios: This is an important supplemental data source. Questionnaires can be designed to include questions addressing decision-making processes, rule understanding, self-assessment, and emotional experiences in specific simulation training scenarios. For example, air traffic controllers could be asked, "When faced with XX special situation, what was the primary basis for your decision Y?" or "Please rate your stress management skills during this simulation training on a scale of 1-5." This type of data can provide information on air traffic controllers' internal cognitive processes and subjective experiences that is difficult to directly capture in operational records alone.

[0047] Expert scoring of the first and second observational indicators refers to the specific scores assigned by senior instructors or domain experts to each observational indicator after observing historical controllers complete the aforementioned simulated training scenarios, compared to the observational indicator system determined in S1. These scores constitute the "labels" or "target variables" for subsequent neural network model training.

[0048] S3. Train the neural network model.

[0049] Based on the historical training data set obtained in S2, this step trains a first neural network model for predicting the first observation indicator score and a second neural network model for predicting the second observation indicator score.

[0050] In a preferred embodiment, the first and second neural network models can utilize the same network architecture, but they are trained independently. Using the same architecture helps reduce the complexity of model design and facilitates maintenance and deployment. Independent training ensures that each model can focus on learning the behavioral patterns and data features most relevant to its target competency domain (work style or psychology). Of course, in other embodiments, different network architectures can be designed for work style and psychology, depending on the characteristics of the observed indicators of psychological competency.

[0051] Specific network architectures may include: The first input processing module is specifically designed to process the temporal features derived from the operation record data and response data. Since the controller's operations and responses are essentially sequential data that occurs over time, this module can employ any of the established sequence models, such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), gated recurrent units (GRUs), or more advanced Transformer networks, to capture the temporal dependencies and dynamic patterns within these data. Specifically, the operation record data can be arranged chronologically to form an event sequence, and the event categories can be converted into a first sequence of numerical vectors using one-hot encoding. The response data can be arranged into a response sequence based on response category, and the temporal features of the response sequence (such as response delay and call duration) are calculated and converted into a second sequence of numerical vectors using one-hot encoding based on the response category. These numerical vector sequences can then be input into models such as LSTMs or GRUs for processing.

[0052] The second input processing module is specifically designed to process semantic features derived from the questionnaire data. For multiple-choice questions, one-hot encoding can be performed; for rated questions (such as the Likert scale), normalization can be performed and used directly as numerical features; for text questions, natural language processing techniques can be used to extract semantic feature vectors. These processed features can then be input into a network structure such as a multilayer perceptron (MLP).

[0053] Feature Fusion Module: This module is responsible for effectively fusing the temporal features output by the first input processing module with the semantic features output by the second input processing module to generate a fused feature vector that comprehensively reflects the various aspects of the controller's performance. This fusion utilizes the attention mechanism. When fusing features from different sources, the attention mechanism enables the model to dynamically learn which features are more important in the current prediction task, assigning them higher weights and improving fusion effectiveness.

[0054] Prediction Output Module: Based on the fused features, the module predicts the corresponding observed indicator score through one or more fully connected layers and corresponding activation functions (for example, for bounded ratings, a Sigmoid function can be used to normalize the score to between 0 and 1 and then map it to the actual rating range; for unbounded ratings or multi-classification tasks, a Softmax function may be used). Because each neural network model (behavior or psychology) typically needs to predict the scores of multiple observed indicators, the prediction output module is typically a multi-output structure.

[0055] By using objective data from historical training datasets as input and expert scores on corresponding observation indicators as output to train these two neural network models, the models can learn the complex mapping relationship between objective behavioral performance and specific observation indicator scores.

[0056] S4. Collect and process data of the controller to be evaluated and make predictions.

[0057] After model training is complete, it can be used to evaluate new controllers. This step first requires collecting objective data reflecting the controller's current performance in at least one standardized simulated training scenario associated with the observation indicators determined in S1. The types of this objective data are consistent with the objective data of historical training data, namely, including operational records and response data in the simulated scenario, as well as the questionnaire data completed after the simulation.

[0058] For operation records and response data, event serialization and encoding, as well as interactive response feature extraction, are performed to form behavioral features. For questionnaire data, multiple-choice question encoding, score question normalization, and semantic feature extraction for text questions are performed to form question-answer features. Because the neural network architecture of this invention uses internal fusion, these two sets of features are processed by different input processing modules of the network.

[0059] Finally, the fused current input features are input into the trained neural network model respectively, so as to obtain the predicted scores of the first observation indicator (style category) and the second observation indicator (psychology category) of the controller to be evaluated.

[0060] S5. Determine the final evaluation results.

[0061] After obtaining the predicted scores of each observation indicator, the last step is to determine the style competency evaluation results and psychological competency evaluation results of the controller to be evaluated based on these scores and the pre-constructed style competency evaluation model and psychological competency evaluation model.

[0062] Specifically, the first observation indicator score predicted by the first neural network model can be substituted into the work style competency evaluation model. This model defines how to aggregate the scores of various observation indicators into a measure of the corresponding first evaluation indicator (work style latent variable), and further aggregate them into an overall work style competency level assessment. Similarly, the second observation indicator score predicted by the second neural network model is substituted into the psychological competency evaluation model to determine its psychological competency evaluation result. This determination process involves preset scoring rules, weight distribution and other contents. This part of the content is not the focus of the present invention. Those skilled in the art can select a suitable weight distribution method in the existing technology according to actual needs for implementation, and the present invention does not make further requirements.

[0063] Through the above steps, the method of the present invention can achieve a comprehensive, objective and in-depth evaluation of the work style and psychological competence of the controller.

[0064] Example 2 To implement the above method, the present invention also provides an air traffic controller style and psychological competency assessment system, which is used to implement the air traffic controller style and psychological competency assessment method described in the first embodiment. The system can be a computer system or a computer-based platform, and its typical hardware environment may include a processor, memory, input / output interface, network interface, etc. Its core functional modules can be designed as follows: The air traffic controller style and psychological competency evaluation system is characterized by including: A model building module, used to build a work style competency evaluation model and a psychological competency evaluation model, wherein the work style competency evaluation model defines a first evaluation indicator and a corresponding first observation indicator, and the psychological competency evaluation model defines a second evaluation indicator and a corresponding second observation indicator; a data acquisition module, configured to acquire a historical training data set generated in a standardized simulated training scenario associated with the first observation indicator and the second observation indicator, the historical training data set comprising objective data reflecting the performance of historical controllers in the scenario and corresponding expert scores on the first observation indicator and the second observation indicator; A model training module, configured to train a first neural network model for predicting the first observation indicator score and a second neural network model for predicting the second observation indicator score based on the historical training data set; a score prediction module, configured to collect objective data reflecting the current performance of the controller to be evaluated in at least one of the standardized simulated training scenarios, and input the objective data into the trained first neural network model and the second neural network model to obtain a predicted score for the first observation indicator and the second observation indicator of the controller to be evaluated; A result determination module is used to determine the work style competency evaluation result of the controller to be evaluated based on the predicted score of the first observation indicator and the work style competency evaluation model; and to determine the psychological competency evaluation result of the controller to be evaluated based on the predicted score of the second observation indicator and the psychological competency evaluation model.

[0065] These modules can be implemented through software programming and deployed on a server or local computer. They interact with each other through defined data interfaces and collaborate to complete the entire evaluation process.

[0066] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. The method for evaluating the style competence and psychological competence of air traffic controllers is characterized by: The following steps are involved: S1. Construct a work style competency evaluation model and a psychological competency evaluation model. The work style competency evaluation model defines a first evaluation indicator and a corresponding first observation indicator, and the psychological competency evaluation model defines a second evaluation indicator and a corresponding second observation indicator. S2. Obtaining a historical training dataset generated in a standardized simulated training scenario associated with the first and second observation indicators, the historical training dataset including objective data reflecting the historical controller's performance in the scenario and corresponding expert scores for the first and second observation indicators; S3. training a first neural network model for predicting the first observation indicator score and a second neural network model for predicting the second observation indicator score based on the historical training data set; S4. Collect objective data reflecting the current performance of the controller to be evaluated in at least one standardized simulated training scenario, and input the data into the first and second neural network models, respectively, to obtain predicted scores for the first and second observation indicators of the controller to be evaluated; S5. Determine the work style competency evaluation result of the controller to be evaluated based on the predicted score of the first observation indicator and the work style competency evaluation model; Based on the predicted score of the second observation indicator and the psychological competence evaluation model, the psychological competence evaluation results of the controller to be evaluated are determined.

2. The method for evaluating the style competence and psychological competence of air traffic controllers according to claim 1, wherein: The step of constructing the work style competency evaluation model and the psychological competency evaluation model in S1 includes: deconstructing the work style competency evaluation elements and the psychological competency evaluation elements through the Delphi method, and defining the first evaluation indicator, the first observation indicator, the second evaluation indicator, and the second observation indicator.

3. The method for evaluating the style competence and psychological competence of air traffic controllers according to claim 1 or 2, wherein: The step of constructing the work style competency evaluation model and the psychological competency evaluation model in S1 further includes: The structural equation model was used to verify the correlation between the first evaluation index and the first observation index, as well as the correlation between the second evaluation index and the second observation index, and the maximum likelihood estimation method was used for parameter estimation and model fit test to optimize and finally determine the first observation index and the second observation index.

4. The method for evaluating the work style and psychological competence of air traffic controllers according to claim 1, wherein: The objective data includes the operation record data and response data of the air traffic controller in the standardized simulation training scenario, as well as the questionnaire data filled out by the controller after completing the standardized simulation training scenario.

5. The method for evaluating the work style competency and psychological competency of air traffic controllers according to claim 4, wherein: The first neural network model and the second neural network model adopt the same network architecture, including: A first input processing module, configured to process time series features derived from the operation record data and response reaction data; A second input processing module, configured to process semantic features derived from the questionnaire data; a feature fusion module, fusing the temporal features output by the first input processing module and the semantic features output by the second input processing module through an attention mechanism to generate a fused feature; The prediction output module predicts the corresponding observation indicator score based on the fusion features through the fully connected layer and the corresponding activation function.

6. The method for evaluating the style competence and psychological competence of air traffic controllers according to claim 5, wherein: The method for processing the time series features by the first input processing module includes: Arranging the operation record data in chronological order to form an event sequence, and converting the event record data into a first numerical vector sequence by one-hot encoding according to the event category; The response reaction data are arranged into a response sequence according to the response category, and the time features in the response sequence are calculated, and converted into a second numerical vector sequence through one-hot encoding according to the response category.

7. The method for evaluating the style competence and psychological competence of air traffic controllers according to claim 5, wherein: The method for processing semantic features by the second input processing module includes: Performing one-hot encoding processing on the multiple-choice question data in the questionnaire data; Normalizing the scoring question data in the questionnaire data; Extract text semantic feature vectors from the text question data in the question-answer sheet data.

8. The air traffic controller style and psychological competency evaluation system is characterized by: include: A model building module, used to build a work style competency evaluation model and a psychological competency evaluation model, wherein the work style competency evaluation model defines a first evaluation indicator and a corresponding first observation indicator, and the psychological competency evaluation model defines a second evaluation indicator and a corresponding second observation indicator; a data acquisition module, configured to acquire a historical training data set generated in a standardized simulated training scenario associated with the first observation indicator and the second observation indicator, the historical training data set comprising objective data reflecting the performance of historical controllers in the scenario and corresponding expert scores on the first observation indicator and the second observation indicator; A model training module, configured to train a first neural network model for predicting the first observation indicator score and a second neural network model for predicting the second observation indicator score based on the historical training data set; a score prediction module, configured to collect objective data reflecting the current performance of the controller to be evaluated in at least one of the standardized simulated training scenarios, and input the objective data into the trained first neural network model and the second neural network model to obtain a predicted score for the first observation indicator and the second observation indicator of the controller to be evaluated; a result determination module, configured to determine a style competency evaluation result of the controller to be evaluated based on the predicted score of the first observation indicator and the style competency evaluation model; Based on the predicted score of the second observation indicator and the psychological competency evaluation model, the psychological competency evaluation result of the controller to be evaluated is determined.

Images