Behavioral Elicitation Event Database for Traceable Soft-Skill Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for evaluating soft skills in competency frameworks, such as those used in pilot training, face challenges in objective assessment and providing consistent feedback due to the subjective nature of instructor observation, leading to inconsistent evaluations and increased costs.
Innovation Solution
Implementing a computer-based behavioral elicitation event (BEE) system that utilizes a database of predefined events to objectively assess soft skills by matching user inputs with elicited actions and observable behaviors in simulated training environments, providing detailed feedback and improving inter-rater reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If instructor observation is used to evaluate soft skills, then evaluation can be performed, but objectivity and consistency of evaluation deteriorates
Solution Approach 1:
The patent replaces the mechanical system of human instructor observation with an automated computer-based evaluation system. The system uses a BEE database stored in memory, processes simulation state updates and action notifications through a logic system, and automatically determines competency framework compliance without requiring instructor presence or subjective judgment.
Solution Approach 2:
The evaluation system performs self-service by automatically monitoring and evaluating training subject actions against predefined BEEs in the database. The logic system continuously processes simulation state updates and action notifications to autonomously determine when elicited actions are met and when competency framework requirements are satisfied, eliminating the need for external instructor evaluation.
2Reliability
If instructor observation is used for evaluation, then feedback can be provided, but cost increases
Solution Approach 1:
The system enables self-service evaluation where the computer-based logic system automatically monitors training performances, processes simulation data, and determines competency compliance without requiring instructor resources. The BEE database and logic system work autonomously to provide continuous evaluation and feedback.
Solution Approach 2:
The patent substitutes the resource-intensive mechanical system of instructor observation with an automated digital system. The logic system processes simulation state updates and action notifications efficiently, providing evaluation feedback without the need for human instructor time or presence.
3Reliability
If automated BEE system is implemented, then objectivity and consistency improve, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the evaluation system into distinct functional components: a BEE database stored in memory containing predefined behavioral elicitation events, a logic system that processes simulation state updates and action notifications, and a mechanism for determining competency framework compliance. This modular architecture manages complexity through functional separation.
Solution Approach 2:
The system implements preliminary action by pre-storing BEEs in the database before training occurs. The logic system is pre-configured with the evaluation criteria, allowing it to automatically process simulation data and determine compliance without requiring complex real-time decision-making or instructor judgment during the training performance.
4Loss of information
If instructor observation is used, then evaluation can be performed, but traceability of evaluation deteriorates
Solution Approach 1:
The patent replaces the untraceable human observation process with an automated digital system that systematically records and processes evaluation data. The logic system receives simulation state updates and action notifications, processes them against predefined BEEs, and generates traceable determinations of competency compliance that can be reviewed and verified.
Data Source
AI summary
One example provides a method of evaluating a training performance of a training subject and performed in a training environment. The method receives a simulation state update from a computing system of the training environment, and queries a behavioral elicitation event (BEE) database to determine a set of active BEEs based upon the simulation state update. The method further receives a plurality of simulation action notifications from the computing system, and for each simulation action notification, determines whether an action represented by the simulation action notification matches an elicited action for an active BEE. The method further stores information regarding the elicited action as a response to the active BEE. The method determines a set of completed BEEs comprising any active BEEs that have all corresponding elicited actions met, and for each completed BEE, stores information regarding the completed BEE.