A pilot training operation automatic evaluation method and system based on rule guidance and artificial intelligence fusion

Abstract

The application relates to the technical field of computers and discloses a pilot training operation automatic evaluation method and system based on rule guidance and artificial intelligence fusion, which comprises the following steps: a probability semantic state machine model is constructed; a benchmark operation rule set constraint is converted into a semantic topological space with a state transition probability matrix; an artificial intelligence recognition algorithm is used to obtain multi-modal features of a target operation behavior of an operation subject, and an observation probability vector sequence representing operation probability distribution at each moment is output; the observation probability vector sequence is mapped to the semantic topological space, a state alignment relative entropy is calculated, an optimal alignment path is recursively searched, and an operation deviation cost is generated, and evaluation indexes are determined according to the operation deviation cost; and the semantic topological space with state transition weights is established, and the fault tolerance of evaluation logic to uncertainty of a perception layer is improved.

Description

Technical Field

[0001] This invention relates to an automatic evaluation method and system for pilot training operations based on rule-guided and artificial intelligence integration, belonging to the field of computer technology. Background Technology

[0002] Current pilot training assessments rely on instructor observations and experience-based interpretations, using rapid access recorders to check flight parameters and comparing the fit between operational processes and standard operating procedures to evaluate training quality. Increased training intensity leads to subjective differences in human evaluations. Existing monitoring systems process video images, cockpit voice, and flight parameters as independent channels, making it difficult to achieve collaborative semantic analysis of actions, commands, and state transitions at the millisecond time-series level.

[0003] Besides the limitations of hardware synchronization and data heterogeneity in the perception layer, the control methods also have shortcomings. For example, Chinese invention patent CN117455272A discloses a pilot training evaluation system. It calculates the correlation of training performance datasets and analyzes the distribution characteristics of values ​​using skewness and kurtosis statistical moment indices to generate training modification suggestions. The above scheme focuses on statistical modeling of the overall dataset and evaluates instructor performance or teaching differences in geographical areas, but ignores the fine-grained verification of the timing logic of pilot operation actions. When trainees' operations experience timeline drift or the recognition module is interfered with and generates instantaneous discrete noise, the static statistics or discrete label matching logic lacks the self-healing ability of operating condition logic and cannot match rigid rule step sizes, leading to evaluation interruption or false alarms. Establishing a nonlinear alignment calculation model that is compatible with the uncertainty of the perception layer and measures the nonlinearity of operation trends is the key to improving the stability of flight training evaluation.

[0004] Therefore, how to establish a computational model that aligns multimodal sensing probability sequences with rigid temporal logic constraints, and achieve nonlinear evaluation of operational trends under the premise of rigorous evaluation, has become the technical problem to be solved by this invention. Summary of the Invention

[0005] To address the problems mentioned in the background art, the technical solution of the present invention is as follows: An automatic evaluation method for pilot training operations based on rule-guided integration and artificial intelligence, comprising the following steps:

[0006] Step S101: Construct a probabilistic semantic state machine model; convert the sequential constraints in the baseline job rule set into a semantic topology space with a state transition probability matrix, where each job node corresponds to a semantic state node in the probabilistic semantic state machine model, and the temporal logic constraints between nodes are mapped to the transition probabilities within the probabilistic semantic state machine model.

[0007] Step S102: Extract the multimodal observation probability vector sequence; use artificial intelligence recognition algorithms to obtain the multimodal features of the working subject in the process of performing the target operation, and output the confidence vector of each operation event at each time moment to form an observation probability vector sequence that represents the probability distribution of each operation at the current time.

[0008] Step S103: Perform elastic temporal path search; map the observation probability vector sequence to the semantic topology space, calculate the state alignment relative entropy between the observation probability vector sequence and the state transition path, and recursively search for the optimal state alignment path that is aligned with the maximum likelihood of the path defined by the baseline job rule set in the semantic topology space to generate the job deviation cost characterized by the state alignment relative entropy.

[0009] Step S104: Generate automated evaluation results; determine and output job performance evaluation indicators based on the job deviation cost of the optimal state alignment path relative to the baseline job rule set and the time characteristics of state transition.

[0010] Preferably, the method for constructing the migration probability matrix in step S101 includes: identifying the temporal dependencies between each operation node in the baseline operation rule set; using a preset high migration probability for node migration paths that conform to the logical order, and using a punitive low migration probability for skip migrations that violate the logical order; and using the migration probability matrix to establish logical guidance constraints in the semantic topological space, so as to fill in the missing operation logic positions based on the prior probability distribution when there is instantaneous discrete noise in the observation probability vector sequence.

[0011] Preferably, step S102 includes: acquiring real-time image sequences, audio data, and multi-dimensional state parameter streams of the subject performing the target task; using a multimodal time alignment algorithm to synchronously process the image sequences, audio data, and multi-dimensional state parameter streams under a unified timestamp benchmark to construct a cross-dimensional feature verification mechanism; and performing dynamic arbitration on the behavior recognition results based on the changing trend of the multi-dimensional state parameter stream.

[0012] Preferably, the cross-dimensional feature verification mechanism evaluates the contribution of each modal data to the representation of the current operation status by calculating the mutual information entropy between different modal features; when the recognition confidence of a single modal feature is lower than a preset threshold, the observation probability vector sequence is reconstructed by weighting based on the feature vector of the complementary modality.

[0013] Preferably, in step S103, the cost of the operation deviation is calculated by accumulating the path cost. Sure: ,in, For a moment observation probability vector With semantic state nodes KL divergence between them The cost of migration between nodes, This is the logical constraint weighting factor.

[0014] Preferably, the method for determining the cost of job deviation in step S104 includes: when the calculated minimum cumulative KL divergence exceeds the preset logical fault tolerance threshold, determining that the target job behavior deviates from the baseline job rule set; assigning anomaly weights according to the hierarchical importance of the deviation position in the semantic topology space, and calculating a compliance score.

[0015] Preferably, step S104 further includes: extracting the work rhythm characteristics of the work subject by analyzing the residence time distribution among nodes in the optimal state alignment path; comparing the work rhythm characteristics with the benchmark behavior model to identify the cognitive load state of the work subject when performing complex tasks, and generating a dimensional assessment report on skill proficiency.

[0016] Preferably, the probabilistic semantic state machine model supports dynamic updates; the system adjusts the relaxation factor in the migration probability matrix according to the high-frequency operation deviation patterns in the historical operation process, so as to adapt to the reasonable operation fluctuation range of operators with different experience levels.

[0017] Preferably, after step S104, the method further includes: visually converting the alignment difference between the optimal alignment path and the standard path; synchronously outputting correction instructions on the operation interaction terminal; using logical guidance constraints to provide real-time guidance feedback with semantic continuity to the operation subject; the extraction process of the observation probability vector sequence in step S102 is implemented in the following way: adopting an architecture that integrates deep residual network and long short-term memory network; capturing the instantaneous features and long-term temporal features of the operation behavior through a sliding time window, with a sampling frequency of not less than 100Hz and a response delay of not more than 50ms, and converting the recognition result into a probability distribution vector that satisfies the unitization constraint.

[0018] An automated evaluation system for pilot training operations based on rule-guided instruction and artificial intelligence integration includes:

[0019] The module includes a model building module, a multimodal feature extraction module, a path search module, and an evaluation result generation module.

[0020] The model building module is used to perform step S101 to transform the sequential constraints in the baseline job rule set into a semantic topological space with a state transition probability matrix.

[0021] The multimodal feature extraction module is used to execute step S102 to obtain the multimodal features of the working subject in the process of performing the target work behavior, and output the confidence vector of each operation event at each time moment to form an observation probability vector sequence that represents the probability distribution of each operation at the current time.

[0022] The path search module is used to perform step S103 to map the observation probability vector sequence to the semantic topology space, calculate the state alignment relative entropy between the observation probability vector sequence and the state transition path, and recursively search for the optimal state alignment path in the semantic topology space to generate the job deviation cost represented by the state alignment relative entropy.

[0023] The evaluation result generation module is used to execute step S104 to determine and output the job performance evaluation index based on the job deviation cost of the optimal state alignment path relative to the baseline job rule set and the time characteristics of state transition.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. In the automatic evaluation of pilot training operations, a probabilistic state machine model is established to transform the determined operation procedure into a semantic topological space with state transition weights. The system calculates the maximum likelihood alignment path between the observation sequence and the standard path, enabling the evaluation logic to have the ability to cope with the uncertainty of the perception layer. When reasonable temporal fluctuations occur in the trainee's operation or instantaneous discrete noise is generated by the identification module, the logic continuity of the evaluation link is maintained by using prior probability, eliminating evaluation interruptions and false alarms caused by rigid logic matching.

[0026] 2. A multimodal time alignment algorithm is used to align video features, audio semantics, and flight data streams under a unified timestamp benchmark, constructing a cross-dimensional feature verification mechanism. Modal information forms logical complementarity in the calculation process. The behavior recognition results are dynamically arbitrated by the flight parameter change trend, so that the failure of a single modal sensor or environmental interference will no longer cause global evaluation bias, improving the stability of operation intention recognition in complex operating environments. A time sequence logic is used to hierarchically model the standard operating procedure, and a flexible time sequence mapping logic is combined to realize the nonlinear measurement of the operation process. The cumulative deviation between the actual operation path and the standard path is calculated based on the information distribution difference evaluation method, which describes the degree of fit between the trainee's operation trend and the standard pattern, extending the evaluation dimension from the single result right or wrong judgment to the mining of operation proficiency and decision consistency.

[0027] 3. Construct a dynamic adjustment mechanism based on mission importance deviation weights. The system calculates the impact cost of operational deviations on mission safety based on the flight phase to which the semantic state node belongs. The logical processing method enables the system to distinguish between non-critical temporal drift and critical procedure missing, making the evaluation process self-healing to actual engineering conditions and ensuring that the evaluation feedback results are consistent with flight safety requirements. Based on a unified structured event definition model, heterogeneous multi-source sensing data is transformed into standardized operational event sequences. This is deeply coupled with reasoning based on a rule knowledge base built on temporal logic. The computing architecture achieves semantic alignment between perceptual uncertainty and specification determinism. When there are short-term missing multimodal data or voice commands are interfered with by environmental noise, the system relies on rule-guided probabilistic state compensation to maintain the continuous operation of the evaluation flow, reducing the system's reliance on high-precision sensors and associated costs. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the method for constructing a probabilistic semantic state machine model and performing elastic temporal path search according to the present invention.

[0029] Figure 2 This is a curve comparing the accuracy of multimodal fusion recognition and single-modal recognition under different noise intensities.

[0030] Figure 3 This is a schematic diagram of the overall system architecture of the present invention, including the sensing and acquisition terminal, the core evaluation host, and the interactive feedback terminal. Detailed Implementation

[0031] The present invention will now be described in conjunction with the accompanying drawings. The embodiments described in this section are only for explaining the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0032] This invention provides an automatic evaluation method and system for pilot training operations based on rule-guided and artificial intelligence fusion. It transforms the sequential constraints of a benchmark operation rule set into a semantic topological space with a state transition probability matrix by constructing a probabilistic semantic state machine model. The system utilizes an artificial intelligence recognition algorithm to acquire multimodal features of the operation subject performing the operation and outputs a sequence of observed probability vectors representing the operation probability distribution at each time step. This sequence is mapped to the semantic topological space to recursively search for the optimal alignment path. The system calculates the relative entropy of state alignment to generate the operation deviation cost and determines the operation performance evaluation index accordingly. The system extracts the standard operation procedure logic from the benchmark operation rule set through a processor, defining each independent operation node as a semantic state node. A migration probability matrix is ​​established based on the temporal dependencies between each operation node. For node migration paths that conform to the logical order, a preset high migration probability is used, while for skip migrations that violate the logical order, a punitive low migration probability is used. The migration probability matrix is ​​used to establish logical guidance constraints in the semantic topology space. When there is instantaneous discrete noise in the observation probability vector sequence, the system fills in the missing operation logic positions according to the prior probability distribution to maintain the logical continuity of the evaluation link.

[0033] During operation, the system acquires real-time image sequences, audio data, and multi-dimensional state parameter streams via sensors. These multi-dimensional state parameter streams include parameters such as throttle position, rudder angle, speed, and altitude. The system utilizes a processor to run a multimodal time alignment algorithm, synchronizing the heterogeneous data under a unified timestamp reference to construct a cross-dimensional feature verification mechanism. The processor acquires a 100Hz synchronization pulse signal from the flight parameter channel, using it as a global clock reference to process 30fps image tensor sequences and 16kHz audio data streams. It performs third-order spline interpolation alignment operations, mapping heterogeneous features to a unified time point. A two-dimensional joint probability matrix is ​​constructed by calculating the joint probability distribution of visual and audio feature vectors. Mutual information entropy is calculated to represent the randomness of the perception layer output. Minimum and maximum values ​​are determined based on historical samples, and linear mapping is performed to adjust the logical constraint weight factors. If the confidence level of modal recognition is lower than a preset threshold, the observed probability vector sequence is weighted and reconstructed based on the complementary modal feature vectors. This outputs a confidence vector for operation events that satisfies the unitization constraint, filtering out non-realistic action phase features caused by hardware sampling delays. A cross-dimensional feature verification mechanism evaluates the contribution of each modal data to the representation of the current job state by calculating the mutual information entropy between different modal features. When the confidence level of a single modal feature is lower than a preset threshold, the observed probability vector sequence is weighted and reconstructed based on the feature vectors of complementary modalities. The artificial intelligence recognition algorithm adopts an architecture that integrates deep residual networks and long short-term memory networks. It captures the features of job behavior through a sliding time window, with a sampling frequency of no less than 100Hz and a response delay of no more than 50ms. This algorithm outputs a confidence vector for each operation event at each time step, forming an observed probability vector sequence representing the probability distribution of each operation at the current time. ; For a moment The observation probability vector, For time indexing.

[0034] The system performs a flexible time-series path search, which involves analyzing the observation probability vector sequence. Mapped to the semantic topology space, the optimal state alignment path, which is most likely aligned with the path defined in the baseline operation rule set, is found through recursive search to generate the operation deviation cost represented by the state alignment relative entropy. The path search module receives the observation probability vector sequence and semantic state nodes, and recursively calculates the cumulative path cost using dynamic programming. The initial state cost is the KL divergence between the observation vector and the starting node. The migration cost between nodes is determined by the negative logarithm mapping of the state migration probability matrix. The logical constraint weight factor is limited to a range of 0.2 to 0.8. Paths conforming to standard program logic are based on a preset high migration probability, while jump migrations violating temporal logic are based on a penalty constant. The recursive search finds the optimal state alignment path that is most likely aligned with the path defined in the baseline operation rule set. If the minimum cumulative KL divergence exceeds the logical fault tolerance threshold, the target behavior is judged to deviate from the rule set. An abnormal weight is assigned according to the hierarchical importance of the deviation position in the semantic topology space to calculate the compliance score. The operation deviation cost is calculated by calculating the cumulative path cost. Confirmed, the calculation formula is as follows: ,in, For a moment observation probability vector With semantic state nodes KL divergence between them The cost of migration between nodes, For logical constraint weighting factors, This represents the total number of moments.

[0035] The evaluation result generation module determines the job performance evaluation index based on the job deviation cost of the optimal state alignment path relative to the benchmark job rule set and the time characteristics of state transition. When the calculated minimum cumulative KL divergence exceeds the preset logical fault tolerance threshold, it determines that the target job behavior deviates from the benchmark job rule set, and assigns anomaly weights according to the hierarchical importance of the deviation position in the semantic topology space to calculate the compliance score. The system extracts the job rhythm characteristics of the job subject by analyzing the residence time distribution between each node in the optimal state alignment path, and compares it with the benchmark behavior model to identify the cognitive load state of the job subject when performing complex tasks, and finally generates a dimension evaluation report for skill proficiency. The probabilistic semantic state machine model supports dynamic updates. The system adjusts the relaxation factor in the migration probability matrix according to the high-frequency operation deviation patterns in the historical job process to adapt to the reasonable operation fluctuation range of job subjects with different experience levels. The alignment difference between the optimal state alignment path and the standard path is visualized and the correction instructions are output synchronously on the job interaction terminal. Logical guidance constraints are used to provide real-time tutoring feedback with semantic continuity to the job subject.

[0036] This invention also provides an automatic evaluation system for pilot training operations based on rule-guided and artificial intelligence fusion, comprising a model building module, a multimodal feature extraction module, a path search module, and an evaluation result generation module. The model building module executes the step of constructing a probabilistic semantic state machine model. The multimodal feature extraction module extracts a sequence of multimodal observation probability vectors. The path search module performs a flexible temporal path search and generates an operational deviation cost. The evaluation result generation module determines and outputs operational performance evaluation indicators based on the optimal state alignment path. All modules of the system work collaboratively through a unified timestamp interface, realizing a shift in flight training evaluation from experience-based judgment to data-driven approaches. During the determination of operational performance evaluation indicators, the system executes a skill proficiency quantification procedure based on node dwell time distribution, and the processor obtains each semantic state node in the current optimal state alignment path. Actual stay time and compared with the reference dwell time pre-stored in memory. Compare to calculate the time deviation rate When the time deviation rate exceeds 20% at critical operation nodes and the path cumulative cost When the value is between 0.8 and 1.2, the processor determines that the operator is in a state of high cognitive load and identifies whether the behavior is a lag deviation caused by slow operation or an advance deviation caused by rapid operation based on the positive or negative value of the deviation. In this way, the processor completes the quantitative perception of the pilot's psychological load state by extracting the characteristics of the operation rhythm.

[0037] Example 1: In a training scenario simulating an engine shutdown emergency response, the operator must execute a standard operating procedure with sequential constraints within 30 seconds, involving throttle cutoff, fuel shut-off, emergency communication, and attitude correction. Under this condition, the trainee pilot exhibits atypical physical movements due to the emergency, and combined with image acquisition noise generated by simulated cockpit vibrations, the confidence level of a single image feature output by the multimodal feature extraction module fluctuates between 0.4 and 0.6. If the identification tag is missing or misaligned for more than 0.5 seconds on the timeline, the conventional evaluation logic is interrupted because it cannot match the rigid rule step size. The system receives the observation probability vector sequence output by the multimodal feature extraction module through the processor. The system maps the image channel to a probabilistic semantic state machine model generated from the standard operating procedure. When the image channel lacks landing gear operation features due to field obstruction, the system calculates the mutual information entropy between the landing gear lowering command features in the audio channel and the landing gear displacement signal in the flight parameter channel. Based on the confidence distribution of complementary modes, the system logically completes the observation probability vector sequence. The weighted reconstruction of the multimodal probability vector provides an input benchmark for path search that includes the confidence of the perception layer, enabling the evaluation process to maintain continuous tracking of the operational status when the physical perception link is partially damaged.

[0038] The path search module performs a recursive search within the semantic topological space, utilizing time... observation probability vector With semantic state nodes The system finds the optimal alignment path by aligning the relative entropy between states and calculating the cumulative cost of the path. To identify the intention of the task, the calculation formula is as follows: ,in, Accumulate cost for the path, For a moment observation probability vector With semantic state nodes KL divergence between them This is a logical constraint weighting factor, and in this embodiment, it takes a value of 0.5. For the previous semantic state node Migrate to the current semantic state node The cost of migration For the total sampling time, although the temporal offset causes the divergence value to increase at specific times during the calculation, the calculated value is still valid because the offset path has a priori migration probability in the migration probability matrix. The value eventually converges to within the logical fault tolerance threshold of 1.5. The evaluation result generation module determines the compliance of the emergency response behavior and outputs a score of 95 based on the cost of the operation deviation generated by the optimal alignment path. The system extracts the operation rhythm characteristics of the operation subject by the dwell time of each semantic state node in the alignment path. When the duration distribution of the execution of the ground notification node deviates from the benchmark behavior model by 15%, it outputs the cognitive load quantification technical index of the trainee in multi-task and parallel processing. This result realizes the perception of nonlinear operation trends and completes the evaluation of operation quality while retaining the rule logic constraints.

[0039] Example 2: The verification test used a simulated flight cockpit platform integrating multi-view vision sensors, acoustic pickup arrays, and a fast access recorder interface. The sampling frequency was set to 100Hz and the real-time response delay of the data bus was 45.2ms, where the logical constraint weighting factor was... The value is determined based on the observation probability vector sequence. The average information entropy within the sliding time window is determined when the perception layer is affected by sudden changes in illumination or simulated aircraft vibration, leading to an alteration in the observation probability vector sequence. When the average information entropy rises above 0.5, the system will execute the prior guidance enhancement rules. Adjusting it to 0.8 increases the constraint weight of the semantic topological space on path search; conversely, when the average information entropy is below 0.3, it will... The signal was adjusted to 0.2 to enhance the sampling sensitivity of real-time observation data. During the experiment, Gaussian white noise with a signal-to-noise ratio of 20dB was actively injected into the visual channel and superimposed with 50Hz power frequency electromagnetic harmonic interference to construct a non-ideal working condition.

[0040] The experiment obtained performance data by comparing the deviation rate of evaluation indicators between the experimental group (the sample group of this invention) and the control group under the simulated engine shutdown emergency handling subject. The control group adopted a discrete action label matching method based on hard threshold criteria, while the experimental group executed a flexible temporal path search procedure using a probabilistic semantic state machine model. During the 1.5-second operation interval from throttle cutoff to fuel shut-off performed by the trained pilot, the action confidence score output by the image feature extraction module fluctuated drastically between 0.35 and 0.52 due to the influence of environmental noise injection. At this time, the control group determined that the operation node was lost and the evaluation link was interrupted because it could not obtain action labels that met the 0.6 confidence threshold. The experimental group, however, calculated the deviation rate of evaluation indicators between the experimental group and the control group at each time step. observation probability vector With semantic state nodes The relative entropy of state alignment between states, and the cumulative cost of recursively searching for paths within the semantic topological space. The alignment path for the minimum value is shown in Table 1.

[0041] Table 1: Example of performance measurement data for the evaluation system under different noise interference intensities

[0042]

[0043] As shown in Table 1, during the process of increasing the injected noise intensity from 40dB to 20dB, although the average information entropy of the observation probability vector increases from 0.115 to 0.521, indicating an increase in the uncertainty of the sensing layer signal, the path accumulation cost is mitigated because the system utilizes the prior logic constraints provided by the state transition probability matrix to perform probability alignment on the observation sequence. The measured values ​​consistently remained within the logic fault tolerance threshold of 1.5, thus keeping the deviation rate of the final output evaluation metric below 3.12%. However, when the injected noise intensity further increased to 10 dB, the path cumulative cost... It increased to 1.785, accompanied by a significant growth rate in the deviation rate of the evaluation indicators.

[0044] Example 3: This example combines Figures 1 to 3 This document describes an automatic evaluation method and system for pilot training operations based on rule-guided instruction and artificial intelligence. Figure 1As shown, the process is initiated in parallel based on the benchmark operation rule set and the multimodal features of the operation subject. Step S101 executes the construction of a probabilistic semantic state machine model, specifically transforming the rule set constraints into a semantic topology space and establishing a state transition probability matrix. Simultaneously, step S102 executes the extraction of multimodal observation probability vector sequences, using artificial intelligence recognition algorithms to obtain multimodal features and outputting a sequence representing the operation probability distribution at each time step. The data streams output from the above two steps converge into step S103 to perform elastic temporal path search. During this process, the observation probability vector sequence is mapped to the semantic topology space, the state alignment relative entropy is calculated, and then the optimal alignment path is recursively searched to generate the operation deviation cost. Finally, the process enters step S104 to generate automated evaluation results.

[0045] like Figure 2 As shown, in a two-dimensional coordinate system with noise intensity in dB as the abscissa and recognition accuracy in % as the ordinate, performance curves for three different recognition modes are presented. The solid line at the top corresponds to the multimodal fusion recognition accuracy, the dashed line in the middle corresponds to the single audio modality recognition accuracy, and the dashed line at the bottom corresponds to the single visual modality recognition accuracy. The three curves show the corresponding recognition accuracy value distribution within the noise intensity range of 5 to 40 marked on the horizontal axis. Figure 3 As shown, the overall architecture of the system mainly consists of a perception and data acquisition end, a data processing and evaluation end, and an interaction and feedback end. The perception and data acquisition end includes a flight simulator cockpit platform, which is equipped with a multi-view vision sensor group for acquiring image sequences, an acoustic pickup array for acquiring audio data, and a flight parameter data interface for acquiring state parameter streams. The above data is transmitted to the core evaluation host of the data processing and evaluation end in the form of a real-time signal stream via a data bus. The central processing unit (running environment) in this host runs a model building module, a multimodal feature extraction module, a path search module, and an evaluation result generation module, and interacts with a memory (database / knowledge base) containing a baseline operation rule set, a probabilistic semantic state machine model, and historical training sample data for data retrieval and storage. The generated correction instructions and evaluation results are finally transmitted to the operation interaction terminal of the interaction and feedback end, which includes a real-time tutoring feedback interface and a dimension evaluation report display function.

[0046] Example 4: In a full-process simulated flight training scenario involving operators with different flight experience levels, the processor faces challenges such as temporal distribution differences caused by individual operating habits of trainees, gain drift and increased dark current noise from airborne sensors as operating time increases. If the transition probability matrix and logic fault tolerance threshold in the probabilistic semantic state machine model remain at static initial settings, it can lead to false alarms when operators with higher experience levels perform fine-tuning of non-critical nodes, or path search failures when the sensor signal-to-noise ratio decreases. This fluctuation in evaluation sensitivity caused by the mismatch between initial parameters and the physical environment poses an objective challenge to the stability of the automatic evaluation system. The system uses the processor to call 500 sets of historical training sample data pre-stored in memory that conform to the baseline operation rule set as input to execute a calibration procedure to initialize the probabilistic semantic state machine model. This calibration procedure defines each independent procedure in the standard operating procedure as a semantic state node. And calculate the average time interval between transitions between adjacent processes in the historical samples. with standard deviation Assign values ​​to the off-diagonal elements of the migration probability matrix, where the node migration probabilities conform to the standard operating procedure logic. The cumulative probability density value is set based on the Gaussian distribution function, while the jump transition probability that violates temporal logic is uniformly based on a penalty constant of 0.01. In the multimodal feature extraction process, the deep residual network consists of 18 cascaded convolutional layers to extract image spatial features. Its output tensor is concatenated with the hidden layer vectors of the long short-term memory network along the time axis, forming a dimension... Operation event confidence vector; logical constraint weighting factor The adaptive adjustment procedure is based on the observation probability vector sequence Real-time mutual information entropy Perform a linear mapping, calculated using the following formula: ,in, For logical constraint weighting factors; It is the lower limit of its value range, and its value is 0.2; It represents the upper limit of its value range, which is 0.8; The mutual information entropy of the multimodal features at the current moment is obtained by calculating the joint probability distribution of the visual feature vector and the audio feature vector, and is used to characterize the randomness of the output of the perception layer. The minimum value of the preset mutual information entropy benchmark; The preset maximum value of mutual information entropy; when sensor aging causes an increase in image channel noise power... When increased, the system adjusts. The numerical value is used to enhance the semantic topology space's correction constraint on the alignment path, and the logical fault tolerance threshold is determined to be 1.5 based on the 95th percentile of the cumulative path cost in historical samples.

[0047] When calculating compliance scores, the system executes an evaluation weight allocation procedure based on node importance. The processor divides the semantic topology space into a critical path region and a general operation region according to the risk level in the baseline job rule set, and assigns semantic state nodes in the critical path region during the compliance calculation process. Assign an evaluation weight of no less than 0.7. The evaluation weight for nodes in the general operation area is no higher than 0.3. The final compliance score is obtained by weighted accumulation of the relative entropy of each node's state alignment and its corresponding evaluation weight. This procedure uses a physical risk mapping mechanism to solve the problem of quantifying the unequal contribution of different types of operational deviations to the evaluation results. After applying the above calibration parameters, the evaluation result generation module performs path search and deviation quantification on the operational behaviors of pilots of different experience levels under the same noise conditions. Under stress testing where the sensor image signal-to-noise ratio drops to 15dB, the system adaptively adjusts the logical constraint weight factor. The path cumulative cost is reduced to 0.72. The measured values ​​converged between 1.15 and 1.38, filtering out transient perception glitches caused by hardware aging and identifying the trainee pilot's premature touch of the fuel shut-off handle. The final output of the operational proficiency score maintained 98.5% consistency with the on-site assessment results of the human instructor.

[0048] Example 5: In the application scenario of deploying the automatic evaluation system to a new type of flight simulator, by reading 200 hours of expert-level flight control data stream corresponding to the standard operating procedures of this aircraft as input to the initial state definition procedure, the processor constructs a transition probability matrix based on the transition frequency between semantic state nodes in the historical training sample data, and performs normalization processing using the following formula to establish the initial probability distribution of the semantic topology space: ,in, To start from semantic state nodes Migration to semantic state node The migration probability; For nodes in historical training sample data Next adjacent node Frequency of occurrence; This represents the total number of semantic state nodes. Before the system enters the real-time evaluation phase, this procedure fills in the probability weights of each node in the semantic topology space. When the system faces timing deviations caused by the asymmetry of sampling frequencies between the visual and audio channels, the processor uses the 100Hz synchronization pulse signal output from the flight parameter channel as the reference time axis. It performs interpolation alignment operations on the image tensor sequence with a sampling rate of 30fps and the audio data stream with a sampling rate of 16kHz. The processor maps heterogeneous features to a unified time point, calculates the mutual information entropy between modal feature vectors at each time step, and performs weighted reconstruction of the observation probability vector sequence. The input path search module filters out non-realistic action phase features caused by hardware sampling delay. In 10 sets of tests including the impact of random clock jitter, the system outputs the cumulative path cost. The measurement standard deviation is kept within 0.05; before collecting the image and voice data of the main body of the operation, the system performs a data de-identification procedure. The processor performs local feature transformation on the collected original multimodal feature stream, and replaces the original pixel and voiceprint information by extracting non-biometric motion vectors and abstract semantic text. All data collection and processing links are triggered based on the authorization interface of the main body of the operation, ensuring that the automatic evaluation system runs at the level of de-identified data while avoiding the risk of data backtracking against specific natural persons from the underlying logic level.

[0049] Example 6: In the application of an automatic evaluation system deployed in a new type of flight simulator cockpit, an offline estimation procedure for the joint distribution of multimodal features is executed. 50 hours of synchronized audiovisual signal streams in the cockpit environment are collected and used as input. The visual feature vector and audio feature vector are divided into 10 equidistant dispersion intervals in the amplitude dimension. The processor constructs a dimension of... Two-dimensional joint probability matrix , where matrix elements Representing visual features is in the first The interval and the audio feature is in the first The processor uses the matrix representing the joint occurrence frequency of intervals to calculate the mutual information entropy between different modes and stores it as a reference data in memory. It is a two-dimensional joint probability matrix. For matrix elements, For visual feature interval index, For audio feature interval indexing, this procedure determines the coupling weight of heterogeneous channels in a cockpit environment through a defined data filling process.

[0050] After obtaining the optimal state alignment path, the system executes a feedback instruction generation procedure based on the logical correction mapping table, and monitors the output of the path search module at every moment in real time. The relative entropy of the state alignment is calculated. When the calculated relative entropy value is greater than the logical fault tolerance threshold of 1.5 for five consecutive sampling points, the processor extracts the current semantic state node. index value and observation probability vector sequence The processor retrieves a unique correction instruction code from memory based on the index value for the operation category corresponding to the maximum value of the medium probability component, and converts the correction instruction code into a visual warning signal to be synchronized to the operation interaction terminal. In 100 tests targeting behaviors that deviate from the standard operating procedure, the accuracy of the feedback instruction generation remains at 99.2% and the system's end-to-end response latency remains constant within 50ms.

[0051] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion, characterized in that, Includes the following steps: Step S101: Construct a probabilistic semantic state machine model; convert the sequential constraints in the baseline job rule set into a semantic topology space with a state transition probability matrix, where each job node corresponds to a semantic state node in the probabilistic semantic state machine model, and the temporal logic constraints between nodes are mapped to the transition probabilities within the probabilistic semantic state machine model. Step S102: Extract the multimodal observation probability vector sequence; use artificial intelligence recognition algorithms to obtain the multimodal features of the working subject in the process of performing the target operation, and output the confidence vector of each operation event at each time moment to form an observation probability vector sequence that represents the probability distribution of each operation at the current time. Step S103: Perform elastic temporal path search; map the observation probability vector sequence to the semantic topology space, calculate the state alignment relative entropy between the observation probability vector sequence and the state transition path, and recursively search for the optimal state alignment path that is aligned with the maximum likelihood of the path defined by the baseline job rule set in the semantic topology space to generate the job deviation cost characterized by the state alignment relative entropy. Step S104: Generate automated evaluation results; determine and output job performance evaluation indicators based on the job deviation cost of the optimal state alignment path relative to the baseline job rule set and the time characteristics of state transition.

2. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, The method for constructing the migration probability matrix in step S101 includes: identifying the temporal dependencies between each operation node in the baseline operation rule set; using a preset high migration probability for node migration paths that conform to the logical order, and using a punitive low migration probability for skip migrations that violate the logical order; and using the migration probability matrix to establish logical guidance constraints in the semantic topology space, so as to fill in the missing operation logic positions based on the prior probability distribution when there is instantaneous discrete noise in the observation probability vector sequence.

3. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, Step S102 includes: acquiring real-time image sequences, audio data, and multi-dimensional state parameter streams of the subject performing the target task; using a multi-modal time alignment algorithm to synchronously process the image sequences, audio data, and multi-dimensional state parameter streams under a unified timestamp benchmark to construct a cross-dimensional feature verification mechanism; and performing dynamic arbitration on the behavior recognition results based on the changing trend of the multi-dimensional state parameter stream.

4. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 3, characterized in that, The cross-dimensional feature verification mechanism evaluates the contribution of each modality data to the representation of the current operation status by calculating the mutual information entropy between different modal features. When the recognition confidence of a single modality feature is lower than a preset threshold, the observation probability vector sequence is reconstructed by weighting based on the feature vectors of complementary modalities.

5. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, In step S103, the cost of operation deviation is calculated by accumulating the path cost. Sure: ,in, For a moment observation probability vector With semantic state nodes KL divergence between them The cost of migration between nodes, This is the logical constraint weighting factor.

6. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, The method for determining the cost of job deviation in step S104 includes: when the calculated minimum cumulative KL divergence exceeds the preset logical fault tolerance threshold, the target job behavior is determined to deviate from the baseline job rule set; according to the hierarchical importance of the deviation position in the semantic topology space, anomaly weights are assigned, and a compliance score is calculated.

7. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, Step S104 further includes: extracting the work rhythm characteristics of the main operator by analyzing the residence time distribution among nodes in the optimal state alignment path; comparing the work rhythm characteristics with the benchmark behavior model to identify the cognitive load state of the main operator when performing complex tasks, and generating a dimensional assessment report on skill proficiency.

8. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, The probabilistic semantic state machine model supports dynamic updates; the system adjusts the relaxation factor in the transition probability matrix based on high-frequency operation deviation patterns in historical operations.

9. The automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, Step S104 and subsequent steps include: visually converting the alignment difference between the optimal alignment path and the standard path; synchronously outputting correction instructions on the operation interaction terminal; using logical guidance constraints to provide real-time guidance feedback with semantic continuity to the operation subject; the extraction process of the observation probability vector sequence in step S102 is achieved in the following way: adopting an architecture that integrates deep residual networks and long short-term memory networks; capturing the instantaneous features and long-term temporal features of the operation behavior through a sliding time window, with a sampling frequency of not less than 100Hz and a response delay of not more than 50ms, and converting the recognition results into a probability distribution vector that satisfies the unitization constraint.

10. An automatic evaluation system for pilot training operations based on rule-guided and artificial intelligence fusion, used to implement the automatic evaluation method for pilot training operations based on rule-guided and artificial intelligence fusion as described in claim 1, characterized in that, include: The module includes a model building module, a multimodal feature extraction module, a path search module, and an evaluation result generation module. The model building module is used to perform step S101 to transform the sequential constraints in the baseline job rule set into a semantic topological space with a state transition probability matrix. The multimodal feature extraction module is used to execute step S102 to obtain the multimodal features of the working subject in the process of performing the target work behavior, and output the confidence vector of each operation event at each time moment to form an observation probability vector sequence that represents the probability distribution of each operation at the current time. The path search module is used to perform step S103 to map the observation probability vector sequence to the semantic topology space, calculate the state alignment relative entropy between the observation probability vector sequence and the state transition path, and recursively search for the optimal state alignment path in the semantic topology space to generate the job deviation cost represented by the state alignment relative entropy. The evaluation result generation module is used to execute step S104 to determine and output the job performance evaluation index based on the job deviation cost of the optimal state alignment path relative to the baseline job rule set and the time characteristics of state transition.

Images