Flight simulator test data state identification method, system and equipment based on multi-modal learning

Abstract

The invention belongs to the field of aircraft simulation, particularly relates to a flight simulator test data state identification method, system and equipment based on multi-modal learning, and aims to solve the problems of low verification efficiency, poor compliance and difficulty in comprehensive identification of equipment risks. The method comprises the following steps: acquiring a time sequence operation curve image, a standard document text and an equipment time sequence, and generating a multi-dimensional equipment state representation vector through a feature extraction network after standardization processing; analyzing a standard document, constructing a structured knowledge base, extracting an initial allowable deviation range, fusing environmental parameters, and calculating a health index by using a health state decision tree to adaptively adjust the upper and lower limits of a deviation interval; and performing intelligent compliance judgment in combination with the compliance probability output by the state classification model and the dynamic deviation range. According to the invention, through collaborative decision-making of multi-modal data analysis, deviation range optimization and equipment health state quantitative evaluation, the accuracy and environmental adaptability of flight simulator test verification are improved.

Description

Background Art

[0002] Flight simulators are core tools for aviation training and equipment verification, and their performance and data accuracy directly affect the safety and effectiveness of flight training. According to current international standards and domestic civil aviation regulatory requirements, the annual appraisal of flight simulators requires manual verification of more than a hundred objective test items. Taking the Airbus A320 flight simulator as an example, a single full-course verification requires 1 engineer to spend 1 day, and approximately 60% of the time is consumed in the manual comparison of curve data with the MQTG standard.

[0003] In the prior art, there are the following significant defects in manual verification: Insufficient error sensitivity: The traditional visual inspection method has a high omission rate for subtle deviations of ±2% standard deviation, making it difficult to meet the requirements of high-precision verification. For example, small deviations in key data such as the control response curve and engine parameter curve of a flight simulator may cause the training effect to deviate from the real flight scenario, and manual comparison is prone to missing such problems due to visual fatigue or subjective judgment errors.

[0004] Lagging standard update: International (such as ICAO, EASA) and domestic civil aviation standards (such as CAAC) are revised and updated, but manual verification relies on engineers' subjective understanding of the standards. After the standards are updated, retraining needs to be reorganized, resulting in the disconnection between the verification process and the latest standards and affecting the compliance of verification results.

[0005] Lack of data correlation: Existing verification methods analyze individual test items in isolation and cannot identify systematic deviations in the device state. For example, sensor errors, software algorithm drift, or hardware aging of a flight simulator may cause correlation anomalies in multiple test items, but manual verification lacks the ability to analyze global data and it is difficult to detect potential device risks in a timely manner.

[0006] The above defects lead to low efficiency (long time-consuming for single verification), high cost (reliance on human input) in flight simulator verification, and the accuracy and comprehensiveness of verification results are limited, unable to meet the high requirements of aviation safety for device performance. Therefore, there is an urgent need for an automated and intelligent data processing method to solve the problems of insufficient error sensitivity, lagging standard update, and lack of data correlation in the prior art, and to improve the accuracy, efficiency, and compliance of flight simulator verification.

[0007] Based on this, the present invention proposes a method, system, and device for identifying the state of flight simulator test data based on multi-modal learning. Summary of the Invention

[0008] To solve the above problems in the prior art, namely, the existing flight simulator calibration relies on manual calibration, with three major technical defects: insufficient error sensitivity (high omission rate of ±2% standard deviation), lagging standard updates (requiring retraining after revision), and lack of data correlation (inability to identify systematic deviations), resulting in low calibration efficiency, poor compliance, and difficulty in comprehensively identifying equipment risks. The present invention provides a method, system, and device for identifying the state of flight simulator test data based on multimodal learning.

[0009] In the first aspect of the present invention, a method for identifying the state of flight simulator test data based on multimodal learning is proposed. The method includes Collecting multimodal test data of the flight simulator and performing standardization processing. The multimodal test data includes time-series operation curve images, standard document texts, and device operation time series; Performing feature extraction on the standardized multimodal test data based on a feature extraction network, and obtaining a multi-dimensional device state representation vector through feature splicing; Constructing a structured knowledge base by parsing the standard document text, and extracting the initial allowable deviation range for each test item; based on the multi-physical parameters in the device state representation vector, calculating the environment-physical correlation feature vector in combination with environmental parameters, and inputting it into a health state decision tree to calculate the health index. According to the health index, dynamically adjusting the upper and lower limits of the initial allowable deviation interval to obtain the final allowable deviation range; Inputting the device state representation vector into a pre-constructed and trained state classification model, outputting a compliance probability value, and generating a final compliance determination result in combination with the final allowable deviation range.

[0010] Further, collecting multimodal test data of the flight simulator and performing standardization processing specifically includes: Extracting the key point coordinates of the curve from the time-series operation curve image through a visual processing model, and performing spatial alignment with the time axis; Extracting the test item names, parameter thresholds, and constraint conditions from the standard document text through a natural language processing model; Decomposing the device operation time series into a periodic component, a trend component, and a noise component.

[0011] Further, performing feature extraction on the standardized multimodal test data based on a feature extraction network to obtain image modality features, text modality features, and time-series modality features; The image modality features are extracted by a deep convolutional network to obtain the spatial distribution pattern of the curve image, including the curvature change between key points and the histogram of gradient directions; The text modality features are extracted by a semantic encoder to obtain the parameter thresholds and constraint conditions in the standard document, including numerical threshold intervals and logical judgment rules; For time - series modal features, it includes extracting the main frequency feature of the periodic component, calculating the long - term change slope of the trend component, and analyzing the abnormal fluctuation intensity of the noise component; Concatenate the image modal features, text modal features and time - series modal features, and input them into a deep auto - encoder to generate a multi - dimensional device state representation vector.

[0012] Furthermore, based on the multi - physical parameters in the device state representation vector, combined with environmental parameters, calculate the environment - physics correlation feature vector, and input it into the health status decision tree to calculate the health index. The specific calculation method is as follows: Input the device state representation vector into a pre - constructed and trained physical parameter decoder to generate physical parameters, where the physical parameters include mechanical wear amount, control response error and sensor degradation degree; Collect the real - time environmental parameters of the flight simulator, where the real - time environmental parameters include temperature data, humidity data and electromagnetic intensity data; Construct an environmental parameter influence matrix, conduct multi - dimensional coupling analysis on the physical parameters and real - time environmental parameters to generate an environment - physics correlation feature vector; Input the environment - physics correlation feature vector into the health status decision tree, output the verification result, and normalize the verification result into a health index. The health index is a continuous value within the range of [0, 1], where 0 represents complete failure and 1 represents the best operating state.

[0013] Furthermore, dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range, specifically: Based on the magnitude of the health index, divide the device state levels, where the levels include normal state, critical state and abnormal state; When the health index belongs to the normal state, expand the upper and lower limits of the initial allowable deviation interval according to a preset first ratio to form a relaxed final allowable deviation range; When the health index belongs to the critical state, keep the upper and lower limits of the initial allowable deviation interval unchanged as the final allowable deviation range; When the health index belongs to the abnormal state, shrink the upper and lower limits of the initial allowable deviation interval according to a preset second ratio to form a tightened final allowable deviation range; Among them, the health index corresponding to the normal state is higher than that of the critical state, and the health index corresponding to the critical state is higher than that of the abnormal state.

[0014] Furthermore, combine the final allowable deviation range to generate a final compliance determination result, specifically including: Based on the compliance probability value output by the state classification model, extract the comparison result between the real - time operating parameters corresponding to each test item and the final allowable deviation range; If the compliance probability value of a certain test item is higher than the preset threshold and its real-time operating parameters are within the final allowable deviation range, then the test item is determined to be in a compliant state; If the compliance probability value is lower than the preset threshold or the real-time operating parameters exceed the final allowable deviation range, it is determined to be in an abnormal state and the specific deviation parameters are marked; After traversing all test items, if there is at least one test item determined to be in an abnormal state, a determination result of unqualified operation of the overall device is generated; otherwise, the final determination conclusion of full-item compliance is output; Among them, the preset threshold is a dynamic threshold set according to the constraint conditions in the standard document, and the determination result of the abnormal state synchronously triggers the adaptive iterative update mechanism of the deviation range.

[0015] Furthermore, after the method generates the final compliance determination result, it further includes: Conduct an association analysis on the compliance results of multiple test items to locate the device components that cause systematic deviations; Generate a visualization report, marking the deviation values, dynamic allowable intervals, and historical change trends of the out-of-standard test items.

[0016] On the other hand, the present invention proposes a flight simulator test data status recognition system based on multi-modal learning. According to a flight simulator test data status recognition method based on multi-modal learning, the system includes: A data acquisition and processing module configured to acquire multi-modal test data of the flight simulator and perform standardization processing. The multi-modal test data includes a time-series operation curve image, a standard document text, and device operation time series; A feature extraction module configured to extract features from the standardized multi-modal test data based on a feature extraction network and obtain a multi-dimensional device status representation vector through feature splicing; A deviation range construction module configured to construct a structured knowledge base by parsing the standard document text and extract the initial allowable deviation range of each test item; based on the multi-physical parameters in the device status representation vector, calculate the environment-physical correlation feature vector in combination with environmental parameters and input it into a health status decision tree to calculate the health index, and dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range; A determination module configured to input the device status representation vector into a pre-constructed and trained status classification model, output a compliance probability value, and generate a final compliance determination result in combination with the final allowable deviation range.

[0017] In the third aspect of the present invention, an electronic device is proposed, including: At least one processor; and A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned method for identifying the state of flight simulator test data based on multimodal learning.

[0018] In a fourth aspect of the present invention, a computer-readable storage medium is proposed. The computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned method for identifying the state of flight simulator test data based on multimodal learning.

[0019] Advantages of the present invention: Improve error sensitivity and verification accuracy: Through the multimodal feature extraction network, in-depth modeling of the test data is carried out, combining multi-dimensional information of images, texts, and time series, significantly enhancing the ability to identify subtle deviations. Compared with the traditional visual inspection method, it can effectively reduce the problem of missed inspections caused by visual fatigue or subjective judgment, ensuring that the verification results of key parameters (such as control response, engine performance, etc.) are closer to the requirements of the real flight scenario.

[0020] Dynamically adapt to standard updates and ensure compliance: By parsing the standard document text to construct a structured knowledge base and dynamically adjusting the allowable deviation range based on the device status representation vector, the limitation of manual verification relying on the subjective understanding of engineers is broken through. This method can respond in real time to international / domestic standard revisions, and can keep the verification process synchronized with the latest standards without additional training, significantly improving the compliance and timeliness of the verification results.

[0021] Realize the global identification of systematic deviations: The multi-dimensional device status representation vector integrates the temporal characteristics and correlation information of the device operation, and can detect abnormal correlations of multiple test items caused by potential factors such as sensor errors, software drift, or hardware aging. Compared with the traditional method of analyzing individual test items in isolation, the present invention can identify the systematic deviations of the device status from a global perspective, early warning of potential risks, and avoiding the overall decline of device performance caused by the accumulation of local problems.

[0022] Reduce labor costs and verification time: Through automated feature extraction, health index calculation, and state classification model reasoning, replacing manual comparison and empirical judgment, significantly shortening the time-consuming of a single verification. At the same time, the standardized processing process reduces the dependence on the individual experience of engineers, reduces the training and labor input costs, and improves the verification efficiency.

[0023] Enhance the adaptability and forward-looking nature of equipment status assessment: The dynamic adjustment mechanism of the health index can flexibly correct the allowable deviation range according to the actual operating status of the equipment (such as the degree of aging, environmental changes, etc.), avoiding misjudgment or missed judgment problems caused by static thresholds. Combining with the compliance probability value output by the status classification model can provide data support for equipment maintenance strategies, extend the service life of the equipment, and optimize resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objectives, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings: Figure 1 It is a flowchart showing a method for identifying the status of flight simulator test data based on multi-modal learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The following further elaborates on the present application with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant invention and do not limit the invention. Additionally, it should be noted that for the sake of description, only parts related to the relevant invention are shown in the drawings.

[0026] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The following will detail the present application with reference to the drawings and embodiments.

[0027] The present invention provides a method for identifying the status of flight simulator test data based on multi-modal learning, and the method includes Step S10: Collect multi-modal test data of the flight simulator and perform standardization processing. The multi-modal test data includes a time-series operation curve image, a standard document text, and an equipment operation time series; Step S20: Extract features from the standardized multi-modal test data based on a feature extraction network, and obtain a multi-dimensional equipment status representation vector through feature splicing; Step S30: Construct a structured knowledge base by parsing the standard document text, and extract the initial allowable deviation range for each test item; based on the multi-physical parameters in the equipment status representation vector, calculate the environment-physical correlation feature vector in combination with environmental parameters, and input it into a health status decision tree to calculate the health index, and dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range; Step S40: Input the equipment status representation vector into a pre-constructed and trained status classification model, output a compliance probability value, and combine it with the final allowable deviation range to generate a final compliance determination result.

[0028] To more clearly illustrate a method for identifying the status of flight simulator test data based on multimodal learning of the present invention, the following will be combined with Figure 1 elaborate on each step in the embodiments of the present invention.

[0029] A method for identifying the status of flight simulator test data based on multimodal learning according to the first embodiment of the present invention includes steps S10 - S40, and each step is described in detail as follows: Step S10, collect multimodal test data of the flight simulator and perform standardization processing. The multimodal test data includes a time-series operation curve image, a standard document text, and equipment operation time series; In this embodiment, collecting multimodal test data of the flight simulator and performing standardization processing specifically includes: Extract the curve key point coordinates from the time-series operation curve image through a visual processing model and perform spatial alignment with the time axis; The present invention first grayscales the time-series operation curve image, removes noise through Gaussian filtering, locates the curve region through Canny edge detection, and uses the YOLOv8 model to identify the curve endpoints, inflection points, and extreme points, and outputs pixel coordinates.

[0030] The method of performing spatial alignment with the time axis is as follows: Calculate the numerical axis scale factor: Extract the numerical range marked on the vertical axis of the time-series operation curve image and the corresponding pixel height, and calculate the scale factor: Scale factor = vertical axis pixel height / numerical range.

[0031] Convert pixel position to actual value: Actual value = maximum value of the vertical axis - (vertical pixel coordinate of the key point / numerical axis scale factor).

[0032] Among them, the maximum value of the vertical axis is the maximum value marked on the vertical axis of the image, and the vertical pixel coordinate of the key point is the y-direction coordinate of the detected curve key point.

[0033] Extract the test item name, parameter threshold, and constraint conditions from the standard document text through a natural language processing model; Among them, in the present invention, the natural language processing model uses a bilingual pre-trained model based on Transformer as the core architecture and performs domain adaptation for the characteristics of aviation standard documents: Annotate 3,200 ICAO, FAA, and CAAC standard documents (Chinese-English bilingual), and construct a corpus containing the following tags: Test item name: such as "Pitch attitude holding test", "Engine windmill state test"; Parameter threshold: including numerical type (±2%), conditional type (≤70% when V≥200 knots), and function expression type; Constraints: such as equipment status constraints, environmental condition constraints, test item dependency constraints, and time validity constraints.

[0034] Use a dynamic masking strategy to enhance the capture of long-distance dependencies, improve the relevance between clause numbers and test items, introduce a Pointer Network to accurately locate the start and end positions of threshold parameters in the text, and train a multi-task learning framework to synchronously complete named entity recognition (NER) and relation extraction (RE).

[0035] The test items in the present invention represent a standardized description of the performance verification objectives of specific devices, and need to include test objects and test types, including modules such as flight control systems, power plants, and avionics systems.

[0036] In the present invention, the rules for extracting test item names are as follows: Match the gerund structure in the clause title (such as "XXXX test", "YYYY verification"); Parse the adverbial clause of the test purpose (such as "evaluate the deflection efficiency of the rudder under crosswind conditions" → extract "rudder crosswind deflection test"); Establish a synonym mapping table (such as "check" → "test", "verify" → "validate").

[0037] In this embodiment, the numerical parameter thresholds include absolute value type and percentage type; For the absolute value type, such as "pitch angle deviation ≤ 1.5°", the parsed output form is: {maximum threshold: 1.5, unit: degree}.

[0038] For the percentage type, such as "airspeed error ± 2%", the parsed output form is: {minimum threshold: -2%, maximum threshold: +2%}.

[0039] For the conditional type, such as "when the altitude ≥ 10,000 feet, the slope limit is ± 25°", the parsed output form is: {condition: altitude ≥ 10,000 feet, threshold: {minimum value: -25°, maximum value: +25°}}.

[0040] For the function expression type, such as "roll rate threshold = 0.8×V / 250 (V: knots)", the parsed output form is: {formula: "0.8×V / 250", variables: ["airspeed V"]}.

[0041] In this embodiment, the equipment status constraint is, for example, "execute only when both channels A and B of the hydraulic system are normal"; The environmental condition constraint is, for example, "under icing conditions, the elevator efficiency threshold is relaxed by 15%". The test item dependency constraint is, for example, "T305 can be started only after test item T201 is completed".

[0042] The time validity constraint is, for example, "Execute the new version threshold (±1.5%) after 2024".

[0043] Decompose the operating time series of the device into a periodic component, a trend component, and a noise component.

[0044] In this embodiment, a filter is used to perform smoothing and denoising processing on the device operating time series, retain the effective signal features, and fill in the missing data points by the linear interpolation method to ensure the continuity of the time series.

[0045] For the periodic component, its decomposition method is as follows: Macroscopic period identification: For the fixed working mode during the device operation (such as the daily start / stop cycle), the seasonal decomposition algorithm (STL) is used to extract the explicit periodic pattern.

[0046] Microscopic period analysis: Detect the hidden high-frequency periodic signals (such as the motor vibration period) through spectrum analysis, and combine the sliding window Fourier transform to locate the start and end time points of the period.

[0047] Periodic component reconstruction: Superimpose the detected multiple periodic signals to generate a synthetic periodic component reflecting the inherent working rhythm of the device.

[0048] For the trend component, its decomposition method is as follows: Use the Hodrick-Prescott filtering algorithm. By dynamically optimizing the smoothing parameter, decompose the time series into a long-term trend term and a short-term fluctuation term. The algorithm iteratively adjusts the smoothness of the trend curve to balance the law of slow decline of device performance (such as mechanical wear) and the influence of sudden fluctuations.

[0049] Introduce the Bayesian optimization strategy to automatically adjust the filtering parameter according to the local fluctuation intensity of the sequence, and avoid the over-smoothing or under-smoothing problems caused by fixed parameters in the traditional method.

[0050] For the noise component, its decomposition method is as follows: Deduct the trend component and the periodic component from the original sequence to obtain the initial residual sequence.

[0051] Separate the steady-state noise related to the inherent characteristics of the device (such as bearing friction noise) through independent component analysis (ICA).

[0052] Use the wavelet threshold denoising technology to identify and strip the external random interference (such as power fluctuations, sudden temperature changes).

[0053] Step S20: Extract features from the standardized multi-modal test data based on the feature extraction network, and obtain a multi-dimensional device state characterization vector through feature splicing; In this embodiment, based on the feature extraction network, feature extraction is performed on the standardized multi-modal test data to obtain image modality features, text modality features, and temporal modality features; The image modality features use a deep convolutional network to extract the spatial distribution pattern of the curve image, including the curvature change between key points and the histogram of oriented gradients; The text modality features extract parameter thresholds and constraint conditions in the standard document through a semantic encoder, including numerical threshold intervals and logical judgment rules; For the temporal modality features, the main frequency features are extracted from the periodic components, the long-term change slope is calculated for the trend components, and the abnormal fluctuation intensity is analyzed for the noise components; The image modality features, text modality features, and temporal modality features are concatenated and input into a deep autoencoder to generate a multi-dimensional device state representation vector.

[0054] Among them, the deep convolutional network in this embodiment is preferably a pre-trained ResNet-50 model, and the semantic encoder is preferably a pre-trained BERT model.

[0055] In this embodiment, the fast Fourier transform (FFT) is performed on the decomposed periodic components to extract the 3 frequency values with the highest energy ratio; Calculate the phase difference between the periodic waveform and the standard reference signal to generate a phase consistency score; Trend component processing: Long-term change slope: Use the least squares method to fit the trend line and calculate the average change rate per thousand hours (such as the hydraulic pressure attenuation rate of 0.03 MPa per thousand hours); Inflection point detection: Identify trend mutation points (such as performance recovery points after maintenance) through the CUSUM algorithm; Noise component processing: Abnormal fluctuation intensity: Calculate the standard deviation to peak ratio of the noise component to quantify the transient interference level; Pulse counting: Count the occurrence frequency of abnormal pulses exceeding the 3σ threshold; Output features: Concatenate the period, trend, and noise features to form a 128-dimensional temporal feature vector; Multi-modal feature fusion and representation generation Feature concatenation: Concatenate the 512-dimensional image modality, 256-dimensional text modality, and 128-dimensional temporal modality feature vectors by dimension to form an 896-dimensional joint feature; Deep autoencoder processing: Encoder structure: Fully connected dimensionality reduction layer: Compress the 896-dimensional input to 512 dimensions (activation function: ReLU); Feature Refinement Layer: Redundant information is removed through a three-layer bottleneck structure (512→256→256→512). Decoder Structure: Feature Expansion Layer: Restore the 256-dimensional encoded vector to 896 dimensions (activation function: Sigmoid). Multi-modal Reconstruction: Split the reconstructed vector into image, text, and time-series components, and calculate the reconstruction error of each modality. Loss Function: Jointly optimize the mean squared error of image pixels, cross-entropy of text intervals, and correlation coefficient of time-series components. Output Representation: Extract a 256-dimensional device status representation vector from the encoder bottleneck layer to represent the comprehensive health status of the device.

[0056] In step S30, construct a structured knowledge base by parsing the standard document text, and extract the initial allowable deviation range of each test item; based on the multi-physical parameters in the device status representation vector, combine the environmental parameters to calculate the environment-physical correlation feature vector, and input it into the health status decision tree to calculate the health index, and dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range. In this embodiment, the method for constructing a structured knowledge base by parsing the standard document text and extracting the initial allowable deviation range of each test item is as follows: In step S31, divide the standard document into paragraphs according to test items (such as "engine speed test", "hydraulic system pressure detection"), and use a natural language processing model to identify the test item name, parameter type (such as temperature, pressure, speed), and constraint conditions (such as unit, threshold range) of each paragraph.

[0057] Extract the numerical threshold intervals (such as "temperature allowable deviation ±2°C", "pressure upper limit ≤8MPa") and logical judgment rules (such as "the average value of three consecutive measurement values needs to meet the conditions") in the document through a semantic encoder (such as BERT or LSTM).

[0058] In step S32, organize the extracted information such as test item names, parameter thresholds, and constraint conditions in the form of entity-relationship-attribute into a knowledge graph. For example: Entities: Test items (such as "engine speed"), parameters (such as "speed value"), constraints (such as "maximum allowable deviation").

[0059] Relationships: Test items contain parameters, and parameters are associated with constraint conditions.

[0060] Use a vector database or graph database to store structured data, and support quick positioning of target test items and corresponding thresholds through natural language queries (such as "query the allowable range of hydraulic system pressure").

[0061] Step S33: Directly extract the explicit numerical intervals from the document (such as "the temperature range is 30°C to 35°C") and convert them into a standardized format (such as [30, 35]). If the document does not explicitly give the numerical values (such as "it is necessary to meet the industry standards"), the missing information is supplemented by associating with an external standard database.

[0062] Convert the constraint conditions into executable rule logics (such as "if the fluctuation of the measured values exceeds 5% for 3 consecutive times, an alarm is triggered") and embed them into the inference engine of the knowledge base.

[0063] Manually review the extracted deviation ranges, correct the errors caused by text ambiguity or model misjudgment, and update the corrected results to the knowledge base.

[0064] In this embodiment, based on the multi-physical parameters in the device state representation vector, the environment-physical correlation feature vector is calculated in combination with the environmental parameters and input into the health state decision tree to calculate the health index. The specific calculation method is as follows: Input the device state representation vector into a pre-constructed and trained physical parameter decoder to generate physical parameters, where the physical parameters include mechanical wear amount, control response error, and sensor degradation degree; Assign initial weights to the physical parameters according to the set numerical values; Collect the real-time environmental parameters of the flight simulator, where the real-time environmental parameters include temperature data, humidity data, and electromagnetic intensity data, and dynamically correct the initial weights based on the environmental parameters; Weight the physical parameters based on the corrected initial weights, input the weighted results into the health state decision tree, output the verification results, and normalize the verification results into a health index. The health index is a continuous value within the range of [0, 1], where 0 represents complete failure and 1 represents the best operating state.

[0065] In this embodiment, a physical parameter decoder (such as a regression model based on deep learning) is pre-trained. The input is the device state representation vector (generated by splicing multi-modal features), and the output is the parameters reflecting the physical state of the device, including: Mechanical wear amount: Infer the wear degree of key components through vibration signals and displacement sensor data.

[0066] Control response error: Compare the difference between the actual control instruction and the device execution result (such as the steering gear deflection angle error).

[0067] Sensor degradation degree: Evaluate its performance decay through the stability of the sensor output (such as the fluctuation amplitude of the temperature sensor).

[0068] Set the initial weights according to the influence degree of each physical parameter on the device health. For example: The weight of mechanical wear is 0.4 (directly affecting the equipment life); The weight of control response error is 0.3 (related to operation accuracy); The weight of sensor degradation degree is 0.3 (affecting data reliability).

[0069] The weight allocation needs to be determined by combining the experience of domain experts or the analysis results of historical fault data.

[0070] The steps of dynamically correcting the initial weight based on real-time environmental parameters specifically include: Divide the scenarios according to the collected real-time environmental parameters (temperature data, humidity data, electromagnetic intensity data), and execute the corresponding weight adjustment rules: High-temperature scenario: When the temperature data exceeds the preset threshold (such as 40°C), increase the weight ratio of mechanical wear by 15%, and at the same time reduce the weight ratio of sensor degradation degree by 15%; High-humidity scenario: When the humidity data is higher than the preset threshold (such as 80%), increase the weight ratio of sensor degradation degree by 10%, and at the same time reduce the weight ratio of control response error by 10%; Strong electromagnetic interference scenario: When the electromagnetic intensity data exceeds the safe range, significantly increase the weight ratio of sensor degradation degree by 20%; (such as initially 0.3, adjusted to 0.5); The adjusted weights of each physical parameter need to satisfy the sum normalization constraint (that is, the sum of all weights is 1), which is specifically achieved through the following method: Scale the adjusted weights proportionally to ensure the sum is 1; If a certain environmental parameter triggers multiple adjustment rules, execute the adjustments in order of priority and finally perform unified normalization.

[0071] Multiply the corrected weights by the physical parameters and sum them to obtain the weighted comprehensive score.

[0072] Input the pre-trained health status decision tree model (such as CART tree or random forest), and output the preliminary verification result of the equipment status (such as "normal", "critical", "abnormal").

[0073] Health index normalization: Map the verification result output by the decision tree to the interval [0, 1]: 0: Complete failure (such as when the decision tree determines severe abnormality and the physical parameters exceed the safety threshold); 1: Optimal operating state (such as when all physical parameters are within the ideal range); The intermediate value is calculated by linear interpolation or non-linear normalization method (such as Sigmoid function).

[0074] For example: If the verification result is "critical", the corresponding health index is 0.5; if it is "normal", the health index is 0.8; if it is "abnormal", the health index is 0.2.

[0075] Device status level classification: Normal status: Health index ≥ 0.7 (The device runs stably without potential risks); Critical status: 0.4 ≤ Health index < 0.7 (Specific parameter fluctuations need to be monitored, but immediate maintenance is not required); Abnormal status: Health index < 0.4 (Immediate repair or shutdown for maintenance is required).

[0076] In this embodiment, another method for calculating the health index is given, specifically: Input the device status characterization vector into a pre - constructed and trained physical parameter decoder to generate physical parameters, where the physical parameters include mechanical wear amount, control response error, and sensor degradation degree; Collect real - time environmental parameters of the flight simulator, where the real - time environmental parameters include temperature data, humidity data, and electromagnetic intensity data; Construct an environmental parameter influence matrix, perform multi - dimensional coupling analysis on the physical parameters and real - time environmental parameters to generate an environment - physical correlation feature vector; Input the environment - physical correlation feature vector into a health status decision tree to output a verification result, and normalize the verification result into a health index. The health index is a continuous value within the range of [0, 1], where 0 represents complete failure and 1 represents the best operating state.

[0077] In this embodiment, the node splitting rule of the health status decision tree is dynamically constructed based on the interaction between environmental parameters and physical parameters; Calculate the state deviation degree by traversing the branch paths of the health status decision tree, and map the state deviation degree to a health index based on an exponential decay function. The health index is a continuous value within the range of [0, 1], where 0 represents complete failure and 1 represents the best operating state. Among them, a high - temperature environment automatically reduces the evaluation weight of the mechanical wear amount, and a high - humidity environment enhances the detection sensitivity of the sensor degradation degree.

[0078] The construction of the environmental parameter influence matrix, performing multi - dimensional coupling analysis on the physical parameters and real - time environmental parameters to generate an environment - physical correlation feature vector specifically includes the following steps: Normalize the physical parameters (mechanical wear amount, control response error, sensor degradation degree) and real - time environmental parameters (temperature, humidity, electromagnetic intensity) respectively, and map them to the range of [0, 1]; Based on a predefined environment - physical coupling rule table, determine the dynamic influence coefficients of each environmental parameter on the physical parameters; In the described coupling rule table, the influence coefficient of the high-temperature environment on the mechanical wear amount is the reciprocal of the temperature value, the influence coefficient of the high-humidity environment on the sensor degradation degree is an exponential function of the humidity value, and the influence coefficient of the electromagnetic intensity on the control response error is generated by piecewise linear interpolation; Construct a three-dimensional environmental parameter influence matrix, with the matrix dimension being (number of environmental parameters × number of physical parameters × coupling depth), where: The row dimension is the environmental parameter category (temperature, humidity, electromagnetic intensity), the column dimension is the physical parameter category (wear amount, response error, degradation degree), and the depth dimension is the coupling level, including basic linear coupling and non-linear interaction coupling; The basic linear coupling layer calculates the element-wise product of the environmental parameters and the physical parameters to generate primary coupling features; Based on the non-linear interaction coupling layer, a non-linear transformation is performed on the product result through an activation function (such as Sigmoid), and the gradient change rate of the environmental parameters is introduced as a dynamic modulation factor; Feature compression is performed on the environmental parameter influence matrix along the coupling depth dimension, and the maximum pooling is used to extract the key response values of each coupling layer, which are expanded into a one-dimensional vector in row-column order; The compressed vector is concatenated with the original value of the physical parameter, and feature fusion is performed through a fully connected layer to generate the final environment-physical correlation feature vector for input to the health status decision tree.

[0079] In this embodiment, the actual detection of the physical parameters is as follows: Mechanical wear amount P 1 = 0.8; Control response error P 2 = 0.6; Sensor degradation degree P 3 = 0.4; The environmental parameters detected in real time are: Temperature E 1 = 0.9; Humidity E 2 = 0.7; Electromagnetic intensity E 3 = 0.5.

[0080] According to the predefined environment-physical coupling rule table: The influence coefficient of temperature on the mechanical wear amount : ; The influence coefficient of electromagnetic intensity on the control response error : If ,then ; If ,then ; If ,then ; In this embodiment, , so .

[0081] Coefficient of influence of humidity on sensor deterioration : ; The matrix dimension is 3×3×2 (environmental parameter × physical parameter × coupling depth): ; Nonlinear interaction coupling layer (including dynamic modulation factor): The activation function in this embodiment is Sigmoid, and the gradient change rate is the time derivative of the environmental parameter: ; Taking as an example: , 0.67×(1 + 0.1) = 0.74; Final nonlinear layer matrix: ; Taking the maximum value along the coupling depth dimension and expanding it into a one-dimensional vector: ; Concatenating the original values of physical parameters: Compressed vector + : ; Fully connected layer fusion (the weight matrix in this embodiment is a randomly initialized 12×6 matrix): Finally generating the environment-physical correlation feature vector: [0.85, 0.72, 0.53, 0.61, 0.49, 0.38]; Inputting the feature vector into the decision tree, according to the node splitting rule (for example, high temperature environment reduces the weight of mechanical wear), finally outputting the health index: Health index = 0.75 (indicating that the device is in a medium health state) Health index = 0.75 (indicating that the device is in a medium health state).

[0082] Calculating the state deviation degree by traversing the branch paths of the health state decision tree, and mapping the state deviation degree to the health index based on the exponential decay function, the method is: Construction of dynamic node splitting rule: The node splitting condition of the health state decision tree is dynamically generated based on the interaction between environmental parameters and physical parameters, where: The node splitting feature is selected from the dimensions in the environment-physical correlation feature vector, and the physical parameter most strongly correlated with the current environmental parameter gradient change rate is selected; The splitting threshold is dynamically adjusted according to the normalized value of the environmental parameter, satisfying the formula: ; Among them, is the environmental sensitivity coefficient, which is obtained by training historical failure data.

[0083] Calculation of the state deviation degree: Traverse from the root node to the leaf node, and record the set of nodes that do not meet the splitting condition in the path; For each unmet node, calculate the absolute deviation between its splitting threshold and the current eigenvalue, and multiply it by the node weight , and the weight is calculated from the real-time change rate of environmental parameters: ; The total state deviation degree D is the sum of the weighted deviations of all unmet nodes, normalized to the interval [0, 1]: ; is the current eigenvalue, is the splitting threshold.

[0084] The state deviation degree is converted into a health index by using an exponential decay function, and the function form is: ; Among them, λ is the preset environmental adaptive decay factor, μ is the preset health floor value, which is preferably 0.2 in this embodiment.

[0085] Dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range, specifically: Based on the magnitude of the health index, divide the device state levels, and the levels include normal state, critical state and abnormal state; When the health index belongs to the normal state, expand the upper and lower limits of the initial allowable deviation interval according to the preset first ratio to form a relaxed final allowable deviation range; When the health index belongs to the critical state, keep the upper and lower limits of the initial allowable deviation interval unchanged as the final allowable deviation range; When the health index belongs to the abnormal state, shrink the upper and lower limits of the initial allowable deviation interval according to the preset second ratio to form a tightened final allowable deviation range; Among them, the health index corresponding to the normal state is higher than that of the critical state, and the health index corresponding to the critical state is higher than that of the abnormal state.

[0086] In this embodiment, the first ratio and the second ratio are preferably 10%-20%.

[0087] Step S40: Input the device status characterization vector into a pre - constructed and trained status classification model to output a compliance probability value, and combine it with the final allowable deviation range to generate a final compliance determination result.

[0088] In this embodiment, the combination with the final allowable deviation range to generate the final compliance determination result specifically includes: Based on the compliance probability value output by the status classification model, extract the comparison results between the real - time operating parameters corresponding to each test item and the final allowable deviation range; If the compliance probability value of a certain test item is higher than the preset threshold and its real - time operating parameters are within the final allowable deviation range, then determine that the test item is in a compliant state; If the compliance probability value is lower than the preset threshold or the real - time operating parameters exceed the final allowable deviation range, then determine it as an abnormal state and mark the specific deviation parameters; After traversing all test items, if there is at least one test item determined to be in an abnormal state, then generate a determination result that the overall device operation is unqualified; otherwise, output a final determination conclusion of full - item compliance; Among them, the preset threshold is a dynamic threshold set according to the constraint conditions in the standard document, and the determination result of the abnormal state synchronously triggers an adaptive iterative update mechanism for the deviation range.

[0089] After this method generates the final compliance determination result, it further includes: Conduct an association analysis on the compliance results of multiple test items to locate the device components that cause systematic deviations; Generate a visualization report, marking the deviation values, dynamic allowable intervals, and historical change trends of the out - of - standard test items.

[0090] Although the above - mentioned embodiments describe each step in the above - mentioned sequential order, those skilled in the art can understand that in order to achieve the effects of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are all within the protection scope of the present invention.

[0091] A flight simulator test data status recognition system based on multi - modal learning according to the second embodiment of the present invention, according to a flight simulator test data status recognition method based on multi - modal learning, the system includes: A data acquisition and processing module configured to acquire multi - modal test data of a flight simulator and perform standardized processing, where the multi - modal test data includes a time - series operation curve image, a standard document text, and a device operation time series; A feature extraction module configured to extract features from the standardized multi - modal test data based on a feature extraction network and obtain a multi - dimensional device status characterization vector through feature splicing; A deviation range construction module configured to construct a structured knowledge base by parsing standard document text, extract the initial allowable deviation ranges of each test item; calculate an environment-physical correlation feature vector based on multi-physical parameters in the device state characterization vector, combined with environmental parameters, and input it into a health status decision tree to calculate a health index, and dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range; A determination module configured to input the device state characterization vector into a pre-constructed and trained state classification model, output a compliance probability value, and generate a final compliance determination result in combination with the final allowable deviation range. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes and related descriptions of the above-described system can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0092] It should be noted that the above-described flight simulator test data state recognition system based on multi-modal learning only uses the above-mentioned division of each functional module as an example. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only used to distinguish each module or step, and are not regarded as an improper limitation of the present invention.

[0093] An electronic device according to a third embodiment of the present invention includes: At least one processor; and A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned method for recognizing the state of flight simulator test data based on multi-modal learning.

[0094] A computer-readable storage medium according to a fourth embodiment of the present invention, the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned method for recognizing the state of flight simulator test data based on multi-modal learning.

[0095] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes and related descriptions of the above-described storage device and processing device can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0096] Those skilled in the art should be able to realize that the modules and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field. To clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0097] The terms "first", "second", etc. are used to distinguish similar objects, rather than to describe or represent a specific order or sequence.

[0098] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, so that a process, method, article, or device / equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent in these processes, methods, articles, or devices / equipment.

[0099] So far, the technical solution of the present invention has been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for identifying the state of flight simulator test data based on multimodal learning, characterized in that The method includes collecting multi-modal test data of a flight simulator and performing standardization processing, where the multi-modal test data includes a time-series operation curve image, a standard document text, and device operation time series; extracting features from the standardized multi-modal test data based on a feature extraction network, and obtaining a multi-dimensional device state representation vector through feature splicing; constructing a structured knowledge base by parsing the standard document text, and extracting the initial allowable deviation range of each test item; based on the multi-physical parameters in the device state representation vector, calculating an environment-physical correlation feature vector in combination with environmental parameters, and inputting it into a health state decision tree to calculate a health index, and dynamically adjusting the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range; inputting the device state representation vector into a pre-constructed and trained state classification model, outputting a compliance probability value, and generating a final compliance determination result in combination with the final allowable deviation range.

2. The method for identifying the state of flight simulator test data based on multimodal learning according to claim 1, wherein Collecting multi-modal test data of a flight simulator and performing standardization processing specifically includes: extracting the key point coordinates of the curve from the time-series operation curve image through a visual processing model, and performing spatial alignment with the time axis; extracting the test item name, parameter threshold, and constraint conditions from the standard document text through a natural language processing model; decomposing the device operation time series into a periodic component, a trend component, and a noise component.

3. The method for identifying the state of flight simulator test data based on multi-modal learning according to claim 2, wherein Extracting features from the standardized multi-modal test data based on a feature extraction network to obtain an image modality feature, a text modality feature, and a time series modality feature; The image modality feature uses a deep convolutional network to extract the spatial distribution pattern of the curve image, including the curvature change between key points and the histogram of gradient directions; The text modality feature extracts the parameter threshold and constraint conditions in the standard document through a semantic encoder, including a numerical threshold interval and a logical judgment rule; For the time series modality feature, it includes extracting the main frequency feature of the periodic component, calculating the long-term change slope of the trend component, and analyzing the abnormal fluctuation intensity of the noise component; Splicing the image modality feature, the text modality feature, and the time series modality feature, and inputting them into a deep autoencoder to generate a multi-dimensional device state representation vector.

4. A method for identifying the state of flight simulator test data based on multimodal learning according to claim 1, characterized in that, Based on the multi-physical parameters in the device state representation vector, calculating an environment-physical correlation feature vector in combination with environmental parameters, and inputting it into a health state decision tree to calculate a health index. The specific calculation method is: inputting the device state representation vector into a pre-constructed and trained physical parameter decoder to generate physical parameters, where the physical parameters include mechanical wear amount, control response error, and sensor deterioration degree; collecting the real-time environmental parameters of the flight simulator, where the real-time environmental parameters include temperature data, humidity data, and electromagnetic intensity data; constructing an environmental parameter influence matrix, performing multi-dimensional coupling analysis on the physical parameters and the real-time environmental parameters, and generating an environment-physical correlation feature vector; inputting the environment-physical correlation feature vector into a health state decision tree, outputting a verification result, and normalizing the verification result into a health index, where the health index is a continuous value in the interval [0, 1], 0 represents complete failure, and 1 represents the best operating state.

5. A method for identifying the state of flight simulator test data based on multimodal learning according to claim 1, characterized in that, Dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range, specifically: Divide the device state level based on the magnitude of the health index, and the levels include normal state, critical state, and abnormal state; When the health index belongs to the normal state, expand the upper and lower limits of the initial allowable deviation interval according to a preset first ratio to form a relaxed final allowable deviation range; When the health index belongs to the critical state, keep the upper and lower limits of the initial allowable deviation interval unchanged as the final allowable deviation range; When the health index belongs to the abnormal state, reduce the upper and lower limits of the initial allowable deviation interval according to a preset second ratio to form a tightened final allowable deviation range; Among them, the health index corresponding to the normal state is higher than that of the critical state, and the health index corresponding to the critical state is higher than that of the abnormal state.

6. The method for identifying the state of flight simulator test data based on multimodal learning according to claim 1, wherein Combined with the final allowable deviation range, generate the final compliance determination result, specifically including: Based on the compliance probability value output by the state classification model, extract the comparison result between the real-time operation parameters corresponding to each test item and the final allowable deviation range; If the compliance probability value of a certain test item is higher than the preset threshold and its real-time operation parameters are within the final allowable deviation range, then determine that the test item is in a compliant state; If the compliance probability value is lower than the preset threshold or the real-time operation parameters exceed the final allowable deviation range, then determine it as an abnormal state and mark the specific deviation parameters; After traversing all test items, if there is at least one test item determined to be in an abnormal state, generate a determination result that the overall device operation is unqualified; otherwise, output the final determination conclusion of full-item compliance; Among them, the preset threshold is a dynamic threshold set according to the constraint conditions in the standard document, and the determination result of the abnormal state synchronously triggers the adaptive iterative update mechanism of the deviation range.

7. A method for identifying the state of flight simulator test data based on multimodal learning according to claim 1, characterized in that After this method generates the final compliance determination result, it also includes: Conduct correlation analysis on the compliance results of multiple test items to locate the device components that cause systematic deviations; Generate a visual report, marking the deviation values, dynamic allowable intervals, and historical change trends of the out-of-standard test items.

8. A flight simulator test data status recognition system based on multimodal learning, according to the method for recognizing the status of flight simulator test data based on multimodal learning according to any one of claims 1-7, characterized in that, The system includes: A data collection and processing module configured to collect multi-modal test data of the flight simulator and perform standardization processing. The multi-modal test data includes time-series operation curve images, standard document texts, and device operation time series; A feature extraction module configured to perform feature extraction on the standardized multi-modal test data based on a feature extraction network and obtain a multi-dimensional device state representation vector through feature splicing; A deviation range construction module configured to construct a structured knowledge base by parsing the standard document text, extract the initial allowable deviation range of each test item; calculate the environment-physical correlation feature vector based on the multi-physical parameters in the device state representation vector and combine with environmental parameters, and input it into the health state decision tree to calculate the health index, and dynamically adjust the upper and lower limits of the initial allowable deviation interval according to the health index to obtain the final allowable deviation range; A determination module configured to input the device state representation vector into a pre-constructed and trained state classification model, output a compliance probability value, and generate a final compliance determination result in combination with the final allowable deviation range.

9. An electronic device, characterized in that, Include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement a method for identifying the state of flight simulator test data based on multimodal learning according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement a method for identifying the state of flight simulator test data based on multimodal learning according to any one of claims 1-7.

Images