Automatic data analysis method, system and equipment for aviation accident investigation

Abstract

The invention belongs to the field of data analysis, particularly relates to an automatic data analysis method, system and equipment for aviation accident investigation, and aims to solve the problems of poor data processing adaptability, insufficient preprocessing, performance bottleneck and modeling diagnosis limitation in the prior art. The method comprises the steps of collecting original flight data and converting the original flight data into a structured format; identifying analog quantity, digital quantity and discrete mixing quantity parameters and executing differential analysis; after standardization processing, constructing a tree structure through recursive feature segmentation, calculating an abnormal score according to a path length, and generating an abnormal parameter set; feature vectors are extracted, potential space representation is generated through a variational auto-encoder, and a parameter correlation network is constructed through mutual information calculation; and generating an analysis report by combining the classification result of the hierarchical convolutional neural network, the parameter network and the abnormal parameter set. Full-process automatic accident analysis is realized, and investigation efficiency and precision are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of data analysis, and particularly relates to a data automatic analysis method, system and equipment for aviation accident investigation. BACKGROUND

[0002] In the field of aviation accident investigation, the automatic analysis technology of flight data has developed to a certain extent, but still faces significant limitations. The existing system has serious deficiencies in processing unknown aircraft types and non-standard format data. For example, in the invention patent with publication number CN105391524A, data archiving relies on predefined aircraft identification codes. When encountering heterogeneous data formats of new or unregistered aircraft types (such as custom HDF5 structures or private binary protocols), the system fails to process due to the lack of dynamic parsing mechanism, greatly limiting the adaptability of the system.

[0003] At the same time, the automation degree and accuracy of the data preprocessing link have obvious defects. In the article titled "Design and Implementation of QAR Data Processing System Based on Teechart and VB Database" published in the journal "Aviation Maintenance and Engineering" by Zhou Baizheng et al., it is pointed out that current tools still rely on manual intervention for common problems such as missing data and redundant header rows. The processing capability for complex scenarios such as parameter jumps and cross-day timestamp abnormalities (e.g., negative timestamp errors in fuel flow data due to cross-day flights) is weak. The dimensional differences of parameters of different aircraft types (such as the mixing of English and metric units) further lead to analysis deviation. These preprocessing defects not only increase the time cost, but also introduce the risk of human error.

[0004] In terms of system performance, the centralized processing architecture and inefficient algorithm design form a double bottleneck. The central server mode adopted in the invention patent with publication number CN105391524A causes data transmission delay, and the full-batch processing model in "Design and Implementation of QAR Data Processing System Based on Teechart and VB Database" makes the analysis time linearly grow with the data volume. Combined with the insufficient classification accuracy of traditional machine learning models for heterogeneous labels, the real-time response capability is severely restricted.

[0005] The deeper problem lies in the limitations of parameter correlation modeling and fault diagnosis. Existing methods generally ignore the dynamics correlation between parameters. For example, rule-based cleaning (such as fixed threshold rejection) may misjudge the coordinated fluctuations of engine speed and oil pressure as abnormal. Fault diagnosis excessively relies on single-dimensional threshold judgment and cannot identify complex fault patterns coupled with multiple parameters, such as concurrent vibration value increase and abnormal exhaust temperature. Static models do not consider the parameter characteristic differences in different flight stages, such as the threshold conflict between takeoff high thrust and cruise steady state, leading to an increase in complex fault detection rate.

[0006] Based on this, the application provides a data automatic analysis method, system and equipment for aviation accident investigation. SUMMARY

[0007] In order to solve the above problems in the prior art, that is, the problems of poor data processing adaptability, insufficient preprocessing, performance bottleneck and modeling diagnosis limitation in the prior art, the application provides a data automatic analysis method, system and equipment for aviation accident investigation.

[0008] In a first aspect, the application provides a data automatic analysis method for aviation accident investigation, which comprises: Collecting original flight data from an aviation data source, and performing data screening based on predefined flight stages and key parameters, and converting the screened data into editable structured format data; Identifying parameter types from the structured format data, and performing differential analysis, wherein the parameter types include analog parameters, digital parameters and discrete mixed parameters; Performing standardization processing on the analyzed parameters, constructing an isolated tree through recursive feature segmentation, calculating an abnormal score based on the average path length of data points in the isolated tree, and marking parameters with abnormal scores exceeding a threshold value to form an abnormal parameter set; Inputting the standardized flight data into an automatic encoder to extract a primary feature vector, and inputting the compressed dimension into a variational autoencoder to generate a latent space representation of the non-linear relationship between encoded parameters, and quantifying the non-linear correlation between parameters by calculating the mutual information between different dimensions to construct a parameter correlation network; Inputting the compressed features into a pre-constructed hierarchical convolutional neural network, identifying flight stages and key parameter categories through multi-level classifiers layer by layer, and eliminating redundant parameters based on probability distribution similarity, combining the classification results of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set to generate an analysis report containing abnormal parameter identification, time points and deviation values.

[0009] Further, the original flight data is collected from an aviation data source, which comprises: Collecting original flight data stored in a hierarchical binary format from an aviation data source.

[0010] Further, the differential analysis is performed, and the method comprises: For analog parameters, determining the numerical precision range according to the original bit width, and decoding the original data into actual physical values according to the physical quantity conversion rule; For digital parameters, mapping binary data to state identifiers according to the original bit width and predefined encoding rules; For discrete mixed parameters, separating the discrete part and the analog part, and processing them according to the digital parameter analysis rule and the analog parameter analysis rule, respectively.

[0011] Further, the physical quantity conversion rule of the analog quantity parameter includes: Converting the integer type data into actual physical values according to a preset scaling factor; When the numerical value exceeds the pre-defined range, it is marked as invalid data.

[0012] Further, the non-linear correlation between parameters is quantified by calculating the mutual information between different dimensions, and a parameter correlation network is constructed, and the method is: Based on the latent space representation, the mutual information value between different dimensions is calculated to generate a dimension correlation strength matrix; Mapping the dimensions in the dimension correlation strength matrix to the original flight parameters to obtain the non-linear correlation strength between parameters; Taking flight parameters as nodes and non-linear correlation strength as edge weight, a graph structure network is constructed as a parameter correlation network.

[0013] Further, the dimension correlation strength matrix is calculated by: Calculate the mutual information value of all pairs of dimensions in the latent space representation to form a symmetric correlation matrix; Marking the pairs of dimensions with element values greater than a preset threshold in the symmetric correlation matrix as a strongly correlated dimension group; Based on the strongly correlated dimension group, a dimension correlation strength matrix is generated, and the matrix element value reflects the non-linear correlation strength between dimensions.

[0014] Further, the pre-constructed hierarchical convolutional neural network includes: A feature input layer for receiving the compressed feature vector; A convolutional feature extraction layer for extracting the spatial correlation pattern of the feature vector through convolution kernel operation and compressing the feature dimension through pooling operation; A first level classification layer, including: A flight phase local classification unit for generating a flight phase classification result based on the output of the convolutional feature extraction layer; A flight phase global fusion unit for integrating the classification result and the initial global information flow to output a first global information flow; A second level classification layer, including: A parameter category local classification unit for generating a parameter category classification result based on the output of the convolutional feature extraction layer and the first global information flow; A parameter category global fusion unit for integrating the classification result and the first global information flow to output a final global information flow; A redundancy elimination layer for merging or deleting redundant parameters according to the probability distribution similarity of parameter categories in the same flight phase; a classification output layer for converting the final global information flow into a multi-class probability distribution.

[0015] Further, in combination with the classification result of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set, an analysis report containing the abnormal parameter identifier, time point and deviation value is generated, and the method is: Resolving the identifier of the abnormal parameter and its corresponding time point from the abnormal parameter set; For the measured value of each abnormal parameter at the corresponding time point, the normal range value in the standardized database is retrieved, and the absolute deviation of the measured value and the normal range value is calculated; In the parameter correlation network, screening the associated parameters whose mutual information with the abnormal parameter exceeds a preset threshold; According to the classification result of the hierarchical convolutional neural network, the flight phase category and parameter category to which the abnormal parameter belongs are labeled; Integrating the structured report containing the following elements: Abnormal parameter identifier and its corresponding time point; deviation value of abnormal parameter measured value and normal range value; associated parameter list strongly associated with abnormal parameter; flight phase category and parameter category to which the abnormal parameter belongs.

[0016] Another aspect of the present application provides a data automatic analysis system for aviation accident investigation, which is used to realize a data automatic analysis method for aviation accident investigation, and the system comprises: A data screening module configured to collect original flight data from an aviation data source and perform data screening based on predefined flight phases and key parameters, and convert the screened data into editable structured format data; An analysis module configured to identify parameter types from the structured format data and perform differential analysis, wherein the parameter types include analog parameters, digital parameters and discrete mixed parameters; An abnormal parameter set acquisition module configured to perform standardization processing on the analyzed parameters, construct an isolation tree through recursive feature segmentation, calculate an abnormal score based on the average path length of data points in the isolation tree, and mark parameters with abnormal scores exceeding a threshold to form an abnormal parameter set; A parameter correlation network construction module configured to input the standardized flight data into an autoencoder to extract a primary feature vector, and input the compressed dimension into a variational autoencoder to generate a latent space representation of the nonlinear relationship between the encoded parameters, and quantify the nonlinear correlation between the parameters by calculating the mutual information between different dimensions to construct a parameter correlation network; The report generation module is configured to input the compressed features into a pre-constructed hierarchical convolutional neural network, identify flight stages and key parameter categories layer by layer through a multi-level classifier, and eliminate redundant parameters based on probability distribution similarity, and generate an analysis report containing abnormal parameter identification, time points and deviation values in combination with the classification results of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set.

[0017] In a third aspect, the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to implement the above-mentioned data automatic analysis method for aviation accident investigation.

[0018] The present application has the following advantages: The present application divides flight parameters into three categories: analog parameters, digital parameters and discrete mixed parameters, and establishes a differential analysis framework based on parameter types. The system uses a combination of pre-defined analysis rules and dictionary list structure to achieve adaptive identification of data from various aircraft models, and completely changes the limitations of traditional solutions that rely on fixed aircraft identification codes. In the data preprocessing stage, the recursive feature segmentation isolation tree algorithm is introduced to automatically identify parameter abnormal fluctuations and timestamp errors based on the average path length of data points in the isolation tree. This intelligent processing mechanism does not require manual presetting of fixed thresholds and can adapt to the characteristics of different flight scenarios, significantly reducing the need for human intervention.

[0019] The present application first extracts a primary feature vector using an autoencoder, and then maps it to a latent space representation using a variational autoencoder. This process not only achieves intelligent compression of data dimensions, but more importantly, it constructs a correlation network that reveals the complex nonlinear relationships between parameters by quantifying the mutual information between different dimensions. Based on this, the system implements automatic identification of flight stage features and key parameter classification using a multi-level classification mechanism of a hierarchical convolutional neural network, while automatically eliminating redundant parameter items based on probability distribution similarity.

[0020] The nonlinear correlation network generated by the variational autoencoder, the abnormal parameter set marked by the isolation tree algorithm, and the flight stage features identified by the hierarchical convolutional network. Through deep coupling analysis of these three-dimensional data, the system can accurately capture complex failure modes, such as coordinated failure of engine vibration abnormalities and oil pressure fluctuations. The entire analysis process realizes an end-to-end automated closed loop, starting from raw data acquisition, going through intelligent analysis and cleaning, to correlation modeling and multi-dimensional feature detection, and finally generating an analysis report containing abnormal parameter time points and deviation values.

[0021] The application has the ability of knowledge self-evolution, and continuously absorbs new fault modes and model data features by constructing an extensible fault diagnosis knowledge base and setting an open rule engine interface. The dynamic updating mechanism enables the system to autonomously optimize parameter identification rules and fault diagnosis logic without reconstructing the underlying architecture when facing different model protocol changes or newly added flight states, thereby realizing an important transformation from passive response to active adaptation. BRIEF DESCRIPTION OF DRAWINGS

[0022] Other features, objects, and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, when read in conjunction with the accompanying drawings: Figure 1 is a flowchart of a data automatic analysis method for aviation accident investigation of the application. DETAILED DESCRIPTION

[0023] The application will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0024] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict. The application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0025] In a first embodiment of the application, a data automatic analysis method for aviation accident investigation is provided, which comprises: Step S10, collecting original flight data from an aviation data source, and performing data screening based on predefined flight stages and key parameters, and converting the screened data into editable structured format data; Step S20, identifying parameter types from the structured format data, and performing differential analysis, wherein the parameter types include analog parameters, digital parameters, and discrete mixed parameters; Step S30, performing standardization processing on the analyzed parameters, constructing an isolation tree through recursive feature segmentation, calculating an abnormal score based on the average path length of data points in the isolation tree, and marking parameters with abnormal scores exceeding a threshold to form an abnormal parameter set; Step S40, inputting the standardized flight data into an automatic encoder to extract a primary feature vector, and inputting the compressed dimension into a variational autoencoder to generate a latent space representation of the non-linear relationship between the encoded parameters, and quantifying the non-linear correlation between the parameters by calculating the mutual information between different dimensions to construct a parameter correlation network; Step S50, input the compressed features into the pre-constructed hierarchical convolutional neural network, identify the flight phase and key parameter category layer by layer through the multi-level classifier, and eliminate redundant parameters based on the probability distribution similarity. Combine the classification results of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set to generate an analysis report containing abnormal parameter identification, time point and deviation value.

[0026] In order to more clearly describe the data automatic analysis method for aviation accident investigation, the following will be described in combination with Figure 1 The steps in the embodiments of the present application will be described in detail, including steps S10-S50, and the detailed description of each step is as follows: Step S10, collecting original flight data from aviation data sources, and performing data screening based on predefined flight phases and key parameters, and converting the screened data into editable structured format data; Specifically, the original flight data is collected from multiple data sources such as airlines, flight data recorders and flight management systems, and the original data is stored in hierarchical binary format, which specifically includes flight parameters, timestamp markers and flight phase metadata and other key information. Based on the predefined flight phase division standard, including but not limited to the take-off phase, the cruise phase and the landing phase, and combined with key flight parameter indicators such as flight altitude, airspeed and engine operating parameters, the original data is preliminarily screened. The screened data is automatically converted into readable and editable structured format according to the data structure characteristics. This conversion process ensures that the data maintains its original hierarchical relationship and parameter integrity.

[0027] In the data conversion process, the characteristics of different types of data are analyzed in depth, including structured data, semi-structured data and unstructured data and other forms. The hierarchical binary format file is checked for standardization to detect whether its format meets the aviation data specification requirements, and data that does not meet the standard is repaired or removed. At the same time, a flexible data integration framework is designed and implemented to realize unified management and efficient integration of heterogeneous data, ensuring the editability and structural consistency of the converted data.

[0028] In view of the quality problems in the data integration process, the system performs a comprehensive data cleaning process, including missing value filling, duplicate value deletion and abnormal value correction. For missing data points, linear interpolation or mean value calculation based on adjacent data points is used for filling; for abnormal data values, based on statistical distribution characteristics (such as Z-score standardization processing or interquartile range IQR analysis) combined with aviation physical meaning for correction. Finally, all parameters are standardized to eliminate the dimensional differences between different model data, unify the value range and scale, and establish a standardized data basis for subsequent analysis.

[0029] Step S20, identifying parameter types from the structured format data, including analog parameters, digital parameters and discrete mixed parameters, and performing differential analysis; In this embodiment, differential analysis is performed, and the method is as follows: For analog parameters, the numerical precision range is determined according to the original bit width, and the original data is decoded into actual physical values according to the physical quantity conversion rule; For digital parameters, binary data is mapped to state identifiers according to the original bit width and the pre-defined encoding rule. For discrete mixed parameters, the discrete part and the analog part are separated and processed according to the digital parameter analysis rule and the analog parameter analysis rule respectively.

[0030] The physical quantity conversion rule for analog parameters includes: Integer type data is converted to actual physical values according to a pre-set scaling factor; When the value exceeds the pre-defined range, it is marked as invalid data.

[0031] In this embodiment, three types of flight parameters are identified from the structured format data, including analog parameters, digital parameters and discrete mixed parameters, and differential analysis is performed. For analog parameters, the numerical precision range is first determined according to the original bit width, for example, when the bit width is 16 bits, the numerical range is 0-65535; Then, according to the physical quantity conversion rule, the original data is decoded into actual physical values, which includes: for integer type data, multiply by a pre-set scaling factor, such as 16-bit temperature data multiplied by 0.01 to convert to actual Celsius value, for floating point type data, perform dimension conversion according to the decimal point position; when the value exceeds the pre-defined range (such as height parameter exceeding 0-50000 feet), it is marked as invalid data.

[0032] For digital parameters, the number of states is determined according to the original bit width, such as 2-bit parameters can represent 4 states, combined with the pre-defined encoding rule, including binary encoding, Gray code or other special encoding method, binary data is mapped to physical state identifier, such as binary value "00" is mapped to engine "off" state, "01" is mapped to "starting" state, 10 indicates "running", 11 indicates "fault"; at the same time, multi-substate combination analysis is supported, such as the first 4 bits of 8-bit parameter are analyzed as engine 1 state, and the last 4 bits are analyzed as engine 2 state, and parity check or redundancy coding mechanism is implemented to detect transmission errors.

[0033] Parity check: if parity check bits are used for digital parameters, it is necessary to define how to detect and correct errors. For example, by checking the parity check bits to determine whether the data has occurred error in the transmission process.

[0034] Redundancy coding: If redundancy coding (e.g., repetition coding) is used, define how to correct errors through redundant information.

[0035] For discrete mixed quantity parameters, separate the discrete state and analog data according to the predefined data structure (e.g., the first 8 bits of a 24-bit parameter are discrete, and the last 16 bits are analog), map the discrete state according to the digital quantity parameter rule, and convert the analog part to physical quantity according to the analog quantity parameter rule; When there is a conflict between the two parts of the data (e.g., the discrete part indicates a sensor failure while the analog part has a normal value), prioritize the state identification of the discrete part. All parsing rules are dynamically managed through configuration files, supporting adaptation to different frame structure differences of different models, and after parsing, the results are verified by comparing standard data or simulation tests to ensure that the parsing efficiency and accuracy meet the real-time processing requirements.

[0036] Through the above parsing methods for different data types of parameters, each parameter is accurately converted and decoded. For the decoded data, use functions in the pandas library (such as dropna(), fillna(), drop_duplicates(), etc.) to handle missing values, duplicate values, and outliers. Format the data, such as uniform date format, numerical format, etc., to ensure data consistency. Use libraries such as dask or modin for parallel processing to improve data processing efficiency. At the same time, write the above steps into an automated script to repeat the data merging task. Use Python's argparse or click library to handle command line arguments to make the script more flexible and easy to use.

[0037] Step S30, performing standardization processing on the parsed parameters, constructing an isolated tree through recursive feature segmentation, calculating an anomaly score based on the average path length of data points in the isolated tree, and marking parameters with anomaly scores exceeding a threshold to form an abnormal parameter set; The step S30 includes the following sequentially executed sub-steps: Step S31, performing standardized conversion on the parsed flight parameters to eliminate dimensional differences between different parameters and generate a standardized data set with uniform dimensions; Step S32, based on the standardized data set, constructing a tree-like detection structure by recursively segmenting the feature space, and randomly selecting feature dimensions and their segmentation thresholds during the segmentation process; Step S33, calculating an anomaly score based on the path length of data points in the tree-like structure, where the path length reflects the difficulty of isolating data points; Step S34, marking parameters and their corresponding time points with anomaly scores exceeding a preset threshold to form a structured abnormal parameter set.

[0038] Specifically, multi-mode standardization is performed on the parsed flight parameters to eliminate dimensional differences: Z-score standardization is adopted for parameters with symmetric data distribution, converting parameter values into a distribution form with a mean of 0 and a standard deviation of 1; for parameters with asymmetric distribution, interquartile range (IQR) standardization is adopted, and data is scaled by the first quartile (Q1) and the third quartile (Q3), and the specific conversion formula is X norm = (X-Q1) / (Q3-Q1). After standardization, an isolation forest anomaly detection model is constructed: random selection of parameter features and their segmentation thresholds, recursively dividing the data space into subspaces until all data points are isolated or the preset upper limit of tree depth (typical value 8 layers) is reached, and multiple independent trees are repeatedly constructed to form an isolated tree set (typical number of trees 100).

[0039] According to the path length of the data points in the isolated trees, the anomaly score is calculated: first, the average path length of each data point in all isolated trees is calculated , where T is the total number of isolated trees, h i ( x ) is the path length in the i-th tree; then the anomaly score is converted by the standardization function , where c ( n ) is the expected value of the path length (obtained by simulating the average path length of a binary search tree of n samples). Set the anomaly score threshold to 0.7, mark all parameters and their corresponding time stamps whose anomaly scores exceed the threshold, and generate a structured abnormal parameter set, which includes the following core fields: parameter name, abnormal time point, original value before standardization, normal value range reference value, and anomaly score value.

[0040] During the anomaly detection process, the feature segmentation path is automatically recorded, including: the selected feature dimension, segmentation threshold and subspace division result at each recursive segmentation; a three-dimensional tree structure diagram is generated by a visualization tool, highlighting the isolated position of the abnormal parameter in the tree structure (such as the short path branch near the root node); at the same time, a feature importance ranking list is output (arranged in descending order according to the frequency of the feature being selected in the segmentation process). Finally, the abnormal parameter set, feature segmentation path log and tree structure visualization diagram are packaged and stored for the accident investigation module to call for review.

[0041] Step S40, input the standardized flight data into the autoencoder to extract the primary feature vector, and input it into the variational autoencoder after dimension compression to generate a latent space representation of the non-linear relationship between the encoded parameters, and quantify the non-linear correlation between the parameters by calculating the mutual information between different dimensions to construct a parameter correlation network; In this embodiment, the nonlinear correlation between the mutual information of different dimensions is calculated to construct a parameter correlation network, and the method is as follows: Based on the latent space representation, the mutual information value between different dimensions is calculated to generate a dimension correlation strength matrix; Mapping the dimensions in the dimension correlation strength matrix to the original flight parameters to obtain the nonlinear correlation strength between parameters; Taking flight parameters as nodes and nonlinear correlation strength as edge weight, a graph structure network is constructed as a parameter correlation network.

[0042] The calculation method of the dimension correlation strength matrix is as follows: Calculate the mutual information value of all pairs of dimensions in the latent space representation to form a symmetric correlation matrix; Mark the dimension pairs with element values greater than the preset threshold in the symmetric correlation matrix as a strong correlation dimension group; Based on the strong correlation dimension group, a dimension correlation strength matrix is generated, and the matrix element value reflects the nonlinear correlation strength between dimensions.

[0043] In this embodiment, based on the latent space representation generated by the variational autoencoder (VAE), a VAE model is first constructed: The encoder uses a fully connected neural network to map the input feature vector E to the latent space mean μ and variance σ 2 , satisfying μ=f θ (E) and log(σ 2 )=g θ (E) ; wherein, f θ and g θ are two parallel neural network branches inside the encoder; Through the reparameterization technique z = μ + σ × to generate a derivable latent variable, where is sampled from a standard normal distribution; the decoder reconstructs the feature vector through the decoder The model training objective function L is the sum of reconstruction error and KL divergence: ; Where, n is the feature dimension, d is the latent space dimension, μ j is the mean of the jth dimension in the latent space,σ j is the standard deviation of the j-th dimension in the latent space, E i is the i-th feature value of the original input data.

[0044] Based on the trained convergent VAE model, the latent space representation z Perform mutual information calculation: use kernel density estimation method to calculate the marginal probability distribution of each dimension p z i and the joint probability distribution of the dimensions p z i z j , through the entropy difference formula: ; Quantify the nonlinear dependence of all dimension pairs to generate a symmetric dimension correlation strength matrix. Set a preset threshold to screen strong correlation dimension groups (such as mutual information value > 0.5) to form a simplified correlation matrix.

[0045] Through the inverse analysis of the decoder weight matrix, the mapping relationship between the latent space dimensions and the original flight parameters is established: locate the maximum response connection in the weight matrix to construct a dimension-parameter mapping dictionary. Convert the dimension correlation strength value to the nonlinear correlation strength between parameters.

[0046] Take flight parameters as nodes (node attributes include parameter name and physical unit), and normalized correlation strength as edge weight to construct a weighted undirected graph network. Perform edge weight pruning: remove weakly correlated edges below the connection threshold (such as 0.3) to retain significant correlation relationships. Through force-directed algorithm layout visualization, strong correlation parameters form spatial clusters (such as engine parameter cluster: speed-oil flow-turbine temperature), and community detection algorithm is applied to identify topological hub parameters. Finally, the sparse network is stored as a graph database, supporting real-time topology query and accident diagnosis module calling, completing the closed-loop conversion from latent space analysis to engineering application.

[0047] Step S50, input the compressed features into the pre-constructed hierarchical convolutional neural network, identify the flight phase and key parameter category layer by layer through the multi-level classifier, and eliminate redundant parameters based on the probability distribution similarity. Combine the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the abnormal parameter set to generate an analysis report containing abnormal parameter identification, time point, and deviation value.

[0048] The pre-constructed hierarchical convolutional neural network includes: a feature input layer for receiving the compressed feature vector; ​​​a convolutional feature extraction layer configured to extract spatial correlation patterns of the feature vector by convolution kernel operation and compress feature dimension by pooling operation; a first hierarchical classification layer comprising: a flight phase local classification unit configured to generate a flight phase classification result based on the output of the convolutional feature extraction layer; a flight phase global fusion unit configured to integrate the classification result and an initial global information flow to output a first global information flow; a second hierarchical classification layer comprising: a parameter category local classification unit configured to generate a parameter category classification result based on the output of the convolutional feature extraction layer and the first global information flow; a parameter category global fusion unit configured to integrate the classification result and the first global information flow to output a final global information flow; a redundancy elimination layer configured to merge or delete redundant parameters according to the probability distribution similarity of parameter categories in the same flight phase; a classification output layer configured to convert the final global information flow into a multi-category probability distribution.

[0049] The pre-constructed hierarchical convolutional neural network comprises a feature input layer configured to receive the compressed and reduced dimension feature vector and convert the flight parameter sequence into multi-channel image data, wherein each channel corresponds to the time series data of a single flight parameter. The convolutional feature extraction layer performs sliding convolution operation on the input data through trainable convolution kernel, captures the local spatial correlation pattern between parameters, and adopts maximum pooling operation to down-sample the convolution output, compressing the feature dimension while preserving the key feature response.

[0050] The first hierarchical classification layer comprises a flight phase local classification unit and a global fusion unit: the local classification unit analyzes the feature map output by the convolutional layer through a fully connected neural network to generate an initial flight phase classification result (including take-off, climb, cruise, descent, and landing); the global fusion unit receives the classification result and an initialized zero-value global information flow, integrates information through a gating mechanism, and outputs a first global information flow carrying flight phase features.

[0051] The second hierarchical classification layer comprises a parameter category local classification unit and a global fusion unit: the local classification unit generates key parameter category classification results such as engine parameters, altitude parameters, and speed parameters based on the convolutional layer feature map and the first global information flow; the global fusion unit fuses the classification result and the first global information flow through attention weight to generate a final global information flow carrying parameter category features.

[0052] The redundancy elimination layer calculates the similarity of the category probability distribution of the parameters in the same flight phase based on the final global information flow, performs clustering analysis on the parameter groups (such as vertical speed, vertical acceleration and radio altitude) whose similarity exceeds the dynamic threshold, and only retains the representative parameters of each category and eliminates the redundant parameters. The classification output layer inputs the final global information flow into the Softmax function to generate a multi-category probability distribution and output the simplified key parameter category label, while recording the classification confidence of each parameter and the redundancy elimination decision path.

[0053] The entire processing process optimizes the network weights through end-to-end training, wherein the flight phase classification loss function and the parameter category classification loss function adopt cross-entropy loss, and the redundancy elimination operation realizes gradient backpropagation through a differentiable clustering algorithm. The network output result and the parameter correlation network and the abnormal parameter set are cooperatively input into the accident diagnosis module to complete the full-process analysis closed loop.

[0054] More specifically, the feature input layer receives the compressed and reduced feature vectors, and converts the flight parameter sequence into multi-channel image data: Data reconstruction: the three-dimensional data of n flight cycles (different aircraft flights), m flight segments, p flight parameters are organized into n × m × p image format Channel definition: each channel corresponds to the time series data of a single flight parameter (such as channel 1: altitude parameter, channel 2: airspeed parameter); The convolution feature extraction layer performs the following operations: Convolution operation: using k trainable convolution kernels with a size of f × f for sliding convolution, the output feature map size is ( n-f +1)×( m-f +1)× k； Feature compression: down-sampling using max-pooling operation (pooling window s × s ), the output size is ; First-level classification layer (flight phase recognition): Local classification unit: fully connected neural network analyzes the convolution feature map, and outputs the flight phase probability distribution : ; wherein E is the CNN feature vector, φIt is a ReLU activation function, and the output dimensions correspond to the five categories of takeoff / climb / cruise / descent / landing. and are the weight vector and bias vector of the first layer of the local classification unit respectively; Global fusion unit: integrates classification results and initial zero-value global information flow : ; in, Represents vector concatenation, generating the first global information flow carrying flight phase characteristics, and are the weight matrix and bias vector of the first layer of the global fusion unit respectively; The second level classification layer (parameter category identification): Local classification unit: based on convolutional feature map and first global information flow : ; in, and are the weight vector and bias vector of the second layer of the local classification unit respectively; Output probability distribution of engine parameters / altitude parameters / speed parameters, etc. Global fusion unit: information fusion through attention weights : ; Generate the final global information flow that carries the parameter category characteristics; Redundant parameter processing The redundancy culling layer does the following: Extract the category probability distribution of r different flight parameters in the same flight phase ; Calculate Jensen-Shannon divergence similarity between parameters : ; in, is the probability distribution of the i-th and j-th in the set P; M is the probability distribution The arithmetic mean distribution of ; is the Kullback-Leibler divergence, also known as relative entropy.

[0055] Perform spectral clustering on groups of parameters (e.g. vertical velocity / vertical acceleration / radio altitude) whose similarity exceeds a dynamic threshold (e.g. 0.85); Retain representative parameters of each group (such as vertical speed) and eliminate redundant parameters; Classification output and training Classification output layer processing: Final global information flow Input Softmax function: ; where, and are the values of the jth and kth elements in , respectively; Output the reduced parameter class labels and confidences; Record the redundancy elimination decision paths (e.g., "radio altitude is represented by vertical speed"); End-to-end training: Loss function: Flight phase classification: cross-entropy loss ; Parameter category classification: cross-entropy loss ; Differentiable clustering: redundancy elimination layer gradient backpropagation through Gumbel-Softmax approximation; Joint optimization: total loss .

[0056] where, is the true label of the flight phase classification, is the true label of each flight parameter category classification; is the predicted probability distribution of the model for the flight parameter categories; λ is the weight hyperparameter of the L2 regularization term; θ is the set of all parameters (weights and biases) in the neural network that need to be trained; is the L2 norm of the parameter vector θ.

[0057] An analysis report containing abnormal parameter identification, time point, and deviation value is generated by combining the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the abnormal parameter set. The method is: Parse the identification of the abnormal parameter and its corresponding occurrence time point from the abnormal parameter set; For the measured value of each abnormal parameter at the corresponding time point, retrieve the normal range value in the standardization database, and calculate the absolute deviation of the measured value from the normal range value; In the parameter correlation network, filter the associated parameters whose mutual information with the abnormal parameter exceeds a preset threshold; According to the classification results of the hierarchical convolutional neural network, label the flight phase category and parameter category to which the abnormal parameter belongs; Integrate the structured report containing the following elements: Abnormal parameter identifier and its corresponding time point; deviation value of abnormal parameter measured value from normal range value; list of strongly associated parameters associated with abnormal parameter; flight phase category and parameter category to which abnormal parameter belongs.

[0058] In this embodiment, the abnormal parameter identifier and its exact time point of occurrence are extracted from the abnormal parameter set. The identifier contains parameter name, data source device code, and physical quantity unit information. For each abnormal parameter at the corresponding time stamp, the pre-constructed standardized database is accessed to retrieve the normal value range of the parameter in the same flight phase (such as the cruise phase); the absolute deviation Δ of the measured value from the normal range value is calculated: Δ = |X 实测 - (X min + X max ) / 2 |; Where X 实测 is the actual measured value of the abnormal parameter, X min and X max are the lower limit and upper limit of the normal value range of the parameter in the same flight phase retrieved from the standardized database.

[0059] At the same time, the critical direction of the measured value exceeding the normal range (such as exceeding the upper limit or falling below the lower limit) is labeled.

[0060] In the parameter correlation network, the current abnormal parameter is taken as the query node, and all associated edges are traversed: Strongly associated parameters with edge weight (i.e. mutual information) exceeding a preset threshold (typical value 0.6) are screened; The associated parameter list is generated in descending order of mutual information; The topological distance and association type (positive / negative) of the associated parameters and the abnormal parameters are recorded; Based on the classification results of hierarchical convolutional neural network: The phase to which the abnormal time belongs (such as "climb phase") is extracted from the flight phase classification unit; The type label of the abnormal parameter (such as "engine parameter") is extracted from the parameter category classification unit; The parameter group representative relationship output by the redundancy removal layer is synchronously obtained (such as "vertical velocity represents vertical acceleration group"); The structured analysis report is integrated and generated, including the following core elements: Report field Data content example Data source Abnormal parameter identification N1_Engine_RPM [Unit: %] Abnormal parameter set Occurrence time point 23:47.4 Abnormal parameter set Measured value / normal range 103.5% (normal range: 98.2%-101.8%) Standardized database + deviation calculation Absolute deviation value Δ=2.1% Deviation calculation formula Strong correlation parameter list Fuel_Flow(0.82), EGT(0.79) Parameter correlation network screening Flight phase category Climb phase (confidence 92%) Hierarchical CNN first level output Parameter category Engine parameter (representative group: core speed / fuel flow) Hierarchical CNN second level output Although the steps are described in the above-mentioned order in the above-mentioned embodiments, it can be understood by those skilled in the art that, in order to achieve the effects of the embodiments, the different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are within the protection scope of the present application.

[0061] The second embodiment of the present application provides a data automatic analysis system for aviation accident investigation, which is used to realize a data automatic analysis method for aviation accident investigation, and the system comprises: a data screening module configured to collect original flight data from an aviation data source and perform data screening based on predefined flight stages and key parameters, and convert the screened data into editable structured format data; an analysis module configured to identify parameter types from the structured format data and perform differential analysis, wherein the parameter types include analog parameters, digital parameters and discrete mixed parameters; an abnormal parameter set acquisition module configured to perform standardized processing on the analyzed parameters, construct an isolation tree through recursive feature segmentation, calculate an abnormal score according to the average path length of data points in the isolation tree, and mark parameters with abnormal scores exceeding a threshold to form an abnormal parameter set; a parameter correlation network construction module configured to input the standardized flight data into an automatic encoder to extract a primary feature vector, input the compressed dimension into a variational autoencoder, generate a latent space representation of the nonlinear relationship between encoded parameters, and construct a parameter correlation network by calculating the mutual information between different dimensions to quantify the nonlinear correlation between parameters; a report generation module configured to input the compressed features into a pre-constructed hierarchical convolutional neural network, identify flight stages and key parameter categories through multi-level classifiers layer by layer, eliminate redundant parameters based on probability distribution similarity, and generate an analysis report containing abnormal parameter identification, time points and deviation values in combination with the classification results of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set.

[0062] Those skilled in the art can clearly understand the specific working process and related description of the system described above by referring to the corresponding process in the foregoing method embodiments for the convenience and brevity of description, which will not be repeated here.

[0063] It should be noted that the above embodiment provides a data automatic analysis system for aviation accident investigation, and only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the modules or steps in the embodiment of the application are further decomposed or combined, for example, the modules of the above embodiment can be combined into one module, or can be further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the application are only for distinguishing the respective modules and steps, and are not considered as improper limitation of the application.

[0064] The electronic device of the third embodiment of the application comprises: at least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the processor, the instructions being for execution by the processor to implement the above-mentioned data automatic analysis method for aviation accident investigation.

[0065] The computer readable storage medium of the fourth embodiment of the application stores computer instructions, the computer instructions being for execution by the computer to implement the above-mentioned data automatic analysis method for aviation accident investigation.

[0066] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the storage device and the processing device described above and the related descriptions can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0067] Those skilled in the art should realize that the modules and method steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. The programs corresponding to the software modules and method steps can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM or any other form of storage medium known in the art. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the foregoing description. Whether the functions are performed by electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0068] The computer program product of the present application can be a computer program product comprising a computer readable storage medium and a computer program mechanism embedded in the computer readable storage medium. Such computer program product can further include a computer readable storage medium and program means for causing a processor or other programmable processing apparatus to function in a particular manner, such that the computer program mechanism that can be executed by such a processor or processing apparatus causes the processor or processing apparatus to implement the functions of the system, method and computer program product of the present application.

[0069] The terms "first", "second", etc. are used to distinguish similar objects, rather than to describe or indicate a particular order or sequence.

[0070] The term "comprising" or any other similar term is intended to encompass the inclusion of non-exclusive inclusion, so that the process, method, article or equipment / device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes the elements inherent to the process, method, article or equipment / device.

[0071] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical solutions after the changes or replacements will all fall within the protection scope of the present application.

Claims

1. A data automation analysis method for aviation accident investigation, characterized in that: The method includes: Collect raw flight data from aviation data sources, filter the data based on predefined flight phases and key parameters, and convert the filtered data into editable structured format; Identify parameter types from structured format data and perform differentiated parsing, wherein the parameter types include analog parameters, digital parameters, and discrete mixed parameters; The parsed parameters are standardized, an isolation tree is constructed through recursive feature segmentation, an anomaly score is calculated based on the average path length of the data points in the isolation tree, and parameters with anomaly scores exceeding the threshold are marked to form an abnormal parameter set; The standardized flight data is fed into an autoencoder to extract primary feature vectors, which are then fed into a variational autoencoder after dimensionality compression to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions, thus constructing a parameter correlation network. The compressed features are input into a pre-built hierarchical convolutional neural network, and the flight phases and key parameter categories are identified layer by layer through a multi-level classifier. Redundant parameters are eliminated based on the similarity of probability distribution. The classification results of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set are combined to generate an analysis report containing abnormal parameter identification, time points and deviation values.

2. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that: Collect raw flight data from aviation data sources, including: Collects raw flight data stored in a hierarchical binary format from aviation data sources.

3. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that: Perform differential parsing as follows: For analog parameters, the numerical precision range is determined according to the original bit width, and the original data is decoded into actual physical values ​​according to the physical quantity conversion rules; For digital parameters, binary data is mapped to status identifiers based on the original bit width and predefined encoding rules; For discrete mixed quantity parameters, the discrete part and the analog part are separated and processed according to the digital quantity parameter analysis rules and analog quantity parameter analysis rules respectively.

4. The automated data analysis method for aviation accident investigation according to claim 3, characterized in that: The physical quantity conversion rules for analog parameters include: Convert integer data into actual physical values ​​according to the preset scaling factor; When the value exceeds the predefined range, it is marked as invalid data.

5. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that: By calculating the nonlinear correlation between the mutual information quantification parameters in different dimensions, a parameter correlation network is constructed. The method is as follows: Based on the latent space representation, calculating the mutual information values ​​between different dimensions to generate a dimension correlation strength matrix; Mapping the dimensions in the dimensional correlation strength matrix to the original flight parameters to obtain the nonlinear correlation strength between the parameters; With flight parameters as nodes and nonlinear correlation strength as edge weights, a graph structure network is constructed as a parameter correlation network.

6. The automated data analysis method for aviation accident investigation according to claim 5, characterized in that: The dimension correlation strength matrix is ​​calculated as follows: Calculate the mutual information value of all dimension pairs in the latent space representation to form a symmetric correlation matrix; Marking the dimension pairs whose element values ​​in the symmetric correlation matrix are greater than a preset threshold as a strongly correlated dimension group; A dimension correlation strength matrix is ​​generated based on the strongly correlated dimension group, and the matrix element values ​​reflect the nonlinear correlation strength between dimensions.

7. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that: The pre-built hierarchical convolutional neural network includes: Feature input layer, used to receive compressed feature vectors; A convolutional feature extraction layer, configured to extract the spatial correlation pattern of the feature vectors through convolution kernel operations and compress the feature dimensions through pooling operations; The first level classification layer includes: The flight phase local classification unit generates flight phase classification results based on the output of the convolutional feature extraction layer; a flight phase global fusion unit, which integrates the classification result with the initial global information flow and outputs a first global information flow; The second level classification layer includes: A parameter category local classification unit, which generates a parameter category classification result based on the output of the convolutional feature extraction layer and the first global information flow; a parameter category global fusion unit, which integrates the classification result with the first global information flow and outputs a final global information flow; Redundancy elimination layer, used to merge or delete redundant parameters based on the similarity of probability distribution of parameter categories in the same flight phase; The classification output layer is used to convert the final global information flow into a multi-category probability distribution.

8. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that: Combining the classification results of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set, an analysis report containing abnormal parameter identification, time point and deviation value is generated, and the method is as follows: Parsing the identifier of the abnormal parameter and the corresponding occurrence time point from the abnormal parameter set; For the measured value of each abnormal parameter at the corresponding time point, the normal range value in the standardized database was retrieved, and the absolute deviation between the measured value and the median of the normal range was calculated; In the parameter correlation network, screening associated parameters whose mutual information with abnormal parameters exceeds a preset threshold; According to the classification results of the hierarchical convolutional neural network, the flight phase category and parameter category to which the abnormal parameters belong are marked; The integrated output is a structured report containing the following elements: Abnormal parameter identification and its corresponding time point; deviation between the measured value of the abnormal parameter and the normal range value; list of associated parameters that are strongly associated with the abnormal parameter; flight phase category and parameter category to which the abnormal parameter belongs.

9. An automated data analysis system for aviation accident investigation, used to implement the automated data analysis method for aviation accident investigation according to any one of claims 1 to 8, characterized in that: The system includes: a data screening module configured to collect raw flight data from aviation data sources, screen the data based on predefined flight phases and key parameters, and convert the data into editable structured format data after screening; a parsing module configured to identify parameter types from structured format data and perform differentiated parsing, wherein the parameter types include analog parameters, digital parameters, and discrete mixed parameters; An abnormal parameter set acquisition module is configured to perform normalization processing on the parsed parameters, construct an isolation tree through recursive feature segmentation, calculate anomaly scores based on the average path length of data points in the isolation tree, and mark parameters with anomaly scores exceeding a threshold to form an abnormal parameter set; a parameter correlation network construction module configured to input the standardized flight data into an autoencoder to extract a primary feature vector, and then input the vector into a variational autoencoder after dimensionality compression to generate a latent space representation encoding the nonlinear relationship between parameters, and to quantify the nonlinear correlation between parameters by calculating the mutual information between different dimensions to construct a parameter correlation network; The report generation module is configured to input the compressed features into a pre-built hierarchical convolutional neural network, identify the flight phase and key parameter categories layer by layer through a multi-level classifier, eliminate redundant parameters based on probability distribution similarity, and combine the classification results of the hierarchical convolutional neural network, the parameter correlation network and the abnormal parameter set to generate an analysis report containing abnormal parameter identification, time points and deviation values.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the data automation analysis method for aviation accident investigation according to any one of claims 1 to 8.

Images