A method, system and apparatus for automated analysis of data for aviation accident investigation

Abstract

The present application belongs to the field of data analysis, and particularly relates to a data automatic analysis method, system and device for aviation accident investigation, aiming at solving the problems of poor data processing adaptability, insufficient preprocessing, performance bottleneck and modeling diagnosis limitation in the prior art. The present application comprises: collecting original flight data and converting into a structured format; identifying analog, digital and discrete mixed quantity parameters and performing differentiated analysis; after standardization processing, constructing a tree structure through recursive feature segmentation, calculating an abnormal score according to path length and generating an abnormal parameter set; extracting a feature vector to generate a latent space representation through a variational autoencoder, and constructing a parameter correlation network through mutual information calculation; combining hierarchical convolutional neural network classification results, parameter networks and abnormal parameter sets to generate an analysis report. Full-process automatic accident analysis is realized, and investigation efficiency and accuracy are improved.

Description

Technical Field

[0001] This invention belongs to the field of data analysis, and specifically relates to an automated data analysis method, system, and equipment for aviation accident investigation. Background Technology

[0002] While automated flight data analysis technology has made some progress in the field of aviation accident investigation, it still faces significant limitations. Existing systems are severely inadequate in processing data from unknown aircraft types and non-standard formats. For example, in the invention patent with publication number CN105391524A, data archiving relies on predefined aircraft identification codes. When encountering heterogeneous data formats of new or unregistered aircraft types (such as custom HDF5 structures or proprietary binary protocols), the lack of a dynamic parsing mechanism leads to processing failures, greatly limiting the system's adaptability.

[0003] Meanwhile, the automation and accuracy of the data preprocessing stage have obvious deficiencies. In their article "Design and Implementation of QAR Data Processing System Based on Teechart and VB Database" published in the journal "Aeronautical Maintenance and Engineering", Zhou Baizheng et al. pointed out that current tools still rely on manual intervention for common problems such as missing data and redundant header rows. In particular, they have weak capabilities in handling complex scenarios such as parameter jumps and cross-day timestamp anomalies (such as negative timestamp errors in fuel flow data due to cross-day flights). Differences in the units of parameters of different aircraft models (such as the mixing of imperial and metric units) further lead to deviations in subsequent analysis. These preprocessing defects not only increase time costs but also introduce the risk of human error.

[0004] At the system performance level, the centralized processing architecture and inefficient algorithm design form a dual bottleneck. The central server mode adopted by the invention patent with publication number CN105391524A causes data transmission delay, while the full batch processing model in "Design and Implementation of QAR Data Processing System Based on Teechart and VB Database" makes the analysis time increase linearly with the amount of data. In addition, the traditional machine learning model has insufficient accuracy in classifying heterogeneous labels, which seriously restricts the real-time response capability.

[0005] A deeper problem lies in the limitations of parameter correlation modeling and fault diagnosis. Existing methods generally ignore the dynamic correlation between parameters. For example, rule-based cleaning (such as fixed threshold elimination) may misjudge the coordinated fluctuations of engine speed and oil pressure as abnormal. Fault diagnosis relies too much on single-dimensional threshold judgment and cannot identify complex fault modes with multiple coupled parameters, such as the concurrent occurrence of increased vibration value and abnormal exhaust temperature. Furthermore, static models do not consider the differences in parameter characteristics at different flight stages, such as the threshold conflict between high thrust during takeoff and steady state during cruise, leading to a surge in the missed detection rate of complex faults.

[0006] Based on this, the present invention proposes an automated data analysis method, system and equipment for aviation accident investigation. Summary of the Invention

[0007] To address the aforementioned problems in the prior art, namely poor data processing adaptability, insufficient preprocessing, performance bottlenecks, and limitations in modeling and diagnosis, this invention provides an automated data analysis method, system, and device for aviation accident investigation.

[0008] In a first aspect, this invention proposes an automated data analysis method for aviation accident investigations, the method comprising:

[0009] Raw flight data is collected from aviation data sources and filtered based on predefined flight phases and key parameters. After filtering, the data is transformed into editable structured format data.

[0010] Identify parameter types from structured data and perform differential parsing; the parameter types include analog parameters, digital parameters, and discrete mixed parameters.

[0011] The parsed parameters are standardized, an isolated tree is constructed by recursive feature segmentation, anomaly scores are calculated based on the average path length of data points in the isolated tree, and parameters with anomaly scores exceeding the threshold are marked to form an anomaly parameter set.

[0012] Standardized flight data is input into an autoencoder to extract primary feature vectors, and after compression of dimensions, it is input into a variational autoencoder to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions, and a parameter correlation network is constructed.

[0013] The compressed features are input into a pre-constructed hierarchical convolutional neural network. Through a multi-level classifier, the flight phase and key parameter categories are identified layer by layer. Redundant parameters are removed based on probability distribution similarity. Combining the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the abnormal parameter set, an analysis report containing abnormal parameter identifiers, time points, and deviation values ​​is generated.

[0014] Furthermore, raw flight data is collected from aviation data sources, including:

[0015] Raw flight data is collected from aviation data sources and stored in a hierarchical binary format.

[0016] Furthermore, differential parsing is performed, and the method is as follows:

[0017] For analog parameters, the numerical accuracy range is determined based on the original bit width, and the original data is decoded into the actual physical value according to the physical quantity conversion rules.

[0018] For digital parameters, binary data is mapped to status identifiers based on the original bit width and predefined encoding rules;

[0019] For discrete mixed parameters, the discrete part and the analog part are separated and processed according to the analytical rules for digital parameters and the analytical rules for analog parameters, respectively.

[0020] Furthermore, the physical quantity conversion rules for analog parameters include:

[0021] Convert integer data to actual physical values ​​using a preset scaling factor;

[0022] Values ​​outside the predefined range are marked as invalid data.

[0023] Furthermore, by calculating the mutual information between different dimensions to quantify the nonlinear correlation between parameters, a parameter correlation network is constructed. The method is as follows:

[0024] Based on the latent space representation, the mutual information values ​​between different dimensions are calculated to generate a dimensional correlation strength matrix;

[0025] Mapping the dimensions in the dimensional correlation strength matrix to the original flight parameters yields the nonlinear correlation strength between the parameters.

[0026] Using flight parameters as nodes and nonlinear correlation strength as edge weights, a graph-structured network is constructed as the parameter correlation network.

[0027] Furthermore, the dimensional correlation strength matrix is ​​calculated as follows:

[0028] Calculate the mutual information values ​​of all dimension pairs in the latent space representation to form a symmetric correlation matrix;

[0029] The dimension pairs in the symmetric correlation matrix whose element values ​​are greater than a preset threshold are marked as strongly correlated dimension groups;

[0030] A dimension correlation strength matrix is ​​generated based on the strongly correlated dimension group, and the matrix element values ​​reflect the nonlinear correlation strength between dimensions.

[0031] Furthermore, the pre-built hierarchical convolutional neural network includes:

[0032] A feature input layer is used to receive the compressed feature vector;

[0033] The convolutional feature extraction layer is used to extract the spatial correlation pattern of the feature vector through convolution kernel operations and to compress the feature dimension through pooling operations.

[0034] The first level of classification includes:

[0035] The local classification unit for the flight phase generates flight phase classification results based on the output of the convolutional feature extraction layer.

[0036] The global fusion unit during the flight phase integrates the classification results with the initial global information stream and outputs the first global information stream;

[0037] The second level of classification includes:

[0038] The parameter category local classification unit generates parameter category classification results based on the output of the convolutional feature extraction layer and the first global information stream;

[0039] The parameter category global fusion unit integrates the classification results with the first global information stream and outputs the final global information stream;

[0040] The redundancy removal layer is used to merge or delete redundant parameters based on the similarity of the probability distribution of parameter categories in the same flight phase.

[0041] The classification output layer is used to transform the final global information stream into a multi-class probability distribution.

[0042] Furthermore, combining the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the set of outlier parameters, an analysis report containing outlier parameter identifiers, time points, and deviation values ​​is generated. The method is as follows:

[0043] The identifiers of the abnormal parameters and their corresponding occurrence times are parsed from the set of abnormal parameters;

[0044] For each abnormal parameter, the measured value at the corresponding time point is retrieved from the normal range value in the standardized database, and the absolute deviation between the measured value and the median value of the normal range is calculated.

[0045] In the parameter correlation network, parameters that have mutual information with abnormal parameters exceeding a preset threshold are filtered out.

[0046] Based on the classification results of the hierarchical convolutional neural network, the flight phase category and parameter category to which the abnormal parameters belong are labeled;

[0047] The integrated output includes a structured report containing the following elements:

[0048] Abnormal parameter identifiers and their corresponding time points; deviations between measured values ​​of abnormal parameters and normal range values; a list of related parameters strongly correlated with abnormal parameters; and the flight phase category and parameter category to which the abnormal parameter belongs.

[0049] In another aspect, the present invention proposes an automated data analysis system for aviation accident investigations, which implements an automated data analysis method for aviation accident investigations. The system includes:

[0050] The data filtering module is configured to collect raw flight data from aviation data sources and filter the data based on predefined flight phases and key parameters, and then convert the filtered data into editable structured format data.

[0051] The parsing module is configured to identify parameter types from structured format data and perform differential parsing, wherein the parameter types include analog parameters, digital parameters, and discrete mixed parameters;

[0052] The abnormal parameter set acquisition module is configured to perform standardization processing on the parsed parameters, construct an isolated tree through recursive feature segmentation, calculate the abnormal score based on the average path length of the data points in the isolated tree, and mark the parameters whose abnormal scores exceed the threshold to form an abnormal parameter set.

[0053] The parameter correlation network construction module is configured to input standardized flight data into an autoencoder to extract primary feature vectors, and then input them into a variational autoencoder after dimensional compression to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions to construct the parameter correlation network.

[0054] The report generation module is configured to input compressed features into a pre-constructed hierarchical convolutional neural network, identify flight phases and key parameter categories layer by layer through a multi-level classifier, remove redundant parameters based on probability distribution similarity, and generate an analysis report containing abnormal parameter identifiers, time points, and deviation values ​​by combining the classification results of the hierarchical convolutional neural network, parameter correlation network, and abnormal parameter set.

[0055] A third aspect of the present invention provides an electronic device comprising:

[0056] At least one processor; and

[0057] A memory communicatively connected to at least one of the processors; wherein,

[0058] The memory stores instructions that can be executed by the processor to implement the aforementioned automated data analysis method for aviation accident investigation.

[0059] The beneficial effects of this invention are:

[0060] This invention establishes a differentiated analytical framework based on parameter type by scientifically classifying flight parameters into three categories: analog parameters, digital parameters, and discrete hybrid parameters. The system employs a technical approach combining predefined analytical rules with a dictionary list structure to achieve adaptive identification of data from various aircraft types, completely overcoming the limitations of traditional solutions that rely on fixed aircraft identification codes. In the data preprocessing stage, a recursive feature segmentation isolated tree algorithm is introduced to automatically identify abnormal parameter fluctuations and timestamp errors based on the average path length of data points within the isolated tree. This intelligent processing mechanism eliminates the need for manually preset fixed thresholds, autonomously adapting to the characteristics of different flight scenarios and significantly reducing the need for human intervention.

[0061] This invention first utilizes an autoencoder to extract primary feature vectors, and then maps them to a latent space representation using a variational autoencoder. This process not only achieves intelligent compression of data dimensions, but more importantly, it constructs a correlation network that reveals the complex nonlinear relationships between parameters by quantifying the mutual information between different dimensions. Based on this, combined with the multi-level classification mechanism of a hierarchical convolutional neural network, the system achieves automatic identification of features and classification of key parameters during the flight phase, while automatically eliminating redundant parameter terms based on probability distribution similarity.

[0062] This invention utilizes a variational autoencoder to generate a nonlinear correlation network, an isolated tree algorithm to label anomaly parameter sets, and a hierarchical convolutional network to identify flight phase features. Through deep coupling analysis of this three-dimensional data, the system can accurately capture complex fault modes, such as the synergistic failure of abnormal engine vibration and oil pressure fluctuations. The entire analysis process achieves end-to-end automated closed-loop processing, starting from raw data acquisition, undergoing intelligent parsing and cleaning, to correlation modeling and multi-dimensional feature detection, and finally automatically generating an analysis report containing the time points and deviation values ​​of anomaly parameters.

[0063] This invention also possesses self-evolving knowledge capabilities. By constructing an extensible fault diagnosis knowledge base and setting up an open rule engine interface, it continuously absorbs new fault modes and aircraft model data characteristics. This dynamic update mechanism enables the system to autonomously optimize parameter identification rules and fault diagnosis logic without reconstructing the underlying architecture when facing protocol changes or new flight states for different aircraft models, achieving a significant transformation from passive response to proactive adaptation. Attached Figure Description

[0064] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0065] Figure 1 This is a flowchart illustrating an automated data analysis method for aviation accident investigation according to the present invention. Detailed Implementation

[0066] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0067] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0068] The first embodiment of the present invention provides an automated data analysis method for aviation accident investigations, the method comprising:

[0069] Step S10: Collect raw flight data from aviation data sources, filter the data based on predefined flight phases and key parameters, and convert the filtered data into editable structured format data.

[0070] Step S20: Identify parameter types from structured format data and perform differential parsing. The parameter types include analog parameters, digital parameters, and discrete mixed parameters.

[0071] Step S30: Standardize the parsed parameters, construct an isolated tree through recursive feature segmentation, calculate the anomaly score based on the average path length of the data points in the isolated tree, and mark the parameters whose anomaly scores exceed the threshold to form an anomaly parameter set.

[0072] Step S40: Standardized flight data is input into an autoencoder to extract primary feature vectors, and after compression of dimensions, it is input into a variational autoencoder to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions, and a parameter correlation network is constructed.

[0073] Step S50: Input the compressed features into the pre-constructed hierarchical convolutional neural network. The multi-level classifier identifies the flight stage and key parameter categories layer by layer, and removes redundant parameters based on probability distribution similarity. Combine the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the abnormal parameter set to generate an analysis report containing abnormal parameter identifiers, time points, and deviation values.

[0074] To more clearly illustrate the automated data analysis method for aviation accident investigation according to the present invention, the following is in conjunction with... Figure 1 The steps in the embodiments of the present invention are described in detail below, including steps S10-S50:

[0075] Step S10: Collect raw flight data from aviation data sources, filter the data based on predefined flight phases and key parameters, and convert the filtered data into editable structured format data.

[0076] Specifically, raw flight data is collected from diverse data sources, including airlines, flight data recorders, and flight management systems. This raw data is stored in a hierarchical binary format and includes key information such as flight parameters, timestamps, and flight phase metadata. Based on predefined flight phase division criteria, including but not limited to takeoff, cruise, and landing phases, and combined with key flight parameter indicators such as altitude, airspeed, and engine operating parameters, the raw data undergoes preliminary screening. The screened data is then automatically converted into a readable and editable structured format according to its data structure characteristics. This conversion process ensures that the data retains its original hierarchical relationships and parameter integrity.

[0077] During the data transformation process, we conduct in-depth analysis of the characteristics and differences of different data types, including structured, semi-structured, and unstructured data. We standardize and validate hierarchical binary format files to ensure they conform to aviation data specifications, and repair or remove non-compliant data. Simultaneously, we design and implement a flexible data integration framework to achieve unified management and efficient integration of heterogeneous data, ensuring editability and structural consistency after data transformation.

[0078] To address quality issues during data integration, the system performs a comprehensive data cleaning process, including key operations such as missing value imputation, duplicate value removal, and outlier correction. For missing data points, linear interpolation or calculation based on the mean of adjacent data points is used for imputation. For outlier data values, corrections are made based on statistical distribution characteristics (such as Z-score standardization or interquartile range (IQR) analysis) combined with aeronautical physical meaning. Finally, all parameters are standardized to eliminate dimensional differences between data from different aircraft types, unify numerical ranges and scales, and establish a standardized data foundation for subsequent analysis.

[0079] Step S20: Identify parameter types from structured format data and perform differential parsing. The parameter types include analog parameters, digital parameters, and discrete mixed parameters.

[0080] In this embodiment, differential parsing is performed using the following method:

[0081] For analog parameters, the numerical accuracy range is determined based on the original bit width, and the original data is decoded into the actual physical value according to the physical quantity conversion rules.

[0082] For digital parameters, binary data is mapped to status identifiers based on the original bit width and predefined encoding rules;

[0083] For discrete mixed parameters, the discrete part and the analog part are separated and processed according to the analytical rules for digital parameters and the analytical rules for analog parameters, respectively.

[0084] The physical quantity conversion rules for analog parameters include:

[0085] Convert integer data to actual physical values ​​using a preset scaling factor;

[0086] Values ​​outside the predefined range are marked as invalid data.

[0087] In this embodiment, three types of flight parameters are identified from structured data: analog parameters, digital parameters, and discrete mixed parameters, and differential parsing processing is performed. For analog parameters, the numerical precision range is first determined based on their original bit width; for example, when the bit width is 16 bits, the numerical range is 0-65535.

[0088] Then, the original data is decoded into actual physical values ​​according to the physical quantity conversion rules. Specifically, this includes: multiplying integer data by a preset scaling factor, such as multiplying 16-bit temperature data by 0.01 to convert it into an actual Celsius value; converting floating-point data according to the decimal point position; and marking data as invalid when the value exceeds the predefined range (such as altitude parameters exceeding 0-50000 feet).

[0089] For digital parameters, the number of states is determined based on the original bit width. For example, a 2-bit parameter can represent 4 states. Combined with predefined encoding rules, including binary encoding, Gray code, or other special encoding methods, binary data is mapped to physical state identifiers. For example, the binary value "00" is mapped to the engine "off" state, "01" is mapped to the "starting" state, 10 represents "running", and 11 represents "fault". It also supports the parsing of multiple sub-state combinations. For example, the first 4 bits of an 8-bit parameter are parsed as engine 1 state, and the last 4 bits are parsed as engine 2 state. Parity checking or redundancy coding mechanisms are implemented to detect transmission errors.

[0090] Parity Check: If a parity bit is used for a digital parameter, it is necessary to define how to detect and correct errors. For example, checking the parity bit can help determine if an error occurred during data transmission.

[0091] Redundant coding: If redundant coding (such as repetitive coding) is used, it is necessary to define how to correct errors using redundant information.

[0092] For discrete mixed parameters, discrete states and analog data are separated according to a predefined data structure (e.g., the first 8 bits of a 24-bit parameter are the discrete part, and the last 16 bits are the analog part). The discrete part is mapped to the state according to the rules of digital parameters, while the analog part is converted to physical quantities according to the rules of analog parameters.

[0093] When two parts of data conflict (e.g., the discrete part indicates a sensor fault while the analog part shows a normal value), the status indicator of the discrete part is processed first. All parsing rules are dynamically managed through configuration files, supporting adaptation to frame structure differences of different models. After parsing, the results are verified by comparing with standard data or simulation tests to ensure that the parsing efficiency and accuracy meet the requirements of real-time processing.

[0094] By employing the above methods for parsing parameters of different data types, each parameter undergoes precise conversion and decoding. For the decoded data, pandas library functions (such as `dropna()`, `fillna()`, `drop_duplicates()`, etc.) are used to handle missing, duplicate, and outlier values. Data is also formatted, such as to standardize date and numeric formats, to ensure data consistency. Libraries like `dask` or `modin` are used for parallel processing to improve data processing efficiency. Furthermore, the above steps are written into an automated script for repeated execution of the data merging task. Python's `argparse` or `click` libraries are used to handle command-line arguments, making the script more flexible and easier to use.

[0095] Step S30: Standardize the parsed parameters, construct an isolated tree through recursive feature segmentation, calculate the anomaly score based on the average path length of the data points in the isolated tree, and mark the parameters whose anomaly scores exceed the threshold to form an anomaly parameter set.

[0096] Step S30 includes the following sub-steps executed in sequence:

[0097] Step S31: Perform standardization transformation on the parsed flight parameters to eliminate the dimensional differences between different parameters and generate a standardized dataset with unified dimensions.

[0098] Step S32: Based on the standardized dataset, a tree-like detection structure is constructed by recursively segmenting the feature space. The segmentation process randomly selects the feature dimension and its segmentation threshold.

[0099] Step S33: Calculate the anomaly score based on the path length of the data point in the tree structure, where the path length reflects the difficulty of the data point being segmented and isolated;

[0100] Step S34: Mark the parameters whose abnormal scores exceed the preset threshold and their corresponding time points to form a structured abnormal parameter set.

[0101] Specifically, multi-mode standardization is performed on the analyzed flight parameters to eliminate dimensional differences: for parameters with symmetrical data distributions, Z-score standardization is used to convert the parameter values ​​into a distribution with a mean of 0 and a standard deviation of 1; for parameters with asymmetrical distributions, interquartile range (IQR) standardization is used, with data scaling performed through the first quartile (Q1) and the third quartile (Q3). The specific transformation formula is X... norm = (X-Q1) / (Q3-Q1). After standardization, construct an isolated forest anomaly detection model: randomly select parameter features and their segmentation thresholds, recursively divide the data space into subspaces until all data points are isolated or the preset tree depth limit is reached (typically 8 layers), and repeatedly construct multiple independent trees to form an isolated tree set (typically 100 trees).

[0102] Calculate outlier scores based on the path length of each data point in all isolated trees: First, calculate the mean path length of each data point in all isolated trees. ,in T The total number of isolated trees, h i ( x Let be the path length in the i-th tree; then, it is normalized using a function. Transform abnormal scores, where c ( n The expected path length is (obtained by simulating the average path length of a binary search tree for n samples). An anomaly score threshold of 0.7 is set, and all parameters with anomaly scores exceeding the threshold and their corresponding timestamps are marked, generating a structured set of anomaly parameters. This set contains the following core fields: parameter name, anomaly time point, original value before standardization, reference value for the normal value range, and anomaly score.

[0103] During anomaly detection, the feature segmentation path is automatically recorded, including the feature dimensions selected for each recursive segmentation, the segmentation threshold, and the subspace partitioning results. A 3D tree structure diagram is generated using visualization tools, highlighting the isolated positions of anomaly parameters in the tree structure (such as short path branches near the root node). Simultaneously, a feature importance ranking list is output (arranged in descending order of the frequency with which features are selected during segmentation). Finally, the anomaly parameter set, feature segmentation path logs, and tree structure visualization diagram are packaged and stored for review by the incident investigation module.

[0104] Step S40: Standardized flight data is input into an autoencoder to extract primary feature vectors, and after compression of dimensions, it is input into a variational autoencoder to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions, and a parameter correlation network is constructed.

[0105] In this embodiment, a parameter correlation network is constructed by calculating the nonlinear correlation between parameters based on mutual information across different dimensions. The method is as follows:

[0106] Based on the latent space representation, the mutual information values ​​between different dimensions are calculated to generate a dimensional correlation strength matrix;

[0107] Mapping the dimensions in the dimensional correlation strength matrix to the original flight parameters yields the nonlinear correlation strength between the parameters.

[0108] Using flight parameters as nodes and nonlinear correlation strength as edge weights, a graph-structured network is constructed as the parameter correlation network.

[0109] The dimensional correlation strength matrix is ​​calculated as follows:

[0110] Calculate the mutual information values ​​of all dimension pairs in the latent space representation to form a symmetric correlation matrix;

[0111] The dimension pairs in the symmetric correlation matrix whose element values ​​are greater than a preset threshold are marked as strongly correlated dimension groups;

[0112] A dimension correlation strength matrix is ​​generated based on the strongly correlated dimension group, and the matrix element values ​​reflect the nonlinear correlation strength between dimensions.

[0113] In this embodiment, based on the latent space representation generated by the variational autoencoder (VAE), a VAE model is first constructed:

[0114] The encoder uses a fully connected neural network to process the input feature vector. E Mapped to the mean of the latent space μ and variance σ 2 ,satisfy μ=f θ (E) and log(σ 2 )=g θ (E) ;in, f θ and g θ These are two parallel neural network branches inside the encoder;

[0115] Through reparameterization techniques z = μ + σ × Generate differentiable latent variables, where It is sampled from a standard normal distribution; the decoder is through the decoder Reconstruct the feature vectors, and the model training objective function L is the sum of the reconstruction error and the KL divergence:

[0116] ;

[0117] in, n For feature dimension, d As a potential spatial dimension, μ j Let be the mean of the j-th dimension in the latent space. σ j Let E be the standard deviation of the j-th dimension in the latent space. i Let i be the i-th feature value of the original input data.

[0118] Based on the training of a converged VAE model, the latent space representation is... z Perform mutual information calculation: use kernel density estimation to statistically analyze the marginal probability distributions of each dimension. p ( z i ) and dimension of the joint probability distribution p ( z i , z j ), using the entropy difference formula:

[0119] ;

[0120] Quantify the nonlinear dependencies of all dimension pairs to generate a symmetric dimension correlation strength matrix. Set a preset threshold to filter strongly correlated dimension groups (e.g., mutual information value > 0.5) to form a simplified correlation matrix.

[0121] By reverse-analyzing the decoder weight matrix, a mapping relationship between the latent spatial dimensions and the original flight parameters is established: the maximum response connectivity in the weight matrix is ​​located, and a dimension-parameter mapping dictionary is constructed. The dimension correlation strength values ​​are then converted into nonlinear correlation strengths between parameters.

[0122] Using flight parameters as nodes (node ​​attributes include parameter name and physical unit) and normalized association strength as edge weights, a weighted undirected graph network is constructed. Edge weight pruning is performed: weakly associated edges below a connection threshold (e.g., 0.3) are removed, while significant associations are preserved. A force-directed algorithm is used for layout visualization, enabling strongly associated parameters to form spatial clusters (e.g., engine parameter clusters: speed-fuel flow-turbine temperature), and a community detection algorithm is applied to identify topological hub parameters. Finally, the sparse network is stored as a graph database, supporting real-time topology queries and accident diagnosis module calls, completing a closed-loop transformation from potential spatial analysis to engineering applications.

[0123] Step S50: Input the compressed features into the pre-constructed hierarchical convolutional neural network. The multi-level classifier identifies the flight stage and key parameter categories layer by layer, and removes redundant parameters based on probability distribution similarity. Combine the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the abnormal parameter set to generate an analysis report containing abnormal parameter identifiers, time points, and deviation values.

[0124] The pre-built hierarchical convolutional neural network includes:

[0125] A feature input layer is used to receive the compressed feature vector;

[0126] The convolutional feature extraction layer is used to extract the spatial correlation pattern of the feature vector through convolution kernel operations and to compress the feature dimension through pooling operations.

[0127] The first level of classification includes:

[0128] The local classification unit for the flight phase generates flight phase classification results based on the output of the convolutional feature extraction layer.

[0129] The global fusion unit during the flight phase integrates the classification results with the initial global information stream and outputs the first global information stream;

[0130] The second level of classification includes:

[0131] The parameter category local classification unit generates parameter category classification results based on the output of the convolutional feature extraction layer and the first global information stream;

[0132] The parameter category global fusion unit integrates the classification results with the first global information stream and outputs the final global information stream;

[0133] The redundancy removal layer is used to merge or delete redundant parameters based on the similarity of the probability distribution of parameter categories in the same flight phase.

[0134] The classification output layer is used to transform the final global information stream into a multi-class probability distribution.

[0135] The pre-constructed hierarchical convolutional neural network includes a feature input layer that receives compressed and dimensionality-reduced feature vectors and converts the flight parameter sequence into multi-channel image data, where each channel corresponds to the time series data of a single flight parameter. The convolutional feature extraction layer performs sliding convolution operations on the input data using trainable convolutional kernels to capture local spatial correlation patterns between parameters and downsamples the convolution output using max pooling, compressing the feature dimension while preserving key feature responses.

[0136] The first-level classification layer includes a local classification unit and a global fusion unit for flight phases: the local classification unit analyzes the feature map output by the convolutional layer through a fully connected neural network to generate an initial flight phase classification result (including five categories: takeoff, climb, cruise, descent, and landing); the global fusion unit receives the classification result and the initialized zero-value global information stream, integrates the information through a gating mechanism, and outputs a first global information stream carrying flight phase features.

[0137] The second-level classification layer includes a local classification unit and a global fusion unit for parameter categories: the local classification unit generates classification results for key parameter categories such as engine parameters, altitude parameters, and speed parameters based on the convolutional layer feature map and the first global information stream; the global fusion unit fuses the classification results with the first global information stream through attention weights to generate the final global information stream carrying parameter category features.

[0138] The redundancy removal layer calculates the similarity of the category probability distributions of parameters within the same flight phase based on the final global information flow. For parameter groups with similarity exceeding a dynamic threshold (such as vertical velocity, vertical acceleration, and radio altitude), cluster analysis is performed, retaining only representative parameters for each category and removing redundant parameters. The classification output layer inputs the final global information flow into a Softmax function, generating multi-class probability distributions and outputting simplified key parameter category labels. Simultaneously, it records the classification confidence of each parameter and the redundancy removal decision path.

[0139] The entire processing involves end-to-end training to optimize network weights. Cross-entropy loss is used for both flight phase classification and parameter category classification loss functions, while redundancy removal is achieved through gradient backpropagation via a differentiable clustering algorithm. The network output, along with the parameter correlation network and abnormal parameter set, is collaboratively input into the accident diagnosis module to complete the closed-loop analysis process.

[0140] More specifically, the feature input layer receives compressed and dimensionality-reduced feature vectors, converting the flight parameter sequence into multi-channel image data:

[0141] Data Reconstruction: n One flight cycle (flights with different aircraft types) m Each flight segment, p The three-dimensional data organization of each flight parameter is as follows n × m × p Image formats

[0142] Channel definition: Each channel corresponds to the time series data of a single flight parameter (e.g., channel 1: altitude parameter, channel 2: airspeed parameter).

[0143] The convolutional feature extraction layer performs the following operations:

[0144] Convolution operation: usingk Each size is f × f The trainable convolutional kernel is used for sliding convolution, and the output feature map size is ( nf +1)×( mf +1)× k；

[0145] Feature compression: Employs max pooling operation (pooling window) s × s Downsampling, output size is ;

[0146] First-level classification layer (flight phase identification):

[0147] Local classification unit: A fully connected neural network analyzes the convolutional feature map and outputs the probability distribution of the flight phase. :

[0148] ;

[0149] in E For CNN feature vectors, φ It uses the ReLU activation function, and the output dimensions correspond to five categories: takeoff, climb, cruise, descent, and landing. and These are the weight vector and bias vector of the first layer of the local classification unit, respectively;

[0150] Global Fusion Unit: Integrates classification results with the global information flow of the initial zero value. :

[0151] ;

[0152] in, The vector concatenation generates a first global information stream carrying features of the flight phase. and These are the weight matrix and bias vector of the first layer of the global fusion unit, respectively;

[0153] Second-level classification layer (parameter category recognition):

[0154] Local classification unit: based on convolutional feature maps and the first global information stream :

[0155] ;

[0156] in, and These are the weight vector and bias vector of the second layer of the local classification unit, respectively;

[0157] Output probability distributions for categories such as engine parameters, altitude parameters, and speed parameters;

[0158] Global Fusion Unit: Fusion of information through attention weights :

[0159] ;

[0160] Generate the final global information stream carrying parameter category features;

[0161] Redundant parameter handling

[0162] The redundancy removal layer performs the following operations:

[0163] Extract the category probability distribution of r different flight parameters within the same flight phase. ;

[0164] Calculate the Jensen-Shannon divergence similarity between parameters :

[0165] ;

[0166] in, Let M be the i-th and j-th probability distributions in set P; M is the probability distribution... Arithmetic mean distribution ; This is the Kullback-Leibler divergence, also known as relative entropy.

[0167] Perform spectral clustering on parameter groups (such as vertical velocity / vertical acceleration / radio altitude) whose similarity exceeds a dynamic threshold (e.g., 0.85);

[0168] Retain representative parameters for each group (such as vertical velocity) and eliminate redundant parameters;

[0169] Classification Output and Training

[0170] Classification output layer processing:

[0171] The final global information flow Input the Softmax function:

[0172] ;

[0173] in, and They are respectively The values ​​of the j-th and k-th elements in the array;

[0174] Output the simplified parameter category labels and confidence levels;

[0175] Record redundant elimination decision paths (e.g., "radio altitude is represented by vertical velocity");

[0176] End-to-end training:

[0177] Loss function:

[0178] Flight phase classification: cross-entropy loss ;

[0179] Parameter category classification: Cross-entropy loss ;

[0180] Differentiable clustering: Backpropagation of gradients in the redundant removal layer is achieved through the Gumbel-Softmax approximation.

[0181] Joint optimization: Total loss .

[0182] in, True labels for flight phase classification Real labels for classifying various flight parameter categories; This represents the probability distribution of the model's predictions for flight parameter categories. λ θ represents the weight hyperparameters of the L2 regularization term; θ is the set of all parameters (weights and biases) that need to be trained in the neural network. Let θ be the L2 norm of the parameter vector θ.

[0183] Combining the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the set of outlier parameters, an analysis report containing outlier parameter identifiers, time points, and deviation values ​​is generated. The method is as follows:

[0184] The identifiers of the abnormal parameters and their corresponding occurrence times are parsed from the set of abnormal parameters;

[0185] For each abnormal parameter, the measured value at the corresponding time point is retrieved from the normal range value in the standardized database, and the absolute deviation between the measured value and the median value of the normal range is calculated.

[0186] In the parameter correlation network, parameters that have mutual information with abnormal parameters exceeding a preset threshold are filtered out.

[0187] Based on the classification results of the hierarchical convolutional neural network, the flight phase category and parameter category to which the abnormal parameters belong are labeled;

[0188] The integrated output includes a structured report containing the following elements:

[0189] Abnormal parameter identifiers and their corresponding time points; deviations between measured values ​​of abnormal parameters and normal range values; a list of related parameters strongly correlated with abnormal parameters; and the flight phase category and parameter category to which the abnormal parameter belongs.

[0190] In this embodiment, the identifiers of the abnormal parameters and their precise occurrence times are extracted one by one from the abnormal parameter set. The identifiers include the parameter name, data source device code, and physical quantity unit information. For the measured value of each abnormal parameter at the corresponding timestamp, a pre-built standardized database is accessed to retrieve the normal value range of the parameter in the same flight phase (e.g., cruise phase); the absolute deviation Δ between the measured value and the midpoint of the normal range is calculated.

[0191] Δ=|X 实测 -(X) min +X max ) / 2∣;

[0192] Among them, X 实测 X represents the actual measured value of the abnormal parameter. min and X max These are the lower and upper limits of the normal range of this parameter, retrieved from a standardized database, under the same flight phase.

[0193] At the same time, the critical direction in which the measured value exceeds the normal range (such as exceeding the upper limit or falling below the lower limit) should be marked.

[0194] In the parameter correlation network, taking the current abnormal parameter as the query node, traverse all its associated edges:

[0195] Filter strongly correlated parameters whose edge weights (i.e., mutual information) exceed a preset threshold (typically 0.6);

[0196] Generate a list of associated parameters in descending order of mutual information.

[0197] Record the topological distance and correlation type (positive correlation / negative correlation) between the correlated parameters and the abnormal parameters;

[0198] Classification results based on hierarchical convolutional neural networks:

[0199] Extract the phase to which the abnormal moment belongs from the flight phase classification unit (e.g., "climb phase");

[0200] Extract the type labels (such as "engine parameters") of abnormal parameters from the parameter category classification unit;

[0201] Synchronously acquire the parameter group representation relationship of the output of the redundancy removal layer (e.g., "vertical velocity represents vertical acceleration group");

[0202] The integrated structured analysis report includes the following core elements:

[0203] Report fields Data content example Data source Abnormal parameter identifier N1_Engine_RPM [Unit: %] Abnormal parameter set Time of occurrence 23:47.4 Abnormal parameter set Measured value / Normal range 103.5% (Normal range: 98.2%-101.8%) Standardized database + deviation calculation absolute deviation value Δ=2.1% Deviation calculation formula Strongly related parameter list Fuel_Flow(0.82), EGT(0.79) Parameter correlation network screening Flight Phase Categories Climbing phase (92% confidence level) Hierarchical CNN first layer output Parameter categories Engine parameters (representing groups: core speed / fuel flow) Hierarchical CNN second-level output

[0204] Although the steps in the above embodiments are described in the above order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not need to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple variations are all within the protection scope of this invention.

[0205] A second embodiment of the present invention proposes an automated data analysis system for aviation accident investigations, which implements an automated data analysis method for aviation accident investigations. The system includes:

[0206] The data filtering module is configured to collect raw flight data from aviation data sources and filter the data based on predefined flight phases and key parameters, and then convert the filtered data into editable structured format data.

[0207] The parsing module is configured to identify parameter types from structured format data and perform differential parsing, wherein the parameter types include analog parameters, digital parameters, and discrete mixed parameters;

[0208] The abnormal parameter set acquisition module is configured to perform standardization processing on the parsed parameters, construct an isolated tree through recursive feature segmentation, calculate the abnormal score based on the average path length of the data points in the isolated tree, and mark the parameters whose abnormal scores exceed the threshold to form an abnormal parameter set.

[0209] The parameter correlation network construction module is configured to input standardized flight data into an autoencoder to extract primary feature vectors, and then input them into a variational autoencoder after dimensional compression to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions to construct the parameter correlation network.

[0210] The report generation module is configured to input compressed features into a pre-constructed hierarchical convolutional neural network, identify flight phases and key parameter categories layer by layer through a multi-level classifier, remove redundant parameters based on probability distribution similarity, and generate an analysis report containing abnormal parameter identifiers, time points, and deviation values ​​by combining the classification results of the hierarchical convolutional neural network, parameter correlation network, and abnormal parameter set.

[0211] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0212] It should be noted that the automated data analysis system for aviation accident investigation provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.

[0213] An electronic device according to a third embodiment of the present invention includes:

[0214] At least one processor; and

[0215] A memory communicatively connected to at least one of the processors; wherein,

[0216] The memory stores instructions that can be executed by the processor to implement the aforementioned automated data analysis method for aviation accident investigation.

[0217] A fourth embodiment of the present invention provides a computer-readable storage medium storing computer instructions, which are executed by the computer to implement the above-described automated data analysis method for aviation accident investigation.

[0218] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the storage device and processing device described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0219] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention.

[0220] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0221] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0222] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0223] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An automated data analysis method for aviation accident investigations, characterized in that, The method includes: Raw flight data is collected from aviation data sources and filtered based on predefined flight phases and key parameters. After filtering, the data is transformed into editable structured format data. Identify parameter types from structured data and perform differential parsing; the parameter types include analog parameters, digital parameters, and discrete mixed parameters. The parsed parameters are standardized, an isolated tree is constructed by recursive feature segmentation, anomaly scores are calculated based on the average path length of data points in the isolated tree, and parameters with anomaly scores exceeding the threshold are marked to form an anomaly parameter set. Standardized flight data is input into an autoencoder to extract primary feature vectors, and after compression of dimensions, it is input into a variational autoencoder to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions, and a parameter correlation network is constructed. The compressed features are input into a pre-constructed hierarchical convolutional neural network. A multi-level classifier identifies flight phases and key parameter categories layer by layer, and redundant parameters are removed based on probability distribution similarity. Combining the classification results of the hierarchical convolutional neural network, the parameter correlation network, and the set of abnormal parameters, an analysis report containing abnormal parameter identifiers, time points, and deviation values ​​is generated. The identifiers of the abnormal parameters and their corresponding occurrence times are parsed from the set of abnormal parameters; For each abnormal parameter, the measured value at the corresponding time point is retrieved from the normal range value in the standardized database, and the absolute deviation between the measured value and the median value of the normal range is calculated. In the parameter correlation network, parameters that have mutual information with abnormal parameters exceeding a preset threshold are filtered out. Based on the classification results of the hierarchical convolutional neural network, the flight phase category and parameter category to which the abnormal parameters belong are labeled; The integrated output includes a structured report containing the following elements: Abnormal parameter identifiers and their corresponding time points; deviations between measured values ​​of abnormal parameters and normal range values; a list of related parameters strongly correlated with abnormal parameters; and the flight phase category and parameter category to which the abnormal parameter belongs.

2. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that, Raw flight data is collected from aviation data sources, including: Raw flight data is collected from aviation data sources and stored in a hierarchical binary format.

3. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that, The method for performing differential parsing is as follows: For analog parameters, the numerical accuracy range is determined based on the original bit width, and the original data is decoded into the actual physical value according to the physical quantity conversion rules. For digital parameters, binary data is mapped to status identifiers based on the original bit width and predefined encoding rules; For discrete mixed parameters, the discrete part and the analog part are separated and processed according to the analytical rules for digital parameters and the analytical rules for analog parameters, respectively.

4. The automated data analysis method for aviation accident investigation according to claim 3, characterized in that, The physical quantity conversion rules for analog parameters include: Convert integer data to actual physical values ​​using a preset scaling factor; Values ​​outside the predefined range are marked as invalid data.

5. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that, A parameter correlation network is constructed by calculating the nonlinear correlation between parameters through mutual information across different dimensions. The method is as follows: Based on the latent space representation, the mutual information values ​​between different dimensions are calculated to generate a dimensional correlation strength matrix; Mapping the dimensions in the dimensional correlation strength matrix to the original flight parameters yields the nonlinear correlation strength between the parameters. Using flight parameters as nodes and nonlinear correlation strength as edge weights, a graph-structured network is constructed as the parameter correlation network.

6. The automated data analysis method for aviation accident investigation according to claim 5, characterized in that, The dimensional correlation strength matrix is ​​calculated as follows: Calculate the mutual information values ​​of all dimension pairs in the latent space representation to form a symmetric correlation matrix; The dimension pairs in the symmetric correlation matrix whose element values ​​are greater than a preset threshold are marked as strongly correlated dimension groups; A dimension correlation strength matrix is ​​generated based on the strongly correlated dimension group, and the matrix element values ​​reflect the nonlinear correlation strength between dimensions.

7. The automated data analysis method for aviation accident investigation according to claim 1, characterized in that, The pre-built hierarchical convolutional neural network includes: The feature input layer is used to receive the compressed feature vector; The convolutional feature extraction layer is used to extract the spatial correlation pattern of the feature vector through convolution kernel operations and to compress the feature dimension through pooling operations. The first level of classification includes: The local classification unit for the flight phase generates flight phase classification results based on the output of the convolutional feature extraction layer. The global fusion unit during the flight phase integrates the classification results with the initial global information stream and outputs the first global information stream; The second level of classification includes: The parameter category local classification unit generates parameter category classification results based on the output of the convolutional feature extraction layer and the first global information stream; The parameter category global fusion unit integrates the classification results with the first global information stream and outputs the final global information stream; The redundancy removal layer is used to merge or delete redundant parameters based on the similarity of the probability distribution of parameter categories in the same flight phase. The classification output layer is used to transform the final global information stream into a multi-class probability distribution.

8. An automated data analysis system for aviation accident investigation, used to implement the automated data analysis method for aviation accident investigation as described in any one of claims 1-7, characterized in that, The system includes: The data filtering module is configured to collect raw flight data from aviation data sources and filter the data based on predefined flight phases and key parameters, and then convert the filtered data into editable structured format data. The parsing module is configured to identify parameter types from structured format data and perform differential parsing, wherein the parameter types include analog parameters, digital parameters, and discrete mixed parameters; The abnormal parameter set acquisition module is configured to perform standardization processing on the parsed parameters, construct an isolated tree through recursive feature segmentation, calculate the abnormal score based on the average path length of the data points in the isolated tree, and mark the parameters whose abnormal scores exceed the threshold to form an abnormal parameter set. The parameter correlation network construction module is configured to input standardized flight data into an autoencoder to extract primary feature vectors, and then input them into a variational autoencoder after dimensional compression to generate a latent space representation of the nonlinear relationship between encoding parameters. The nonlinear correlation between parameters is quantified by calculating the mutual information between different dimensions to construct the parameter correlation network. The report generation module is configured to input compressed features into a pre-constructed hierarchical convolutional neural network, identify flight phases and key parameter categories layer by layer through a multi-level classifier, remove redundant parameters based on probability distribution similarity, and generate an analysis report containing abnormal parameter identifiers, time points, and deviation values ​​by combining the classification results of the hierarchical convolutional neural network, parameter correlation network, and abnormal parameter set.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor to implement the automated data analysis method for aviation accident investigation as described in any one of claims 1-7.

Images