A method for constructing a causal graph for fault analysis in the aviation field

Abstract

The present invention relates to a method for constructing a fault analysis graph in the aviation field, comprising the following steps: Step 1: extracting aviation events based on entity labeling; Step 2: extracting aviation event relationships based on ensemble learning; and Step 3: generating abstract aviation events based on a copy network to construct a fault analysis graph. The present invention utilizes a technique for introducing tag bit features from entity information, employs multiple optimization techniques including soft weight regularization and a stable weight decay method, extracts event relationships based on ensemble learning, and generates abstract events based on a copy network, combining them with aviation scenarios to construct a complete fault analysis graph. The method effectively improves event extraction accuracy, combines the latest optimizer technology, optimizes the event extraction module, and integrates multiple features to achieve better prediction performance than a single algorithm.

Description

Technical Field

[0001] The present application relates to the technical field of aviation fault analysis, and in particular to a method for constructing a fault analysis causal graph in the aviation field. Background Art

[0002] Events are a core concept in human society, and human social activities are often driven by events. The evolutionary patterns and laws governing the temporal succession of events are highly valuable knowledge, and exploring this logic of events is crucial for understanding human behavior and the patterns of social development and change. Among the many types of human knowledge, logic of events—including the sequential and causal relationships between events—is a crucial and ubiquitous form of common sense.

[0003] In the aviation field, especially the aviation manufacturing industry, it has the characteristics of a long industrial chain, multiple structural levels, complex supporting facilities, and cross-regional R&D, manufacturing and services. As the digital transformation of the aviation manufacturing industry continues to advance, a large amount of multimodal, cross-domain, and strongly correlated data will be generated throughout the life cycle process, which contains rich logical knowledge of the aviation field.

[0004] At present, the aviation field lacks a unified data governance system as well as methods and tools for automated knowledge processing and intelligent application tailored to the characteristics of aviation data. As a result, historical experience, logical knowledge, etc. in massive data cannot be summarized and accumulated in a timely and effective manner, seriously affecting the demand for intelligent use of common data and general knowledge during the digital transformation of the aviation manufacturing industry, requiring automated extraction, integration, and association.

[0005] This invention automatically extracts logical knowledge from large-scale unstructured text data and organizes this knowledge into a directed cyclic graph structure to describe the evolutionary laws and patterns between events. This knowledge base is called a "logical graph." The invention proposes the application of multiple optimization techniques, such as soft weight regularization and stable weight decay methods, event relationship extraction based on ensemble learning, and abstract event generation technology based on copy networks. Combined with aviation scenarios, this method constructs a complete logical graph for fault analysis. Summary of the Invention

[0006] In order to overcome the shortcomings of the existing technology, the present invention introduces the tag bit feature of entity information, applies multiple optimization technologies such as soft weight regularization and stable weight decay method, event relationship extraction based on ensemble learning and abstract event generation technology based on copy network, and combines it with aviation field scenarios to construct a fault analysis cause and effect graph; it can effectively improve the accuracy of event extraction, combine with the latest optimizer technology, optimize the event extraction module, and integrate multiple features to obtain better prediction performance than a single algorithm.

[0007] To achieve the above objectives, the solution adopted by the present invention is to provide a method for constructing a fault analysis graph in the aviation field, which includes the following steps:

[0008] Step 1: Aviation domain event extraction based on entity labeling;

[0009] Based on the first BERT model, we use entity tagging to extract aviation events. First, we define and identify aviation events. Then, we perform trigger word recognition. Finally, we add the entity tagging to the string sequence and trigger word information in the aviation event as input, which is used as the input for event argument prediction to obtain aviation events.

[0010] Step 2: Extract aviation event relations based on ensemble learning;

[0011] Obtain the aviation events in step 1, combine event relationship extraction based on text classification with event relationship extraction fused with sequence annotation features, and establish associations for aviation event pairs with causal and sequential relationships in the aviation events;

[0012] Step 21: Extract aviation event features based on text classification;

[0013] First, we transform event relationship extraction into a classification problem by determining whether pairs of aviation events in a sentence have a specified relationship. This involves concatenating the text and events, separating the original text from the event text using the CLS and SEP tags built into the second BERT model, and then feeding them into the second BERT model for feature extraction.

[0014] Step 22: Convert the result of aviation field event extraction in step 1 into sequence labels. Based on the event arguments, obtain the label of the first event as Evt1 and the label of the second event as Evt2. Then, combine the sequence labels with the CLS tags of the text classification in step 21.

[0015] Step 23: Extract aviation event relations based on fused sequence annotation features;

[0016] An improved AdamW optimizer is constructed by combining soft weight regularization and stable weight decay. The first BERT model is used for the second round of prediction, and the features at the CLS marker are used to obtain the marker classification of the event relationship.

[0017] The soft weight regularization attenuation model is as follows:

[0018] d AdamW = -ηλθ;

[0019] Where: d AdamWRepresents the weight decay of the AdamW optimizer; η represents the learning rate; λ represents the parameter for scaling weight decay; θ represents the parameter to be optimized;

[0020] The stable weight decay method is specifically to use the gradient accumulation parameter v in the AdamW optimizer t , perform sliding average on the gradient and the square of the gradient so that each update is related to the historical value of the gradient, as shown below:

[0021]

[0022] Where: v t represents the gradient accumulation parameter at time t; β2 represents the sliding average optimization coefficient; v t-1 represents the gradient accumulation parameter at time t-1; g t represents the gradient of the stable weight decay method at time t;

[0023] Step 3: Generate abstract aviation domain events based on the copy network and construct a fault analysis event graph;

[0024] Obtain the aviation event relationship extracted in step 2, use the generative model combined with the copy network, and select the generative mode and copy mode to enable the model to copy the required entity fragments from the aviation event relationship in the input as output; in the encoder part, the entity fragments pass through the bidirectional recurrent neural network LSTM to obtain the encoding state h t The decoder is a bidirectional recurrent neural network LSTM, which inputs the reference summary words in the entity segment encoding state in sequence and obtains the reference summary word state s of the current time step at time t t , and finally output the event graph; specifically:

[0025] First, determine the attention score of the input position of the abstract aviation field event; construct the attention score vector of the input position and determine the weight vector obtained after normalization; at time t, calculate the aviation field event relationship vocabulary probability based on the weighted sum of the hidden states of the input sequence, as shown below:

[0026]

[0027] Where: P vocab represents the probability of aviation event words; p represents the probability function of aviation event words; softmax represents the normalization function; V′ represents the first word probability matrix; V represents the second word probability matrix; s t Indicates the state of the current time step at time t; represents the aviation domain context vector; b represents the first word probability bias vector; b′ represents the second word probability bias vector; ω represents the aviation domain event word probability coefficient;

[0028] Then, in the encoder part, increase the generation probability p gen The method to obtain is as follows:

[0029]

[0030] Where: p gen Indicates an increase in the probability of generation; σ indicates an increase in the probability of generation exponent; represents the weight coefficient of aviation domain context vector; The weight coefficient representing the state; Represents the weight coefficient of the input vector; x t represents the input vector at the tth time step; b ptr Indicates the increase in generation probability bias vector;

[0031] Finally, by increasing the generation probability p gen The next reference summary word is determined by copying it from the original text or generating it through a sequence-to-sequence model (seq2seq), thereby constructing a fault analysis causal graph for the aviation field.

[0032] Preferably, the first BERT model, the second BERT model, the improved AdamW optimizer, the bidirectional recurrent neural network LSTM and the encoding and decoding structure seq2seq are specifically:

[0033] The first BERT model and the second BERT model are respectively built based on the Transformer Encoder, and are pre-trained language models obtained by training multiple unlabeled texts in an unsupervised manner;

[0034] The improved AdamW optimizer can use the momentum information of past gradients to determine the size of the learning rate in each direction and automatically adjust the learning rate. It is an improved algorithm based on Adam+L2 regularization.

[0035] The bidirectional recurrent neural network LSTM is a time recurrent neural network suitable for processing and predicting important events with long intervals and delays in time series;

[0036] The encoding and decoding structure seq2seq belongs to the encoder-decoder structure. The encoder is responsible for compressing the input sequence into a vector of a specified length. The vector of the specified length is the semantics of this sequence, which is called the encoding process; the decoder is responsible for generating a specified sequence based on the semantic vector, which is called the decoding process.

[0037] Preferably, the soft weight regularization attenuation model in step 23 needs to subtract the weight attenuation term from the parameters, as shown below:

[0038]

[0039] Where: Represents the weight attenuation coefficient calculated by NormLoss; ||θ co || represents the Euclidean norm of the weight matrix;

[0040] Set the weight matrix ‖co‖, which can be associated with a linear layer or a convolutional layer. NormLoss calculates the Euclidean norm of the weight matrix so that the weight is eventually updated to a unit norm.

[0041] Preferably, the stable weight decay method in step 23 changes during the actual step size iteration process, and when When it is close to zero, the weight decay calibrated for the first iteration of training is too large. Use a stable weight decay method to replace The current network layer The covariance of the means is the mean, as follows:

[0042]

[0043] Where: d stable Represents the weight decay term after optimization using the stable weight decay method.

[0044] Preferably, the attention score of the input position of the abstract aviation field event is determined in step 3, specifically:

[0045]

[0046] Where: represents the attention score for the i-th input position at the t-th time step; v T represents the attention score vector; tanh represents the hyperbolic tangent function; W h represents the first weight matrix; h i represents the hidden state of the i-th position in the input sequence; W s represents the second weight matrix; s t represents the state of the tth time step; b attn Represents the attention score bias vector.

[0047] Preferably, in step 3, the attention score vector of the input position is constructed, and the weight vector obtained after normalization is determined, specifically:

[0048] a t =softmax(e t );

[0049] Where: at represents the weight vector obtained after normalization at the t-th time step; e t A vector of attention scores representing the input position at the t-th time step.

[0050] Preferably, the weighted sum of the hidden states of the input sequence in step 3 is specifically:

[0051]

[0052] Where: Represents the normalized weight vector for the ith time step.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] (1) The present invention introduces the tag bit feature of entity information, applies multiple optimization technologies such as soft weight regularization and stable weight decay method, extracts event relations based on ensemble learning, and generates abstract events based on copy network, and combines it with aviation field scenarios to construct a complete fault analysis graph.

[0055] (2) The present invention uses entity tag information as event extraction features, which can effectively improve the accuracy of event extraction. It combines the latest optimizer technology to optimize the event extraction module. For event relationship extraction, it integrates multiple features to obtain better prediction performance than a single algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a control block diagram of a method for constructing a fault analysis graph in the aviation field according to an embodiment of the present invention;

[0057] Figure 2 This is a diagram of the event extraction model architecture based on entity tagging according to an embodiment of the present invention;

[0058] Figure 3 This is an event relationship extraction graph based on feature integration according to an embodiment of the present invention;

[0059] Figure 4 A diagram of events and principles constructed for an embodiment of the present invention;

[0060] Figure 5 This is an overall event diagram of multiple groups of events in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0062] The embodiment of the present invention is based on aviation fault data, introduces marker bit features based on entity information, applies multiple optimization techniques such as soft weight regularization and stable weight decay methods, extracts event relationships based on ensemble learning, and generates abstract events based on copy networks. Combined with aviation scenarios, it constructs a complete fault analysis graph, effectively improving the accuracy of event extraction. In combination with the latest optimizer technology, it optimizes the event extraction module and integrates multiple features for event relationship extraction to achieve better prediction performance than a single algorithm. Figure 1 The figure shows a control block diagram of a method for constructing a fault analysis graph in the aviation field according to an embodiment of the present invention. The analysis of the embodiment proves that the method has a good application effect.

[0063] The present invention provides a method for constructing a fault analysis graph in the aviation field. To demonstrate the applicability of the present invention, the method is applied to an example. The method specifically includes the following steps:

[0064] S1: Aviation event extraction based on entity labeling;

[0065] Based on the first BERT model, we use entity tagging to extract aviation events. First, we define and identify aviation events. Then, we perform trigger word recognition. Finally, we add the entity tagging to the string sequence and trigger word information in the aviation event as input, which is used as the input for event argument prediction to obtain aviation events.

[0066] The first BERT model and the second BERT model are respectively built based on Transformer Encoder, and use pre-trained language models obtained by training multiple unlabeled texts in an unsupervised manner.

[0067] S2: Extracting aviation event relations based on ensemble learning;

[0068] Obtain aviation events in S1, combine event relationship extraction based on text classification and event relationship extraction fused with sequence annotation features, and establish associations for aviation event pairs with causal and sequential relationships in aviation events; Figure 3 Shown is an event relationship extraction diagram based on feature integration according to an embodiment of the present invention.

[0069] S21: Extracting aviation event features based on text classification;

[0070] First, by determining whether pairs of aviation events in a sentence have a specified relationship, event relationship extraction is transformed into a classification problem. This involves concatenating the text and events, separating the original text and event text using the CLS and SEP tags built into the second BERT model, and then inputting them into the second BERT model for feature extraction.

[0071] S22: Convert the results of aviation field event extraction in S1 into sequence labels. According to the event arguments, the label of the first event is Evt1 and the label of the second event is Evt2. Then, the sequence labels are combined with the CLS tags of text classification in S21.

[0072] S23: Extraction of aviation event relations based on fusion of sequence annotation features;

[0073] An improved AdamW optimizer is constructed by combining weight soft regularization and stable weight decay method. The improved AdamW optimizer can use the momentum information of past gradients to determine the size of the learning rate in each direction and automatically adjust the learning rate. It is an improved algorithm based on Adam+L2 regularization.

[0074] The first BERT model is used for the second round of prediction, and the features at the CLS token are used to obtain the token classification of the event relationship.

[0075] The soft weight regularization decay model is as follows:

[0076] d AdamW = -ηλθ;

[0077] Where: d AdamW Represents the weight decay of the AdamW optimizer; η represents the learning rate; λ represents the parameter for scaling weight decay; θ represents the parameter to be optimized.

[0078] The soft weight regularization decay model requires subtracting the weight decay term from the parameters as follows:

[0079]

[0080] Where: Represents the weight attenuation coefficient calculated by NormLoss; ||θ co || represents the Euclidean norm of the weight matrix.

[0081] Set the weight matrix ‖co‖, which can be associated with a linear layer or a convolutional layer. NormLoss calculates the Euclidean norm of the weight matrix so that the weight is eventually updated to a unit norm.

[0082] The stable weight decay method is specifically to use the gradient accumulation parameter v in the AdamW optimizer t , perform sliding average on the gradient and the square of the gradient so that each update is related to the historical value of the gradient, as shown below:

[0083]

[0084] Where: vt represents the gradient accumulation parameter at time t; β2 represents the sliding average optimization coefficient; v t-1 represents the gradient accumulation parameter at time t-1; g t represents the gradient of the stable weight decay method at time t.

[0085] The stable weight decay method changes during the actual step size iteration, and when When it is close to zero, the weight decay calibrated for the first iteration of training is too large. Use a stable weight decay method to replace The current network layer The covariance of the means is the mean, as follows:

[0086]

[0087] Where: d stable Represents the weight decay term after optimization using the stable weight decay method.

[0088] S3: Generate abstract aviation events based on the copy network and construct a fault analysis event graph;

[0089] Obtain the aviation event relationship extracted from S2, use the generative model combined with the copy network, and select the generative mode and copy mode to enable the model to copy the required entity fragments from the aviation event relationship in the input as output; in the encoder part, the entity fragments pass through the bidirectional recurrent neural network LSTM to obtain the encoding state h t ; Bidirectional recurrent neural network LSTM is a time recurrent neural network that is suitable for processing and predicting important events with long intervals and delays in time series.

[0090] The decoder is a bidirectional recurrent neural network LSTM, which inputs the reference summary words in the entity segment encoding state in sequence and obtains the reference summary word state s of the current time step at time t t , and finally output the event graph; specifically:

[0091] First, the attention score of the input position of the abstract aviation domain event is determined, specifically:

[0092]

[0093] Where: represents the attention score for the i-th input position at the t-th time step; v T represents the attention score vector; tanh represents the hyperbolic tangent function; W h represents the first weight matrix; h i represents the hidden state of the i-th position in the input sequence; W s represents the second weight matrix; st represents the state of the tth time step; b attn Represents the attention score bias vector.

[0094] Construct the attention score vector of the input position and determine the normalized weight vector, specifically:

[0095] a t =softmax(e t );

[0096] Where: a t represents the weight vector obtained after normalization at the t-th time step; e t A vector of attention scores representing the input position at the t-th time step.

[0097] At time t, the weighted sum of the hidden states of the input sequence is:

[0098]

[0099] Where: Represents the normalized weight vector for the ith time step.

[0100] Furthermore, the probability of event relationship words in the aviation field is calculated as follows:

[0101]

[0102] Where: P vocab represents the probability of aviation event words; p represents the probability function of aviation event words; softmax represents the normalization function; V′ represents the first word probability matrix; V represents the second word probability matrix; s t Indicates the state of the current time step at time t; represents the aviation domain context vector; b represents the first word probability bias vector; b′ represents the second word probability bias vector; ω represents the aviation domain event word probability coefficient.

[0103] Then, in the encoder part, increase the generation probability p gen The method to obtain is as follows:

[0104]

[0105] Where: p gen Indicates an increase in the probability of generation; σ indicates an increase in the probability of generation exponent; represents the weight coefficient of aviation domain context vector; The weight coefficient representing the state; Represents the weight coefficient of the input vector; x trepresents the input vector at the tth time step; b ptr Represents the bias vector that increases the generation probability.

[0106] Finally, by increasing the generation probability p gen The next reference summary word is determined by copying it from the original text or generating it through a sequence-to-sequence model (seq2seq), thereby constructing a fault analysis causal graph for the aviation field.

[0107] The encoding and decoding structure seq2seq belongs to the encoder-decoder structure. The encoder is responsible for compressing the input sequence into a vector of a specified length. The vector of the specified length is the semantics of this sequence, which is called the encoding process; the decoder is responsible for generating the specified sequence based on the semantic vector, which is called the decoding process.

[0108] Increase the generation probability p gen The next word is determined by copying it from the original text or generating it through the encoding and decoding structure seq2seq. This method can significantly improve the effect of abstract event generation and ultimately obtain an accurate fault analysis diagram in the aviation field.

[0109] The encoding and decoding structure seq2seq belongs to the encoder-decoder structure. The encoder is responsible for compressing the input sequence into a vector of a specified length. This vector can be regarded as the semantics of the sequence, which is called the encoding process; the decoder is responsible for generating the specified sequence based on the semantic vector, which is called the decoding process.

[0110] In this embodiment of the present invention, the input "ABX aircraft fire, causing the oxygen tube to ignite and burn through" is used as an example. The system extracts the events: "ABX aircraft fire", "oxygen tube ignition", "oxygen tube burn through"; the system extracts the event relationships "ABX aircraft fire" -> "oxygen tube ignition", "aircraft fire" -> "oxygen tube burn through". The system generates abstract events: "aircraft fire", "oxygen tube ignition", "oxygen tube burn through"; the system constructs a graph, and obtains a causal graph as follows: Figure 4 As shown in the figure, the system stores multiple sets of event relationships in the graph, and obtains the overall event graph of multiple sets of events through query, such as Figure 5 shown.

[0111] In summary, the prediction results of the method for constructing a causal graph for fault analysis in the aviation field in this embodiment have proven to be effective.

[0112] (1) The embodiment of the present invention is based on the introduction technology of the tag bit feature of entity information, applies multiple optimization technologies such as soft weight regularization and stable weight decay method, event relationship extraction based on ensemble learning and abstract event generation technology based on copy network, and combines it with the aviation field scenario to construct a complete fault analysis cause and effect diagram.

[0113] (2) The embodiment of the present invention uses entity tag information as event extraction features, which can effectively improve the accuracy of event extraction. Combined with the latest optimizer technology, the event extraction module is optimized. For event relationship extraction, multiple features are integrated to obtain better prediction performance than a single algorithm. The analysis of the embodiment can prove that this method has a good application effect.

[0114] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for constructing a fault analysis graph in the aviation field, characterized in that: It includes the following steps: Step 1: Aviation domain event extraction based on entity labeling; Based on the first BERT model, trigger word recognition and event extraction in the aviation field are performed using entity tagging. First, aviation field events are defined and identified. Then, trigger word recognition is performed. Finally, the entity recognition tag is added to the string sequence and trigger word information in the aviation field event as input, which is used as the input for event argument prediction to obtain the aviation field event. Step 2: Extract aviation event relations based on ensemble learning; Obtain the aviation events in step 1, combine event relationship extraction based on text classification with event relationship extraction fused with sequence annotation features, and establish associations for aviation event pairs with causal and sequential relationships in the aviation events; Step 21: Extract aviation event features based on text classification; First, we transform event relationship extraction into a classification problem by determining whether the aviation event pairs in the sentence have a specified relationship. This includes concatenating the text and the event, separating the original text and the event text using the CLS and SEP tags built into the second BERT model, and then inputting them into the second BERT model for feature extraction. Step 22: Convert the result of aviation field event extraction in step 1 into sequence labels. Based on the event arguments, obtain the label of the first event as Evt1 and the label of the second event as Evt2. Then, combine the sequence labels with the CLS tags of the text classification in step 21. Step 23: Extract aviation event relations based on fused sequence annotation features; An improved AdamW optimizer is constructed by combining weight soft regularization and stable weight decay method; Use the first BERT model for the second round of prediction and use the features at the CLS tag to obtain the tag classification of the event relationship; The weight soft regularization attenuation model is as follows: d AdamW =-elth; Where: d AdamW Represents the weight decay of the AdamW optimizer; η represents the learning rate; λ represents the parameter for scaling weight decay; θ represents the parameter to be optimized; The stable weight decay method is specifically to use the gradient accumulation parameter v in the AdamW optimizer t , perform sliding average on the gradient and the square of the gradient so that each update is related to the historical value of the gradient, as shown below: Where: v t represents the gradient accumulation parameter at time t; β2 represents the sliding average optimization coefficient; v t-1 represents the gradient accumulation parameter at time t-1; g t represents the gradient of the stable weight decay method at time t; Step 3: Generate abstract aviation domain events based on the copy network and construct a fault analysis event graph; Obtain the aviation event relationship extracted in step 2, use the generative model combined with the copy network, and select the generative mode and copy mode to enable the model to copy the required entity fragments from the aviation event relationship in the input as output; in the encoder part, the entity fragments pass through the bidirectional recurrent neural network LSTM to obtain the encoding state h t ; The decoder is a bidirectional recurrent neural network LSTM, which inputs the reference summary words in the entity segment encoding state in sequence and obtains the reference summary word state s of the current time step at time t t , and finally output the event graph; specifically: First, determine the attention score of the input position of the abstract aviation field event; construct the attention score vector of the input position and determine the weight vector obtained after normalization; at time t, calculate the aviation field event relationship vocabulary probability based on the weighted sum of the hidden states of the input sequence, as shown below: Where: P vocab represents the probability of aviation event words; p represents the probability function of aviation event words; softmax represents the normalization function; V′ represents the first word probability matrix; V represents the second word probability matrix; s t Indicates the state of the current time step at time t; represents the aviation domain context vector; b represents the first word probability bias vector; b′ represents the second word probability bias vector; ω represents the aviation domain event word probability coefficient; Then, in the encoder part, increase the generation probability p gen The method to obtain is as follows: Where: p gen Indicates an increase in the probability of generation; σ indicates an increase in the probability of generation exponent; represents the weight coefficient of aviation domain context vector; The weight coefficient representing the state; Represents the weight coefficient of the input vector; x t represents the input vector at the tth time step; b ptr Indicates the increase in generation probability bias vector; Finally, by increasing the generation probability p gen The next reference summary word is determined by copying it from the original text or generating it through a sequence-to-sequence model (seq2seq), thereby constructing a fault analysis causal graph for the aviation field.

2. The method for constructing a fault analysis graph in the aviation field according to claim 1, characterized in that: The first BERT model, the second BERT model, the improved AdamW optimizer, the bidirectional recurrent neural network LSTM, and the encoding and decoding structure seq2seq are specifically: The first BERT model and the second BERT model are respectively built based on the Transformer Encoder, and are pre-trained language models obtained by training multiple unlabeled texts in an unsupervised manner; The improved AdamW optimizer can use the momentum information of past gradients to determine the size of the learning rate in each direction and automatically adjust the learning rate. It is an improved algorithm based on Adam+L2 regularization. The bidirectional recurrent neural network LSTM is a time recurrent neural network suitable for processing and predicting important events with long intervals and delays in time series; The encoding and decoding structure seq2seq belongs to the encoder-decoder structure. The encoder is responsible for compressing the input sequence into a vector of a specified length. The vector of the specified length is the semantics of this sequence, which is called the encoding process. The decoder is responsible for generating the specified sequence based on the semantic vector, which is called the decoding process.

3. The method for constructing a fault analysis graph in the aviation field according to claim 1, characterized in that: The weight soft regularization attenuation model in step 23 needs to subtract the weight attenuation term from the parameters, as shown below: Where: Represents the weight attenuation coefficient calculated by NormLoss; ||θ co || represents the Euclidean norm of the weight matrix; Set the weight matrix ‖co‖, which can be associated with a linear layer or a convolutional layer. NormLoss calculates the Euclidean norm of the weight matrix so that the weight is eventually updated to a unit norm.

4. The method for constructing a fault analysis graph in the aviation field according to claim 1, characterized in that: The stable weight decay method in step 23 changes during the actual step size iteration process, and when When it is close to zero, the weight decay calibrated for the first iteration of training is too large. Use a stable weight decay method to replace The current network layer The covariance of the means is the mean, as follows: Where: d stable Represents the weight decay term after optimization using the stable weight decay method.

5. The method for constructing a fault analysis graph in the aviation field according to claim 1, characterized in that: The attention score of the input position of the abstract aviation field event is determined in step 3, specifically: Where: represents the attention score for the i-th input position at the t-th time step; v T represents the attention score vector; tanh represents the hyperbolic tangent function; W h represents the first weight matrix; h i represents the hidden state of the i-th position in the input sequence; W s represents the second weight matrix; s t represents the state of the tth time step; b attn Represents the attention score bias vector.

6. The method for constructing a fault analysis graph in the aviation field according to claim 1, characterized in that: In step 3, the attention score vector of the input position is constructed, and the weight vector obtained after normalization is determined, specifically: the t =softmax(e t ); Where: a t represents the weight vector obtained after normalization at the t-th time step; e t A vector of attention scores representing the input position at the t-th time step.

7. The method for constructing a fault analysis graph in the aviation field according to claim 1, characterized in that: The weighted sum of the hidden states of the input sequence in step 3 is specifically: Where: Represents the normalized weight vector for the ith time step.

Images